Hydraulic turbine top cover in-situ robot polishing and shaping processing method based on 3D vision

Abstract

A kind of water turbine top cover in situ robot polishing and shaping processing method based on 3D vision, comprising the following steps: S1. arranging mark points to water turbine top cover, obtaining three-dimensional model of water turbine top cover and three-dimensional coordinate information of global mark points, resampling obtains the global point cloud data of water turbine top cover;S2. measure the repair area of repair welding, realize the matching of local shaping area and global model of top cover;S3. calculate the distance using local point cloud data and top cover global point cloud data, extract the protruding area point cloud data after repair welding repair;S4. extract the boundary of protruding area point cloud, segment the protruding point cloud data;S5. use the minimum directed bounding box to approximate describe protruding shaping area, obtain target shaping surface using bounding box, solve polishing track using geometric allowance information, send to robot, carry out polishing work, and then realize water turbine top cover in situ robot polishing and shaping processing automation.

Description

Technical Field

[0001] This invention relates to the field of in-situ robotic grinding and shaping technology, and in particular to an in-situ robotic grinding and shaping method for turbine top covers based on 3D vision. Background Technology

[0002] Hydropower turbines are crucial equipment for converting water energy into electrical energy. However, the turbine top cover is subject to varying degrees of cavitation and abrasion during operation, severely impacting the turbine unit's power generation efficiency. Therefore, regular maintenance of the turbine top cover is necessary. Currently, the common maintenance method involves cleaning the abrasion defects, welding, and then manually grinding them. However, due to the large size of the turbine runner, manual grinding is labor-intensive, making it difficult to control repair precision and potentially accelerating the abrasion rate, disrupting the runner's balance, and reducing the unit's power generation efficiency. Therefore, researching automated grinding and repair methods for turbine top covers is of significant importance.

[0003] To achieve automated grinding, the first step is to determine the location of the grinding and shaping area within the overall structure of the turbine top cover. However, the turbine top cover is an irregular curved surface with minimal surface curvature variation and indistinct features, making it difficult to match and locate using the workpiece's own characteristics. While manually designing markers and employing 3D target recognition methods can achieve matching, these markers are difficult to fix on the turbine top cover surface. Secondly, determining the machining allowance is crucial; traditional manual measurement is time-consuming, labor-intensive, and lacks accuracy. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the problems existing in the background technology mentioned above, and to provide a method for in-situ robotic grinding and shaping of turbine top covers based on 3D vision. The method uses a handheld scanner to acquire measurement data, uses photogrammetric markers to locate the grinding area, calculates the machining allowance based on the grinding area and the global area after matching, and realizes the segmentation and extraction of the protruding parts of the grinding area. The grinding trajectory is accurately calculated for the protrusions, and the allowance information and trajectory are sent to the robot, thereby realizing the automation of in-situ robotic grinding and shaping of turbine top covers.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for in-situ robotic grinding and shaping of a turbine top cover based on 3D vision, comprising the following steps:

[0006] S1. Marker points are set up on the turbine top cover, and the three-dimensional coordinate information of the turbine top cover and the global marker points are obtained by using a scanner. The global point cloud data of the turbine top cover is obtained by resampling.

[0007] S2. Measure the repaired area, identify and extract local marker points, and match the local repair area with the global model of the top cover based on the coordinate information of the local and global marker points.

[0008] S3. Calculate the distance using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Extract the point cloud data of the protruding area after the welding repair by using the remaining threshold.

[0009] S4. Extract the point cloud boundary of the convex region, perform spline curve fitting on the boundary data, and segment the convex point cloud data according to the curve curvature change;

[0010] S5. For each data segment, combined with the margin information, the minimum directed bounding box is used to approximate the protrusion shaping area, the target shaping surface is obtained using the bounding box, the grinding trajectory is solved using the geometric margin information, and sent to the robot for grinding work.

[0011] In S1, a number of circular photogrammetric markers are arranged on the turbine top cover. A handheld scanner is used to measure and obtain the three-dimensional coordinate information of the turbine top cover 3D model and global markers. Based on the turbine top cover 3D model, resampling is performed. The global point cloud data of the turbine top cover is obtained by setting the total number of points after sampling and the distance between two points after sampling.

[0012] In S2, a 3D measuring head is installed at the end of the robot arm. After hand-eye calibration, the area to be repaired is measured to obtain the image and point cloud data of the area to be repaired. Then, using image methods, the coordinate information of the local area marker points is identified and extracted, and matched with the global marker points to achieve the matching of the local repair area with the global point cloud data of the top cover, thereby unifying the data under the robot coordinate system.

[0013] Using image-based methods, the coordinate information of local landmark points within a region is identified and extracted, and then matched with global landmark points to achieve matching between the local reshaping area and the global point cloud data of the roof. This process includes the following steps:

[0014] S21: After the surface of the turbine top cover is repaired by welding, the three-dimensional measuring head is installed at the end of the robot. After hand-eye calibration, the area to be repaired is measured. At least three marker points are required in the area. Color images, depth images and point cloud data are obtained. The center of the ring in the color image is extracted using an ellipse fitting algorithm.

[0015] S22. The extracted center of the circular marker is the pixel coordinate. Combining the imaging principle, depth map, and camera intrinsic parameter information, the three-dimensional coordinate information (X,Y,Z) corresponding to the center (x,y) of the two-dimensional ellipse is calculated according to the formula below:

[0016] ;

[0017] In the formula, the camera's intrinsic parameters are: ,in This represents the length of the focal length along the x-axis. This represents the length of the focal length along the y-axis. ) represents the pixel coordinates of the principal point, the coordinates of the ellipse center are (x,y), and its corresponding 3D coordinates are (X,Y,Z). D represents the pixel value at the (x,y) position in the depth map. The depth camera used in this case has pixel values ​​in millimeters and Z coordinates in meters. Therefore, the value of D needs to be divided by 1000 to convert it to meters.

[0018] S23. After obtaining the three-dimensional coordinate information of the local region marker points, match the local region marker points with the global marker points. The set of local marker points is U1, and the set of global marker points is U2. For each point in U1, a corresponding point can be found in U2. At the same time, distance and angle are introduced as constraints to convert point matching into edge matching.

[0019] S24: After matching, register the local modification area with the global point cloud data of the turbine top cover, and then use the results of hand-eye calibration. The measurement data of the local shaping area and the global point cloud data are converted into robot coordinates for subsequent robot operations;

[0020] In S23, the specific steps of the matching algorithm are as follows:

[0021] S2301: Starting from any point a in U1, find any point b, and construct a vector to obtain... Then find any side ac with the same starting point, and form a vector to obtain... ;

[0022] S2302: Calculate the magnitudes of the vectors along sides ab and ac. , :

[0023] ,

[0024] ;

[0025] vector , Normalization yields:

[0026] ;

[0027] ;

[0028] calculate , dot product of vectors External product :

[0029] ,

[0030] ;

[0031] S2303: Find any side de in U2, and construct a vector to obtain... Calculate the magnitude of the edge vector de. :

[0032] ;

[0033] For vectors Normalization yields:

[0034] ;

[0035] calculate , inner product :

[0036] ;

[0037] S2304: Starting from the origin d of edge de, find any edge df in U2, and construct a vector to obtain... Calculate the magnitude of the edge df vector. ;

[0038] ;

[0039] For vectors Normalization yields:

[0040] ;

[0041] calculate , inner product External product :

[0042] ,

[0043] ;

[0044] S2305: Determine if points a, b, c correspond one-to-one with points d, e, f, while introducing distance and angle as constraints. Compare the triangles formed by a, b, c and d, e, f in space. Due to the random placement of marker points, multiple approximately equal triangles may appear in space, leading to incorrect point pair matching. Therefore, an enhanced condition is introduced to calculate... , inner product :

[0045] The necessary and sufficient condition for points a, b, c and points d, e, f to form a point pair is:

[0046] ;

[0047] Due to measurement errors, a distance threshold exists when determining whether side lengths are equal. When determining whether the included angles on both sides are equal, there exists an angle threshold. If the above conditions are met, proceed to the next step; otherwise, execute S2303-S2305 until the corresponding points d, e, and f of a, b, and c are found.

[0048] S2306: All edges in U1 that start from 'a' are used to find the corresponding edges in U2 through S2301-S2305, and the corresponding points are stored in array V;

[0049] S2307: In U1, edges are formed starting from non-a. Any two edges bc and bd are selected to find the corresponding edges.

[0050] S2308: Repeat steps S2301-S2305. The corresponding edges de and df found in U2, and the points d, e, and f should already exist in array V. If this condition is met, proceed to the next step; otherwise, repeat operations S2301-S2305.

[0051] S2309: Repeat steps S2307-S2308 until both sides of U1 starting from all points have corresponding edges in U2;

[0052] S2310: Using the found corresponding edges, construct corresponding points. Use singular value decomposition to solve for the rotation matrix between the two point sets. Solving the rotation matrix requires at least three pairs of points. Therefore, when measuring the local reshaping area, it is essential to ensure at least three marker points within the survey area. Considering measurement errors, arbitrarily select three points in U1 and their corresponding points in U2, solve for the rotation matrix R, and then solve for the translation matrix T through its centroid. All points in U1... After rotation and translation transformation, we obtain U0, and then convert the corresponding points in U2. Treating the true value as the actual value, calculate the mean squared error (MSE):

[0053] ;

[0054] S2311: For all three points of the triangles that can be formed in U1, use singular value decomposition to obtain the rotation matrix and calculate the MSE. Take the rotation matrix R and translation matrix T corresponding to the minimum MSE value as the final result.

[0055] In S3, distances are calculated using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Based on the remaining threshold, point cloud data of the protruding area after welding repair is extracted. The specific steps are as follows:

[0056] S31: Calculate the distance between corresponding points in the local point cloud data and the global point cloud data of the roof. Use eigenvalue decomposition to obtain the normal vector of each point in the global point cloud data of the roof. Take any point in the local shaping region as the search point. The K-nearest neighbor search algorithm is used to find the nearest point in the global point cloud data of the roof. , The normal vector information is :

[0057] Constructing vectors ;

[0058] arrive The distance is :

[0059] ;

[0060] This method is used to calculate the distance from each point in the local point cloud data to the corresponding point, and the distance information is stored in the intensity field;

[0061] S32. Calculate the distance from the local reshaping area to the global data of the top cover, create a histogram, and based on the histogram information, specify a distance threshold. Points with a distance less than a threshold are selected as target point clouds. Due to insufficient precision and measurement errors in the threshold setting, the extracted point cloud data of the convex region is filtered. Based on the traditional Euclidean spatial distance, a third-party library (nanoflann) is used to speed up the search for spatially nearby points and complete clustering. Based on the geometric characteristics of the convex region: it is a long and narrow strip, and the filtered convex region is found.

[0062] In S4, the method for segmenting convex point cloud data includes the following steps:

[0063] S41: Extract the boundary points of the raised region, calculate the normal vector of the raised region, and determine whether it is a boundary point based on the angle between the normal vectors;

[0064] S42: Project the boundary points onto the plane, take the two longer sides of the boundary and fit them with spline curves to obtain two curve equations;

[0065] S43: Along the direction of the raised region, find the corresponding points of the two curves. derivative at point , derivative at point Given a threshold , ;in, Indicates the same At a certain point, the threshold difference between the derivatives of the two curves is closer to 0, indicating that the two curves are at a certain point (where...). - ), The boundaries within the range are parallel, and the bounding box is used for precise approximation. This represents the threshold for the slope change of the curve; the slope change is within the threshold. If the data is within the same segment, it is considered as a single segment; this method divides the raised area into several segments of data.

[0066] In S5, the method of approximating the shaping protrusion area using the minimum directed bounding box and allowance information, and then using this to obtain the grinding trajectory, includes the following steps:

[0067] S51: For each protruding region, solve for its minimum directed bounding box, and use the bounding box to find the common area between the protruding region and the three-dimensional model of the turbine top cover, which is regarded as the target shaping surface.

[0068] S52: Unify the normal vector direction of the target surface to be shaped, solve for the contour of the target surface to be shaped, and sample the set of trajectory points based on the contour direction. ;

[0069] S53: Find the pose of the trajectory points; Constructing vectors ,Regulation x-axis direction The normal vector at is The z-axis direction is [missing information], and the y-axis direction is [missing information]. , The outer product is used to determine the pose of the trajectory point;

[0070] S53: Send the pose of the trajectory points and the corresponding margin information to the robot for polishing.

[0071] The present invention has the following beneficial effects:

[0072] 1. Local positioning of curved workpieces based on photogrammetric markers can avoid mismatches similar to those caused by feature matching. Because curved workpieces have small curvature variations and indistinct features, feature-based matching is prone to failures and mismatches. Furthermore, it eliminates the need for manually designed markers; it directly utilizes the photogrammetric markers required by handheld scanners, avoiding secondary marker placement and reducing workload. It also improves matching efficiency, as the fewer coordinates of the markers, the more effectively key points in the 3D data are sampled, significantly enhancing the efficiency of the matching algorithm.

[0073] 2. Based on point cloud data, it can accurately calculate machining allowance, bounding box, trajectory and other information, reduce human measurement errors and improve the accuracy of grinding and shaping.

[0074] 3. Instead of grinding the entire area, the raised area after the weld repair is segmented and extracted for grinding, making the grinding more precise.

[0075] 4. It improves the automation level of grinding and shaping processes and reduces manpower. Attached Figure Description

[0076] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0077] Figure 1 This is a schematic diagram of the process of the present invention.

[0078] Figure 2 This is a schematic diagram of the color image data of the modification area of ​​the present invention.

[0079] Figure 3 This is a schematic diagram of a partial modification area of ​​the present invention.

[0080] Figure 4 This is a schematic diagram showing the matching of the local modification area and the global model of the top cover in this invention.

[0081] Figure 5 This is a schematic diagram of the point cloud data of the raised region in this invention.

[0082] Figure 6 This is a schematic diagram illustrating the extraction of point cloud data from the raised region according to the present invention.

[0083] Figure 7 This is a schematic diagram of the modified area after segmentation of the raised area extracted according to the present invention. Detailed Implementation

[0084] See Figure 1 A method for in-situ robotic grinding and shaping of a turbine top cover based on 3D vision includes the following steps:

[0085] S1. Marker points are set up on the turbine top cover, and the three-dimensional coordinate information of the turbine top cover and the global marker points are obtained by using a scanner. The global point cloud data of the turbine top cover is obtained by resampling.

[0086] S2. Measure the repaired area, identify and extract local marker points, and match the local repair area with the global model of the top cover based on the coordinate information of the local and global marker points.

[0087] S3. Calculate the distance using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Extract the point cloud data of the protruding area after the welding repair by using the remaining threshold.

[0088] S4. Extract the point cloud boundary of the convex region, perform spline curve fitting on the boundary data, and segment the convex point cloud data according to the curve curvature change and the tangential angle change not exceeding 30°;

[0089] S5. For each data segment, combined with the margin information, the minimum directed bounding box (OBB bounding box) is used to approximate the protrusion shaping area. The target shaping surface is obtained using the bounding box, and the grinding trajectory is solved using the geometric margin information. The trajectory is then sent to the robot for grinding.

[0090] In S1, a number of circular photogrammetric markers are arranged on the turbine top cover. A handheld scanner is used to measure and obtain the three-dimensional coordinate information of the turbine top cover 3D model and global markers. Based on the turbine top cover 3D model, resampling is performed. The global point cloud data of the turbine top cover is obtained by setting the total number of points after sampling and the distance between two points after sampling.

[0091] Specifically, in step S1, when arranging photogrammetric markers, random and uniform distribution should be ensured to avoid areas with fewer markers, which could affect subsequent matching. The 3D coordinate information of the turbine roof 3D model and global markers can be exported using the corresponding software of the handheld scanner. To facilitate subsequent calculations using the roof data, the roof model needs to be sampled to obtain the roof point cloud data. This can be done using CAD model sampling algorithms or open-source point cloud data processing software. As a specific implementation, CloudCompare can be used for model sampling. The obtained roof point cloud data may contain non-roof data. To reduce the amount of subsequent calculations, the region of interest can be extracted from the roof data to obtain clean roof point cloud data. As a specific example, manual cropping can be performed using CloudCompare.

[0092] After the surface of the turbine top cover is repaired by welding, the protruding parts need to be polished using a robot.

[0093] See Figure 3 , 4 In S2, a three-dimensional measuring head is installed at the end of the robot arm. After hand-eye calibration, the area to be repaired is measured to obtain the image and point cloud data of the area to be repaired. Using image methods, the coordinate information of local area marker points is identified and extracted, and matched with global marker points to achieve matching between the local repair area and the global point cloud data of the top cover, thereby unifying the data under the robot coordinate system.

[0094] Specifically, using image methods, the coordinate information of local marker points within the region is identified and extracted, and then matched with global marker points to achieve matching between the local reshaping area and the global point cloud data of the roof. This includes the following steps:

[0095] S21: As a specific embodiment, after the surface of the turbine top cover is repaired by welding, the Meka measuring head (three-dimensional measuring head) can be installed at the end of the robot. After hand-eye calibration, the area to be repaired is measured. It is required that there are at least three marker points in the area. Data such as color images, depth images and point clouds are obtained, and the center of the ring in the depth image is extracted using an ellipse fitting algorithm. Figure 2 In the middle, there is the color image data of the reshaping area.

[0096] S22. The extracted center of the circular marker is the pixel coordinate. Combining the imaging principle, depth map, and camera intrinsic parameter information, the three-dimensional coordinate information (X,Y,Z) corresponding to the center (x,y) of the two-dimensional ellipse is calculated according to the formula below:

[0097] ;

[0098] In the formula, the camera's intrinsic parameters are: ,in This represents the length of the focal length along the x-axis. This represents the length of the focal length along the y-axis. ) represents the pixel coordinates of the principal point, the coordinates of the ellipse center are (x,y), and its corresponding 3D coordinates are (X,Y,Z). D represents the pixel value at the (x,y) position in the depth map. The depth camera used in this case has pixel values ​​in millimeters, so the value of D needs to be divided by 1000 to convert it to meters.

[0099] S23. After obtaining the 3D coordinate information of the local region markers, match the local region markers with the global markers. The set of local markers is U1, and the set of global markers is U2. The matching idea is that for each point in U1, a corresponding point can be found in U2. To ensure a one-to-one correspondence, distance and angle are introduced as constraints, thus converting point matching into edge matching.

[0100] S24: After matching, register the local modification area with the global point cloud data of the turbine top cover, and then use the results of hand-eye calibration. This involves converting the measurement data of the local shaping area and the global point cloud data into robot coordinates to facilitate subsequent robot operations. Rotation matrix for hand-eye calibration. The translation matrix for hand-eye calibration.

[0101] In S23, the specific steps of the matching algorithm are as follows:

[0102] S2301: Starting from any point a in U1, find any point b, and construct a vector to obtain... Then find any side ac with the same starting point, and form a vector to obtain... .

[0103] S2302: Calculate the magnitudes of the vectors along sides ab and ac. , :

[0104] ,

[0105] ;

[0106] vector , Normalization yields:

[0107] ;

[0108] ;

[0109] calculate , dot product of vectors External product :

[0110] ,

[0111] ;

[0112] S2303: Find any side de in U2, and construct a vector to obtain... Calculate the magnitude of the edge vector de. :

[0113] ;

[0114] For vectors Normalization yields:

[0115] ;

[0116] calculate , inner product :

[0117] ;

[0118] S2304: Starting from the origin d of edge de, find any edge df in U2, and construct a vector to obtain... Calculate the magnitude of the edge df vector. ;

[0119] ;

[0120] For vectors Normalization yields:

[0121] ;

[0122] calculate , inner product External product :

[0123] ,

[0124] ;

[0125] S2305: Determine if points a, b, c correspond one-to-one with points d, e, f. This problem can be transformed into whether the triangles formed by a, b, c and d, e, f are approximately congruent in space. According to the axiom for triangle congruence: two sides and their included angle are equal, we know that we only need to determine whether the lengths of the two pairs of corresponding sides are equal and whether their corresponding included angles are equal.

[0126] Because the marker points are randomly placed, multiple approximately equal triangles may appear in space, leading to point pair matching errors. Considering that this task is for local point cloud localization on a curved surface, and both sets of points are obtained through a camera with the positive Z-axis of both sets of points pointing in the same direction, and that the local point cloud is captured directly facing the curved surface when using the measuring head, the angle between the positive Z-axis of the two coordinate systems will not exceed 90 degrees. Therefore, enhancement conditions are introduced to calculate... , inner product :

[0127] The necessary and sufficient condition for points a, b, c and points d, e, f to form a point pair is:

[0128] ;

[0129] Due to measurement errors, a distance threshold exists when determining whether side lengths are equal. When determining whether the included angles on both sides are equal, there exists an angle threshold. .

[0130] As a specific embodiment, set ;

[0131] If the above conditions are met, proceed to the next step; otherwise, execute S2303-S2305 until the corresponding points d, e, and f of a, b, and c are found.

[0132] S2306: All edges in U1 that start from 'a' are used to find the corresponding edges in U2 through S2301-S2305, and the corresponding points are stored in array V.

[0133] S2307: In U1, edges are formed starting from non-a. Two edges bc and bd are arbitrarily selected to find corresponding edges.

[0134] S2308: Repeat steps S2301-S2305. The corresponding edges de and df found in U2, and the points d, e, and f should already exist in array V. If this condition is met, proceed to the next step; otherwise, repeat operations S2301-S2305.

[0135] S2309: Repeat steps S2307-S2308 until both sides of U1 starting from all points have corresponding edges in U2.

[0136] S2310: Using the found corresponding edges, construct corresponding points. Use singular value decomposition to solve for the rotation matrix between the two point sets. Solving the rotation matrix requires at least three pairs of points. Therefore, when measuring the local reshaping area, it is essential to ensure at least three marker points within the survey area. Considering measurement errors, arbitrarily select three points in U1 and their corresponding points in U2, solve for the rotation matrix R, and then solve for the translation matrix T through its centroid. All points in U1... After rotation and translation transformation, we obtain U0, and then convert the corresponding points in U2. Treating the true value as the actual value, calculate the mean squared error (MSE):

[0137] ;

[0138] S2311: For all three points of the triangles that can be formed in U1, use SVD decomposition to obtain the rotation matrix and calculate the MSE. Take the rotation matrix R and translation matrix T corresponding to the minimum MSE value as the final result.

[0139] See Figure 4-6 In S3, distances are calculated using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Based on the remaining threshold, point cloud data of the protruding area after welding repair is extracted. The specific steps are as follows:

[0140] S31: Calculate the distance between corresponding points in the local point cloud data and the global point cloud data of the roof. Use eigenvalue decomposition to obtain the normal vector of each point in the global point cloud data of the roof. Take any point in the local shaping region as the search point. The K-nearest neighbor search algorithm is used to find the nearest point in the global point cloud data of the roof. , The normal vector information is :

[0141] Constructing vectors ;

[0142] arrive The distance is :

[0143] ;

[0144] This method is used to calculate the distance from each point in the local point cloud data to the corresponding point, and the distance information is stored in the intensity field.

[0145] S32. Calculate the distance from the local reshaping area to the global data of the top cover, create a histogram, and based on the histogram information, specify a distance threshold. .

[0146] As a specific example, a given distance threshold In order to extract the raised area after the welding repair as completely as possible, points with a distance less than the threshold are selected as the target point cloud.

[0147] Due to insufficient precision in threshold setting and the influence of measurement errors, the extracted point cloud data of the convex region is filtered. Traditional Euclidean clustering methods use KD-Tree for nearest neighbor search, employing Euclidean distance—the absolute distance between two points in space—to measure the nearest neighbor. To filter the point cloud data of the convex region, the idea of ​​Euclidean clustering can be used to remove discrete points. However, due to the large volume of point cloud data, using traditional Euclidean clustering results in high time complexity. To improve algorithm efficiency, a third-party library (nanoflann) is used instead of KD-Tree for nearest neighbor search and cluster analysis. Based on the clustering results and the geometric characteristics of the convex region—its elongated shape—the filtered convex region is identified.

[0148] In S4, the method for segmenting convex point cloud data includes the following steps:

[0149] S41: Extract the boundary points of the raised region, calculate the normal vector of the raised region, and determine whether it is a boundary point based on the angle between the normal vectors. Generally, the normal vector of the boundary point will change abruptly; this feature is used to extract the boundary points.

[0150] S42: Project the boundary points onto the plane, and perform spline curve fitting on the two longer sides of the boundary to obtain two curve equations.

[0151] S43: Along the direction of the raised region, find the corresponding points of the two curves. derivative at point , derivative at point Given a threshold , .

[0152] As a specific embodiment, a given threshold , .

[0153] in, Indicates the same At a certain point, the threshold difference between the derivatives of the two curves is closer to 0, indicating that the two curves are at a certain point (where...). - ), If the boundaries within the range are parallel, it can be approximated very accurately using a bounding box.

[0154] This represents the threshold for the slope change of the curve; the slope change is within the threshold. If the text is within the same section, it is considered part of the same paragraph. See also... Figure 7 This method divides the raised area into several data segments.

[0155] In S5, the method of approximating the shaping protrusion area using the minimum directed bounding box (OBB bounding box) and margin information, and then using this to obtain the grinding trajectory, includes the following steps:

[0156] S51: For each protruding region, solve for its minimum directed bounding box. Use the bounding box to find the common area between the protruding region and the 3D model of the turbine top cover, and consider it as the target shaping surface.

[0157] S52: Unify the normal vector direction of the target surface to be shaped, solve for the contour of the target surface to be shaped, and sample the set of trajectory points based on the contour direction. }

[0158] S53: Find the pose of the trajectory points. Constructing vectors ,Regulation x-axis direction The normal vector at is The z-axis direction is [missing information], and the y-axis direction is [missing information]. , The outer product is determined, and the pose of the trajectory point is determined accordingly.

[0159] S53: Send the pose of the trajectory points and the corresponding margin information to the robot for polishing.

Claims

1. A method for in-situ robotic grinding and shaping of a turbine top cover based on 3D vision, characterized in that: Includes the following steps: S1. Marker points are set up on the turbine top cover, and the three-dimensional coordinate information of the turbine top cover and the global marker points are obtained by using a scanner. The global point cloud data of the turbine top cover is obtained by resampling. S2. Measure the repaired area, identify and extract local marker points, and match the local repair area with the global model of the top cover based on the coordinate information of the local and global marker points. S3. Calculate the distance using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Extract the point cloud data of the protruding area after the welding repair by using the remaining threshold. S4. Extract the point cloud boundary of the convex region, perform spline curve fitting on the boundary data, and segment the convex point cloud data according to the curve curvature change; S5. For each data segment, combined with the margin information, the minimum directed bounding box is used to approximate the protrusion shaping area, the target shaping surface is obtained using the bounding box, the grinding trajectory is solved using the geometric margin information, and sent to the robot to perform the grinding work.

2. The method for in-situ robotic grinding and shaping of a turbine top cover based on 3D vision according to claim 1, characterized in that: In S1, a number of circular photogrammetric markers are arranged on the turbine top cover. A handheld scanner is used to measure and obtain the three-dimensional coordinate information of the turbine top cover 3D model and global markers. Based on the turbine top cover 3D model, resampling is performed. The global point cloud data of the turbine top cover is obtained by setting the total number of points after sampling and the distance between two points after sampling.

3. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 1, characterized in that: In S2, a 3D measuring head is installed at the end of the robot arm. After hand-eye calibration, the area to be repaired is measured to obtain the image and point cloud data of the area to be repaired. Then, using image methods, the coordinate information of the local area marker points is identified and extracted, and matched with the global marker points to achieve the matching of the local repair area with the global point cloud data of the top cover, thereby unifying the data under the robot coordinate system.

4. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 3, characterized in that: Using image-based methods, the coordinate information of local landmark points within a region is identified and extracted, and then matched with global landmark points to achieve matching between the local reshaping area and the global point cloud data of the roof. This process includes the following steps: S21: After the surface of the turbine top cover is repaired by welding, the three-dimensional measuring head is installed at the end of the robot. After hand-eye calibration, the area to be repaired is measured. At least three marker points are required in the area. Color images, depth images and point cloud data are obtained. The center of the ring in the color image is extracted using an ellipse fitting algorithm. S22. The extracted center of the circular marker is the pixel coordinate. Combining the imaging principle, depth map, and camera intrinsic parameter information, the three-dimensional coordinate information (X,Y,Z) corresponding to the center (x,y) of the two-dimensional ellipse is calculated according to the formula below: ； In the formula, the camera's intrinsic parameters are: ,in This represents the length of the focal length along the x-axis. This represents the length of the focal length along the y-axis. ) represents the pixel coordinates of the principal point, the coordinates of the ellipse center are (x,y), and its corresponding 3D coordinates are (X,Y,Z). D represents the pixel value at the (x,y) position in the depth map. The depth camera used in this case has pixel values ​​in millimeters and Z coordinates in meters. Therefore, the value of D needs to be divided by 1000 to convert it to meters. S23. After obtaining the three-dimensional coordinate information of the local region marker points, match the local region marker points with the global marker points. The set of local region marker points is U1, and the set of global marker points is U2. For each point in U1, a corresponding point can be found in U2. At the same time, distance and angle are introduced as constraints to convert point matching into edge matching. S24: After matching, register the local modification area with the global point cloud data of the turbine top cover, and then use the results of hand-eye calibration. The measurement data of the local shaping area and the global point cloud data are converted into robot coordinates for subsequent robot operations.

5. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 4, characterized in that: In S23, the specific steps of the matching algorithm are as follows: S2301: Starting from any point a in U1, find any point b, and construct a vector to obtain... Then find any side ac with the same starting point, and form a vector to obtain... ; S2302: Calculate the magnitudes of the vectors along sides ab and ac. , : ， ； vector , Normalization yields: ； ； calculate , dot product of vectors External product : ， ； S2303: Find any side de in U2, and construct a vector to obtain... Calculate the magnitude of the edge vector de. : ； For vectors Normalization yields: ； calculate , inner product : ； S2304: Starting from the origin d of edge de, find any edge df in U2, and construct a vector to obtain... Calculate the magnitude of the edge df vector. ; ； For vectors Normalization yields: ； calculate , inner product External product : ， ； S2305: Determine if points a, b, c correspond one-to-one with points d, e, f, while introducing distance and angle as constraints. Compare the triangles formed by a, b, c and d, e, f in space. Due to the random placement of marker points, multiple approximately equal triangles may appear in space, leading to incorrect point pair matching. Therefore, an enhanced condition is introduced to calculate... , inner product : The necessary and sufficient condition for points a, b, c and points d, e, f to form a point pair is: ； Due to measurement errors, a distance threshold exists when determining whether side lengths are equal. When determining whether the included angles on both sides are equal, there exists an angle threshold. If the above conditions are met, proceed to the next step; otherwise, execute S2303-S2305 until the corresponding points d, e, and f of a, b, and c are found. S2306: All edges in U1 that start from 'a' are used to find the corresponding edges in U2 through S2301-S2305, and the corresponding points are stored in array V; S2307: In U1, edges are formed starting from non-a. Any two edges bc and bd are selected to find the corresponding edges. S2308: Repeat steps S2301-S2305. The corresponding edges de and df found in U2, and the points d, e, and f should already exist in array V. If this condition is met, proceed to the next step; otherwise, repeat operations S2301-S2305. S2309: Repeat steps S2307-S2308 until both sides of U1 starting from all points have corresponding edges in U2; S2310: Using the found corresponding edges, construct corresponding points. Use singular value decomposition to solve for the rotation matrix between the two point sets. Solving the rotation matrix requires at least three pairs of points. Therefore, when measuring the local reshaping area, it is essential to ensure at least three marker points within the survey area. Considering measurement errors, arbitrarily select three points in U1 and their corresponding points in U2, solve for the rotation matrix R, and then solve for the translation matrix T through its centroid. All points in U1... After rotation and translation transformation, we obtain U0, and then convert the corresponding points in U2. Treating the true value as the actual value, calculate the mean squared error (MSE): ； S2311: For all three points of the triangles that can be formed in U1, use singular value decomposition to obtain the rotation matrix and calculate the MSE. Take the rotation matrix R and translation matrix T corresponding to the minimum MSE value as the final result.

6. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 1, characterized in that: In S3, distances are calculated using local point cloud data and global point cloud data of the top cover to obtain the remaining information of the local point cloud. Based on the remaining threshold, point cloud data of the protruding area after welding repair is extracted. The specific steps are as follows: S31: Calculate the distance between corresponding points in the local point cloud data and the global point cloud data of the roof. Use eigenvalue decomposition to obtain the normal vector of each point in the global point cloud data of the roof. Take any point in the local shaping region as the search point. The K-nearest neighbor search algorithm is used to find the nearest point in the global point cloud data of the roof. , The normal vector information is ; Constructing vectors ; arrive The distance is : ； This method is used to calculate the distance from each point in the local point cloud data to the corresponding point, and the distance information is stored in the intensity field; S32. Calculate the distance from the local reshaping area to the global data of the top cover, create a histogram, and based on the histogram information, specify a distance threshold. Points with a distance less than a threshold are selected as target point clouds. Due to insufficient precision in threshold setting and the influence of measurement error, the extracted point cloud data of the convex region is filtered. Based on the traditional Euclidean spatial distance, a third-party library is used to speed up the search for spatially nearby points and complete clustering. According to the geometric characteristics of the convex region: it is a long and narrow strip, the filtered convex region is found.

7. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 1, characterized in that: In S4, the method for segmenting convex point cloud data includes the following steps: S41: Extract the boundary points of the raised region, calculate the normal vector of the raised region, and determine whether it is a boundary point based on the angle between the normal vectors; S42: Project the boundary points onto the plane, take the two longer sides of the boundary and fit them with spline curves to obtain two curve equations; S43: Along the direction of the raised region, find the corresponding points of the two curves. derivative at point , derivative at point Given a threshold , ;in, Indicates the same At a certain point, the threshold difference between the derivatives of the two curves is closer to 0, indicating that the two curves are at a certain point (where...). - ), The boundaries within the range are parallel, and the bounding box is used for precise approximation. This represents the threshold for the slope change of the curve; the slope change is within the threshold. If the data is within the same segment, it is considered as a single segment; this method divides the raised area into several segments of data.

8. The in-situ robotic grinding and shaping method for turbine top cover based on 3D vision according to claim 1, characterized in that: In S5, the method of approximating the shaping protrusion area using the minimum directed bounding box and allowance information, and then using this to obtain the grinding trajectory, includes the following steps: S51: For each protruding region, solve for its minimum directed bounding box, and use the bounding box to find the common area between the protruding region and the three-dimensional model of the turbine top cover, which is regarded as the target shaping surface. S52: Unify the normal vector direction of the target surface to be shaped, solve for the contour of the target surface to be shaped, and obtain the trajectory points based on the contour direction. ; S53: Find the pose of the trajectory points; Constructing vectors ,Regulation x-axis direction The normal vector at is The z-axis direction is [missing information], and the y-axis direction is [missing information]. , The outer product is used to determine the pose of the trajectory point; S53: Send the pose of the trajectory points and the corresponding margin information to the robot for polishing.

Images