Point cloud profile extraction method for polishing robot trajectory planning of numerous die bus bodies

Abstract

This invention discloses a point cloud contour extraction method for trajectory planning of a grinding robot for a bus body with numerous models. The method includes: projecting the point cloud of one side of the bus body into a binary image in the XoY plane; obtaining the upper left and lower right corner points of the circumscribed rectangles of the bus body and various hole areas on the body using image processing methods; drawing horizontal and vertical lines through the upper left and lower right corner points of all circumscribed rectangles to divide the bus body into several cells; merging the cells into several regions using a region partitioning algorithm, thereby segmenting the bus body point cloud into different regions; randomly downsampling the point cloud of each region according to a certain sampling rate, and then using a region growing method to segment the downsampled point cloud; using density to separate regions that need edge extraction from those that do not; and extracting the contours of each clustered region using an angle criterion algorithm.

Description

Technical Field

[0001] This invention relates to a method for edge extraction from point cloud data, and more particularly to a method for point cloud contour extraction for trajectory planning of a grinding robot for a bus body with numerous models. Background Technology

[0002] Due to factors such as the precision of the frame welding and the deformation of molded parts during the welding process, the surface of a bus body has relatively poor flatness. To ensure a smooth appearance, body filler needs to be applied, thus requiring sanding of body filler compared to passenger cars. Furthermore, buses often have more personalized requirements and complex patterns, frequently necessitating the spraying of patterns over the topcoat. Before spraying the patterns, the topcoat surface needs to be sanded. The number of sanding sessions varies from one to multiple times depending on the complexity of the pattern. Some bus manufacturers with high requirements for appearance quality use a sanding-and-clear coat process, where the entire exterior surface is sanded after the patterns are dried, and then a clear coat is applied. This means that compared to passenger cars, buses involve more sanding processes, including putty sanding, sanding before pattern spraying, and sanding before applying the clear coat. Currently, bus sanding generally combines manual and machine sanding, with robotic sanding as the primary method and manual sanding as a supplement.

[0003] In bus body polishing, the current common strategy for robotic polishing involves manually teaching the robot to polish for each new bus model, recording the polishing trajectory in a database, and then searching the database for a pre-set polishing trajectory before polishing a new bus. However, this manual teaching approach requires highly skilled workers, is labor-intensive, inefficient, and has a long teaching cycle, making it difficult to meet the needs of rapid iteration of bus models.

[0004] With the widespread use of 3D imaging equipment in robotics, point clouds, as an efficient data format containing descriptive information of 3D objects, are widely used in this field. One approach is to reduce the dimensionality of the point cloud and then, based on image processing technology, obtain the edges of the workpiece's processing area through grayscale transformation information at the edges to assist in subsequent trajectory planning. However, the edges of the grinding area in a bus body are sheet metal parts with multiple spatial curves in different positions, and image-based methods can only extract the edges of holes. Another approach is to extract edge points from the actual scanned point cloud by registering the edges in the given digital model. Although this method has high accuracy, it relies on manually set conditions and has poor applicability to different product models. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a point cloud contour extraction method for trajectory planning of a grinding robot for various bus models. This method can quickly and accurately obtain the edge information of different grinding areas of the bus body for different bus models, thereby effectively shortening the time spent in the trajectory planning process of the grinding robot, improving efficiency, and ensuring productivity.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A point cloud contour extraction method for trajectory planning of a grinding robot for a bus body with numerous models includes the following steps:

[0008] Step 1: Project the point cloud of one side of the bus body into a binary image in the XoY plane;

[0009] Step 2: Use image processing methods to obtain the upper left and lower right corner points of the bounding rectangle of the car body and various hole areas on the car body;

[0010] Step 3: Draw horizontal and vertical lines through the top left and bottom right corners of all the bounding rectangles in Step 2 to divide the car body into several cells;

[0011] Step 4: Use a region partitioning algorithm to merge the cells obtained in Step 3 according to certain rules, thereby dividing the point cloud on one side of the vehicle into several regions.

[0012] Step 5: According to the area divided in Step 4, randomly downsample the point cloud on one side of the vehicle body at a certain downsampling rate to reduce the size of the point cloud.

[0013] Step 6: Use the region growing algorithm to segment the point clouds of each region after downsampling in Step 5, and separate the point clouds of the regions to be polished in each point cloud.

[0014] Step 7: Use the angle criterion algorithm to extract the point cloud contour of the area to be polished in Step 6.

[0015] Preferably, the specific method for projecting the point cloud into a binary image in step 1 is as follows:

[0016] Traverse the point cloud of the bus body in the XoY plane. When the z-axis coordinate exists, set the gray value of the pixel to 1, otherwise set it to 0, thus obtaining a binary image of the bus body projected onto the XoY plane.

[0017] Preferably, the specific method for obtaining the circumscribed rectangle of the body-level body hole in step 2 is as follows:

[0018] Step 2-1: Find all contours in the binary image, traverse all contours, and the contour with the largest area is the contour of the car body.

[0019] Step 2-2: Perform an opening operation on the area contained in the vehicle body outline to separate the areas in the vehicle body that are slightly connected due to noise.

[0020] Steps 2-3: Calculate the circumscribed rectangle of the vehicle body;

[0021] Steps 2-4: Find the intersection of the binary image and the vehicle body, remove the wheels and outliers located outside the vehicle body, and obtain a binary image containing only the vehicle body;

[0022] Steps 2-5: Perform a closing operation on the binary graph containing only the car body to deconnect the hole regions in the car body;

[0023] Steps 2-6: Find all contours in the binary image from Step 2-5. Traverse all contours, and the areas of the regions contained within the threshold range are the openings such as windows and doors in the car body.

[0024] Step 2-7: Calculate the bounding rectangle of all hole regions in Step 2-6.

[0025] Preferably, the specific method for dividing the vehicle body into several cells in step 3 is as follows:

[0026] First, perform quick sorting on the x and y coordinates of the top left and bottom right corners of the outer rectangle of the car body and the outer rectangle of various hole areas on the car body obtained in step 2. Then, use the absolute value of the difference between adjacent x coordinates to represent the long side of the cell and the absolute value of the difference between adjacent y coordinates to represent the short side of the cell.

[0027] Preferably, the specific method for merging cells according to certain rules in step 4 is as follows:

[0028] Assume the merging direction can only be right or down. When a region can be merged both right and down, it should be merged in the direction that makes its aspect ratio approach 1. Assume each region after division is a rectangle. Assume the cell merging status has three states: not merged, merging in progress, and merging completed. Assume the conditions for determining whether merging can proceed in the current direction are: the maximum number of white pixels in the merged region does not exceed a threshold C; the merged region cannot contain cells in the merged state; and the merged region cannot be completely broken by cells whose white pixel ratio is ≤1%.

[0029] Step 4-1: Calculate the size of each cell, the number of white pixels in each cell, and the proportion of white pixels in each cell;

[0030] Step 4-2: Initialize the merged state of each cell to unmerged;

[0031] Step 4-3: Traverse each cell. If the proportion of white pixels in the current cell is ≥1%, and the proportion of white pixels in the two adjacent cells above and below it is ≤1%, then the current cell is column B, and its merge status is set to merge complete.

[0032] Step 4-4: Traverse from the top left corner to the bottom right corner, looking for the top left cell that is in the process of merging. If it exists, jump to step 4-5; if it does not exist, jump to step 4-10.

[0033] Steps 4-5: Traverse from the bottom right corner to the top left corner to find the bottom right cell that is in the process of merging;

[0034] Steps 4-6: Calculate the number of white pixels contained in the area being merged, as well as the length and width of the area, using the coordinates of the top-left corner of the top-left cell and the bottom-right corner of the bottom-right cell;

[0035] Step 4-7: Determine if the current region can be merged. If it can be merged, proceed to step 4-8; if it cannot be merged, proceed to step 4-10.

[0036] Steps 4-8: Determine the merging direction of the current region and merge along that direction;

[0037] Step 4-9: Repeat steps 4-5 to 4-8 until the current area no longer meets the merging conditions. Output the coordinates of the four endpoints of the merged area and set the merged status of all cells in the area to merged. Then jump to step 4-4.

[0038] Step 4-10: Traverse from the top left corner to the bottom right corner, find the cell in the top left corner whose merge status is not merged, set its merge status to merge in progress, and then jump to step 4-4;

[0039] Step 4-11: Repeat steps 4-4 to 4-10 until all cells are in the merged state.

[0040] Compared with the prior art, the present invention has the following obvious and prominent substantive features and significant advantages:

[0041] 1. This invention is particularly suitable for extracting the edges of the polished areas of bus bodies without digital models. The extraction process requires no human intervention and is automated. It solves the problems that image-based methods can only extract the edges of holes and cannot extract the edges of polished areas located in sheet metal parts of bus bodies, as well as the problem that digital model-based contour extraction methods cannot adapt to different products.

[0042] 2. Before performing edge extraction on the point cloud data, this invention utilizes the sparsity of the bus body point cloud obtained by a line-scan structured light camera in the sheet metal area due to its own structural characteristics. By first performing random downsampling at a certain sampling rate, the size of the point cloud is reduced while the density difference between the flat area and the sheet metal area of ​​the bus body point cloud is enhanced, resulting in a higher accuracy of point cloud contour extraction.

[0043] 3. This invention uses an intelligent segmentation algorithm to divide the point cloud of the bus body, so as to facilitate multi-threaded edge extraction of each segmented point cloud. This method is applicable to trajectory planning of grinding robots for various types of bus bodies and has a fast processing speed. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method of the present invention.

[0045] Figure 2 This is a binary graph that only contains the vehicle body.

[0046] Figure 3 This refers to the areas with openings such as windows and doors in the vehicle body.

[0047] Figure 4 To divide the vehicle body into several cells for the overclosing operation.

[0048] Figure 5 This refers to several regions of the vehicle body after merging cells according to certain rules using a region partitioning algorithm.

[0049] Figure 6 The point cloud outlines are extracted from each area to be polished. Detailed Implementation

[0050] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0051] Figure 1 As shown in the flowchart of the present invention, the present invention provides a point cloud contour extraction method for trajectory planning of a grinding robot for a multi-model bus body, comprising:

[0052] Step 1: Project a portion of the point cloud from one side of the bus body into a binary image in the XoY plane. In this step:

[0053] Traverse the point cloud of the bus body in the XoY plane. When the z-axis coordinate exists, set the gray value of the pixel to 1, otherwise set it to 0, thus obtaining a binary image of the bus body projected onto the XoY plane.

[0054] Step 2: Use image processing methods to obtain the upper left and lower right corner points of the bounding rectangles of the vehicle body and various hole areas on the body. In this example, due to the limitations of the point cloud acquisition equipment, only a portion of the bus body was obtained. The specific steps are as follows:

[0055] Step 2-1: Find all contours in the binary image, traverse all contours, and the contour with the largest area is the contour of the car body.

[0056] Step 2-2: Perform an opening operation on the area contained in the vehicle body outline to separate the areas in the vehicle body that are slightly connected due to noise.

[0057] Steps 2-3: Calculate the circumscribed rectangle of the vehicle body;

[0058] Steps 2-4 involve finding the intersection of the binary image and the vehicle body, removing wheels and outliers located outside the vehicle body, resulting in a binary image containing only the vehicle body, as shown below. Figure 2 As shown;

[0059] Steps 2-5: Perform a closing operation on the binary graph containing only the car body to deconnect the hole regions in the car body;

[0060] Steps 2-6: Find all contours in the binary image from Step 2-5. Traverse all contours; the areas containing these contours that fall within a threshold range are the openings in the vehicle body, such as windows and doors. Figure 3 As shown;

[0061] Step 2-7: Calculate the bounding rectangle of all hole regions in Step 2-6.

[0062] Step 3: Draw horizontal and vertical lines through the top-left and bottom-right corners of all the bounding rectangles from Step 2. Divide the car body into several cells for the through-closing operation, such as... Figure 4 As shown; firstly, quickly sort the horizontal and vertical coordinates of the upper left and lower right corners of the outer rectangle of the car body and the outer rectangle of various hole areas on the car body obtained in step 2. Then, use the absolute value of the difference between adjacent horizontal coordinates to represent the long side of the cell and the absolute value of the difference between adjacent vertical coordinates to represent the short side of the cell.

[0063] Step 4: Use a region partitioning algorithm to merge the cells obtained in Step 3 according to certain rules, thereby dividing the point cloud on one side of the vehicle into several regions, such as... Figure 5 As shown.

[0064] Assume the merging direction can only be right or down. When a region can be merged both right and down, it should be merged in the direction that makes its aspect ratio approach 1. Each region after division is a rectangle. The cell merging status has three options: not merged, merging in progress, and merging completed. The conditions for determining whether the current direction can be merged are: the maximum number of white pixels in the merged region does not exceed the threshold C; there cannot be any cells in the merged region that are in the merged state; and the merged region cannot be completely broken by cells whose proportion of white pixels is ≤1%.

[0065] Step 4-1: Calculate the size of each cell, the number of white pixels in each cell, and the proportion of white pixels in each cell;

[0066] Step 4-2: Initialize the merged state of each cell to unmerged;

[0067] Step 4-3: Traverse each cell. If the proportion of white pixels in the current cell is ≥1%, and the proportion of white pixels in the two adjacent cells above and below it is ≤1%, then the current cell is column B, and its merge status is set to merge complete.

[0068] Step 4-4: Traverse from the top left corner to the bottom right corner, looking for the top left cell that is in the process of merging. If it exists, jump to step 4-5; if it does not exist, jump to step 4-10.

[0069] Steps 4-5: Traverse from the bottom right corner to the top left corner to find the bottom right cell that is in the process of merging;

[0070] Steps 4-6: Calculate the number of white pixels contained in the area being merged, as well as the length and width of the area, using the coordinates of the top-left corner of the top-left cell and the bottom-right corner of the bottom-right cell;

[0071] Step 4-7: Determine if the current region can be merged. If it can be merged, proceed to step 4-8; if it cannot be merged, proceed to step 4-10.

[0072] Steps 4-8: Determine the merging direction of the current region and merge along that direction;

[0073] Step 4-9: Repeat steps 4-5 to 4-8 until the current area no longer meets the merging conditions. Output the coordinates of the four endpoints of the merged area and set the merged status of all cells in the area to merged. Then jump to step 4-4.

[0074] Step 4-10: Traverse from the top left corner to the bottom right corner, find the cell in the top left corner whose merge status is not merged, set its merge status to merge in progress, and then jump to step 4-4;

[0075] Step 4-11: Repeat steps 4-4 to 4-10 until all cells are in the merged state.

[0076] Step 5: According to the area divided in Step 4, the point cloud on one side of the vehicle body is randomly downsampled at a certain downsampling rate to reduce the point cloud size. Since the surface morphology of the measured object is complex and the curvature changes greatly, the line structure light stripe information may be lost due to the camera's field of view, resulting in incomplete final three-dimensional measurement data. This invention utilizes the sparseness of the point cloud in the sheet metal area of ​​the bus body obtained by the line scan structure light camera due to its own structural characteristics. By first performing random downsampling at a certain sampling rate in Step 5, the point cloud size is reduced while strengthening the difference in density between the point cloud in the flat area (i.e., the area that needs to be polished) and the sheet metal area in the bus body point cloud.

[0077] Step 6: Use the region growing algorithm to segment the point clouds of each region after downsampling in Step 5, separating the point cloud of the region to be polished from each point cloud. The steps of the region growing algorithm are given below:

[0078] The point cloud is sorted according to the curvature value of the points. The point with the smallest curvature is set as the initial seed point P. The region where the initial seed point is located is the smoothest region. Starting from the region where the initial seed point is located can reduce the total number of segmented segments, thereby improving the efficiency of the algorithm. An empty clustering region C and an empty seed point sequence Q are set, and a clustering array L is set. The smoothness threshold is set as ε and the curvature threshold is set as ξ.

[0079] 1) Start growing from seed point P. Select an initial seed point, add it to the seed point sequence Q, and search for the neighboring points of the seed point.

[0080] 2) Calculate the angle between the normal of each neighboring point and the normal of the seed point P. If the angle is less than the set smoothing threshold, add the neighboring point to the clustering region C. At the same time, determine whether the curvature value of the neighboring point is less than the curvature threshold ξ. Add the neighboring points with curvature less than ξ to the seed point sequence Q.

[0081] 3) After all the neighboring points have been judged, delete the current seed point P, select a new seed point in Q and repeat the above steps until the point sequence Q is empty. When a region has been grown, add it to the clustering array L.

[0082] 4) Select points from the input point set sequentially as seed points and add them to the seed point sequence, repeating the above growth steps.

[0083] Step 7: Use the angle criterion algorithm to extract the point cloud contour of the area to be polished in Step 6. The extracted point cloud contour is shown below. Figure 6 As shown, the steps of the angle criterion algorithm are given below:

[0084] Let the current point be P0, and its neighborhood be N. P0 There are k+1 points in the neighborhood, and the specified angle threshold is ξ.

[0085] 1) When the neighborhood N P0 Projecting the k+1 points within the plane onto the tangent plane of the current point P0, we obtain the projection points P′0, P′1, ..., P′. k With P0′ as the vertex of the angle, connect it to P′1, ..., P′ in clockwise / counterclockwise order. k The set of connected angles θ = {θ1, ..., θ2} k}

[0086] 2) Use the quicksort algorithm to find the set of maximum values ​​α in set θ. max .

[0087] 3) If θ max If ξ > ξ, then the current point P0 is an edge point.

[0088] In summary, the point cloud contour extraction method for trajectory planning of a grinding robot for a multi-modal bus body according to the above embodiments of the present invention includes: projecting the point cloud on one side of the bus body into a binary image in the XoY plane; obtaining the upper left and lower right corner points of the circumscribed rectangles of the bus body and various hole areas on the bus body using image processing methods; drawing horizontal and vertical lines through the upper left and lower right corner points of all circumscribed rectangles to divide the bus body into several cells; merging the cells into several regions using a region partitioning algorithm, thereby dividing the bus body point cloud into different regions; randomly downsampling the point cloud of each region according to a certain sampling rate, thereby segmenting the downsampled point cloud using a region growing method, and using density to separate the regions that need edge extraction from the regions that do not need edge extraction; and extracting the contours of each clustered region using an angle criterion algorithm.

[0089] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made according to the purpose of the invention. Any changes, modifications, substitutions, combinations or simplifications made based on the spirit and principle of the technical solution of the present invention shall be equivalent substitutions. As long as they meet the purpose of the invention and do not deviate from the technical principle and inventive concept of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A point cloud contour extraction method for trajectory planning of a grinding robot for countless bus bodies, characterized in that, Includes the following steps: Step 1: Project the point cloud of one side of the bus body into a binary image in the XoY plane; Step 2: Use image processing methods to obtain the upper left and lower right corner points of the bounding rectangle of the car body and various hole areas on the car body; Step 3: Draw horizontal and vertical lines through the top left and bottom right corners of all the bounding rectangles in Step 2 to divide the car body into several cells; Step 4: Use a region partitioning algorithm to merge the cells obtained in Step 3 according to certain rules, thereby dividing the point cloud on one side of the vehicle into several regions. Step 5: According to the area divided in Step 4, randomly downsample the point cloud on one side of the vehicle body at a certain downsampling rate to reduce the size of the point cloud. Step 6: Use the region growing algorithm to segment the point clouds of each region after downsampling in Step 5, and separate the point clouds of the regions to be polished in each point cloud. Step 7: Use the angle criterion algorithm to extract the point cloud contour of the area to be polished in Step 6.

2. The point cloud contour extraction method for trajectory planning of a grinding robot for multi-model passenger vehicle bodies according to claim 1, characterized in that, In step 1, the point cloud of the bus body is traversed in the XoY plane. When the z-axis coordinate exists, the gray value of the pixel is set to 1, otherwise it is set to 0, thus obtaining a binary image of the bus body projected onto the XoY plane.

3. The point cloud contour extraction method for trajectory planning of a grinding robot for multi-model passenger vehicle bodies according to claim 1, characterized in that, The specific method for step 2 is as follows: Step 2-1: Find all contours in the binary image, traverse all contours, and the contour with the largest area is the contour of the car body. Step 2-2: Perform an opening operation on the area contained in the vehicle body outline to separate the areas in the vehicle body that are slightly connected due to noise. Steps 2-3: Calculate the circumscribed rectangle of the vehicle body; Steps 2-4: Find the intersection of the binary image and the vehicle body, remove the wheels and outliers located outside the vehicle body, and obtain a binary image containing only the vehicle body; Steps 2-5: Perform a closing operation on the binary graph containing only the car body to deconnect the hole regions in the car body; Step 2-6: Find all contours in the binary image of Step 2-5, traverse all contours, and the areas of the regions contained within the threshold range are the window and door opening areas in the car body. Step 2-7: Calculate the bounding rectangle of all hole regions in Step 2-6.

4. The point cloud contour extraction method for trajectory planning of a grinding robot for multi-model passenger vehicle bodies according to claim 1, characterized in that, In step 3, the horizontal and vertical coordinates of the upper left and lower right corners of the outer rectangle of the vehicle body and the outer rectangle of various hole areas on the vehicle body obtained in step 2 are first quickly sorted. Then, the absolute value of the difference between adjacent horizontal coordinates is used to represent the long side of the cell, and the absolute value of the difference between adjacent vertical coordinates is used to represent the short side of the cell.

5. The point cloud contour extraction method for trajectory planning of a grinding robot for multi-model passenger vehicle bodies according to claim 1, characterized in that, The specific method for step 4 is as follows: Assuming the merging direction can only be right or down, when a region in the process of merging can be merged both to the right and down, the region should be merged in the direction that makes its aspect ratio approach 1. Each region after division is a rectangle; the cell merging status has three options: not merged, merging in progress, and merging completed. The conditions for determining whether a merge can be performed in the current direction are: the maximum number of white pixels in the merged area does not exceed the threshold C; there are no cells in the merged area that are in the merged state; and the merged area cannot be completely separated by cells whose proportion of white pixels is ≤1%. Step 4-1: Calculate the size of each cell, the number of white pixels in each cell, and the proportion of white pixels in each cell; Step 4-2: Initialize the merged state of each cell to unmerged; Step 4-3: Traverse each cell. If the proportion of white pixels in the current cell is ≥1%, and the proportion of white pixels in the two adjacent cells above and below it is ≤1%, then the current cell is column B, and its merge status is set to merge complete. Step 4-4: Traverse from the top left corner to the bottom right corner, looking for the top left cell that is in the process of merging. If it exists, jump to step 4-5; if it does not exist, jump to step 4-10. Steps 4-5: Traverse from the bottom right corner to the top left corner to find the bottom right cell that is in the process of merging; Steps 4-6: Calculate the number of white pixels contained in the area being merged, as well as the length and width of the area, using the coordinates of the top-left corner of the top-left cell and the bottom-right corner of the bottom-right cell; Step 4-7: Determine if the current region can be merged. If it can be merged, proceed to step 4-8; if it cannot be merged, proceed to step 4-10. Steps 4-8: Determine the merging direction of the current region and merge along that direction; Step 4-9: Repeat steps 4-5 to 4-8 until the current area no longer meets the merging conditions. Output the coordinates of the four endpoints of the merged area and set the merged status of all cells in the area to merged. Then jump to step 4-4. Step 4-10: Traverse from the top left corner to the bottom right corner, find the cell in the top left corner whose merge status is not merged, set its merge status to merge in progress, and then jump to step 4-4; Step 4-11: Repeat steps 4-4 to 4-10 until all cells are in the merged state.

Images