A method for extracting a weld seam of a complex workpiece based on a point cloud

Abstract

The application discloses a kind of based on point cloud's complex workpiece weld extraction method, after using sensor or other ways to obtain workpiece point cloud data, according to the method of plane fitting combined clustering split, the point cloud data of each part of workpiece is separated, then the nearest point searching mode can simultaneously obtain the weld feature point between the bottom plate and the weld in the web plate of workpiece;When fitting each type weld according to weld feature point, the intersection of weld is fully considered, and the point cloud belonging to each weld is obtained by the method of secondary fitting straight line, to reduce false fitting, and according to the characteristics of ship workpiece, after linear fitting is finished, the information of curved weld in workpiece is obtained by the method of arc linear fitting;The scheme of the application realizes complex workpiece complete weld information extraction weld fitting based on point cloud data, and can optimize the welding process and shorten the operation time.

Description

Technical Field

[0001] This invention belongs to the field of advanced shipbuilding intelligent manufacturing, specifically relating to a method for extracting weld seams from complex workpieces based on point clouds. Background Technology

[0002] Welding plays a significant role in shipbuilding, and weld information extraction is a crucial step in achieving automated welding. In some shipyards, welding robots still primarily rely on manual teaching. Complex workpieces often have numerous and varied weld types, making manual teaching difficult and time-consuming. Furthermore, real-time adjustments during the welding process are not possible after teaching is complete. Integrating sensors with robotic systems allows for automated welding by leveraging sensor perception of the welding environment. Existing research can be broadly categorized into weld recognition based on two-dimensional and three-dimensional information.

[0003] The weld seam extraction method using two-dimensional information is based on the deformation generated by the projection of a single-line laser sensor onto the workpiece. It identifies weld seam feature points in a single two-dimensional image, taking into account the weld seam type. Finally, all weld seam feature points are summarized to obtain weld seam information. Weld seam information extraction mainly uses methods such as Hough transform and grayscale transformation. While two-dimensional information-based recognition methods offer high detection accuracy, they become unsuitable when the workpiece surface has high reflectivity or the workpiece size is large.

[0004] Weld extraction using 3D information first requires acquiring complete workpiece information using sensors before weld identification. When the workpiece exceeds the sensor's field of view, multiple images are needed to stitch together the point cloud data from different parts. For welds with relatively simple geometric information, such as straight lines or pipe lines, the intersection line can be obtained by fitting planes or cylinders. For welds with many irregular spatial curves, a local identification method is constructed based on the characteristic of large changes in normal at weld feature points for filtering. When the workpiece size is large, both point cloud data acquisition and processing will consume a lot of time, affecting the efficiency of weld information acquisition.

[0005] Existing methods primarily identify small workpieces or single-type welds, and cannot effectively extract complete information on multiple weld types from large, complex workpieces. There is an urgent need for methods capable of extracting weld information from small batches of multi-shaped workpieces during shipbuilding. Summary of the Invention

[0006] To address the aforementioned problems, the present invention aims to provide a method for extracting weld seams from complex workpieces based on point clouds, thereby achieving complete extraction of weld seam information from the workpiece and fitting of the weld seam, optimizing the welding process, shortening operation time, and meeting the needs of intelligent and unmanned operations.

[0007] The specific technical solution for achieving the objective of this invention is as follows:

[0008] A method for extracting weld seams from complex workpieces based on point clouds includes the following steps:

[0009] Step 1: Scan the workpiece to be identified to obtain point cloud data;

[0010] Step 2: Process the obtained point cloud data to obtain the feature points of the workpiece weld;

[0011] Step 3: Based on the feature points of the workpiece weld, fit the information of each straight and curved weld.

[0012] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0013] The present invention uses sensors to acquire workpiece point cloud data, then separates the point cloud data of each part of the workpiece using a combination of planar fitting and clustering. The weld feature points between the bottom plate and stiffening plates can then be obtained simultaneously through nearest-neighbor search. When fitting different types of welds based on these feature points, the intersection of welds is fully considered. A quadratic straight-line fitting method is used to obtain the point cloud belonging to each weld, reducing misfitting. Furthermore, considering the characteristics of ship workpieces, after straight-line fitting, a circular arc straight-line fitting method is used to obtain information on curved welds in the workpiece.

[0014] This solution uses point cloud data to extract complete weld information from complex workpieces and fit the weld seam, thereby optimizing the welding process and shortening the operation time.

[0015] The present invention will be further described below with reference to specific embodiments. Attached Figure Description

[0016] Figure 1 This is a flowchart of the point cloud-based method for extracting weld seams from complex workpieces according to the present invention.

[0017] Figure 2 This is a diagram showing the separation result of the workpiece point cloud base plate rib in an embodiment of the present invention.

[0018] Figure 3 This is a diagram showing the extraction result of the complete point cloud weld seam of the workpiece in an embodiment of the present invention.

[0019] Figure 4 This is a flowchart of the weld fitting process based on weld feature points according to the present invention.

[0020] Figure 5 This is a fitting result diagram of the base plate point cloud weld in an embodiment of the present invention. Detailed Implementation

[0021] Example

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not specifically refer to the singular and may also include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0024] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application. It should also be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following drawings denote similar items; therefore, once an item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0025] Combination Figure 1 A method for extracting weld seams from complex workpieces based on point clouds includes the following steps:

[0026] Step 1: Scan the workpiece to be identified to obtain point cloud data;

[0027] In actual operation, the workpiece is scanned either by the vision recognition system built into the robot welding system, or by scanning the complex assembly workpiece of the ship using a 3D camera or lidar sensor to obtain the workpiece point cloud data.

[0028] Step 2: Process the obtained point cloud data to obtain the feature points of the workpiece weld:

[0029] Step 2-1: Use the random sample consensus algorithm to perform plane fitting on the workpiece point cloud data to obtain the point cloud data belonging to the base plate, and simultaneously separate the rib plate point cloud clusters.

[0030] Specifically, the bottom plate area of ​​ship workpieces is generally much larger than that of each rib plate, and the corresponding data proportion in the point cloud file is also larger. Therefore, by using the random sample consensus algorithm to perform a plane fitting in the workpiece point cloud and outputting the model with the most interior points, the point cloud data belonging to the bottom plate can be obtained, and the rib plate point cloud clusters can be separated simultaneously.

[0031] In this embodiment, the point cloud clusters of the workpiece before separation and the base plate and rib plate after separation are as follows: Figure 2 As shown;

[0032] Step 2-2: The obtained rib point cloud clusters are separated using a clustering algorithm to obtain the point cloud data of each rib, thus achieving rib separation for complex workpieces.

[0033] Step 2-2-1: The point cloud cluster of the rib plate is separated by the region growing clustering algorithm. The point cloud data of different rib plates can be obtained in advance. In complex workpieces, there are both curved and straight rib plates. Ship workpieces are large in size. The normal vector of the point cloud data of the curved rib plate part changes little. Therefore, when the curved rib plate and the straight rib plate are spliced ​​and the geometric information at the connection point changes little, the spliced ​​curved rib plate and the straight rib plate are easily clustered into the same point cloud cluster. A single clustering angle θ cannot cope with this.

[0034] To address this issue, an angle accumulation threshold is set during the region growing clustering process. It also records the cumulative angle of the normal vector error at each point in the seed queue during the current cluster generation process. .pass and The comparison ensures that point cloud data belonging to different ribs are correctly separated during the clustering process. When curved ribs intersect, the clustering angle threshold needs to be dynamically adjusted. To obtain the complete point cloud of the curved rib, the clustering process is as follows;

[0035] During the growth of the seed queue, the current seed point is accumulated. The cumulative value of the angle between the normal vectors of the k nearest neighboring points and the k nearest neighboring points. First, determine the accumulated value at the current point. Does it exceed the angle accumulation threshold? ,like Use the original region-growing clustering; if The neighboring points of the current seed point are not added to the seed queue, and clustering is stopped;

[0036] Step 2-2-2: After the current clustering is completed, repeat step 2-2-1 to cluster new clusters in the remaining point cloud until the initial normal vector angle threshold is reached. If the value cannot generate a new cluster, the angle accumulation threshold during the clustering process is adaptively adjusted based on the normal vector information of the neighboring points in the remaining point cloud. Use the adjusted angle to accumulate the threshold Cluster again until point cloud data of all curved ribs are obtained;

[0037] in, The average absolute error of the change in the normal vector of each neighboring point of the seed point is calculated using the values ​​of each point within the neighborhood. normal vector mean of neighborhood normal vectors The modulus of the difference is calculated.

[0038] Steps 2-3: The search for weld feature points between the base plate and the rib plate is hampered by the large amount of base plate point cloud data, resulting in low efficiency when searching directly. Therefore, the obtained base plate point cloud and rib plate point cloud are hash-encoded, and the nearest point search method is used to obtain the weld point cloud at the intersection of the base plate and the rib plate.

[0039] Step 2-3-1: Process the point clouds of the base plate and rib plate using hash value encoding:

[0040]

[0041] in, The three-dimensional coordinates of the point cloud;

[0042] Step 2-3-2: Calculate the Euclidean distance between the base plate point cloud and the rib plate point cloud. Using a set distance threshold, select base plate and rib plate point cloud pairs with distances less than the threshold as the weld feature points at the intersection of the base plate and rib plate. In this embodiment, the nearest point extraction algorithm is used to extract all weld information, such as... Figure 3 As shown.

[0043] Step 3, Combining Figure 4 Based on the characteristic points of the workpiece weld, the information of each straight and curved weld is obtained by fitting:

[0044] Step 3-1: For the extracted weld feature points, perform a second-order straight-line fitting using the random consensus algorithm. Based on the average distance between the nearest points in the point cloud, separate the point cloud data belonging to different intersecting straight lines, including:

[0045] Step 3-1-1: In the weld feature points, use the random consistency algorithm to fit a straight weld with a large threshold. First, fit the longer weld. All points in the same direction of the straight line will be fitted as interior points of the current straight line. Some points that do not belong to the current straight line and points at the intersection of the straight lines will affect the accuracy of the fitting results and will also affect the fitting of the remaining straight lines. First, use a large distance threshold to ensure that points belonging to the previous weld will be included in the first fitting result. Then, based on the average distance of the nearest neighbor points in the weld point cloud and the rib thickness, reasonably set the threshold for selecting interior points of the straight line. ;

[0046] Step 3-1-2: Set a threshold for selecting points within a straight line based on the average distance between the nearest neighbors of the weld feature points and the rib thickness. :

[0047]

[0048] in The average distance to the nearest neighbor. This is the proportionality coefficient;

[0049] Step 3-1-3: Perform a second fitting on the straight line obtained in the first fitting in step 3-1-1:

[0050] Select the segment with the most inliers in the first fitting, reduce the distance threshold for inlier selection in this segment, and perform line fitting again;

[0051] When there are many welds, points that do not belong to the current straight line in the straight direction will be mistakenly fitted into the points inside the line. When fitting a second time, it is necessary to sort the points inside the current straight line first, and the fitted points inside the line are unordered.

[0052] The interior points are sorted by the projection length of the vector formed by the interior points of the line and the origin selected by the fitted line onto the line direction. Then, the distance between adjacent points after sorting is used as the basis for further sorting. With segmented threshold The line is divided into segments of varying sizes, and the segment with the most interior points is selected for a second line fitting.

[0053] Step 3-2: For each extracted straight line, according to the workpiece welding task requirements, the straight lines that have an angle exceeding a certain value (e.g., 60°) with the current straight line are recorded as intersecting straight lines and each intersecting straight line is recorded.

[0054] Step 3-3: To divide the intersecting straight weld seams, the point closest to each intersecting straight line calculated and recorded in the previous straight line point cloud is used as a candidate dividing point, and the effective dividing point is selected according to the number of internal points within a certain range of the dividing point in the two intersecting straight lines.

[0055] For each possible split point, calculate the threshold for selecting points on the current line and the corresponding intersecting line. The number of interior points within the range is set according to the sensor accuracy, i. Between the two straight lines... If the number of interior points within the range is greater than i, the candidate split point is taken as the valid split point.

[0056] When dividing a line, among all the lines that intersect the current line selected based on the angle between the direction vectors of different lines, some lines may only intersect the current line in the extended direction and there is no actual intersection point.

[0057] After calculating the closest point of each intersecting line, the number of points within a certain range of the dividing point between the two lines is counted to determine whether the two lines actually intersect.

[0058] Steps 3-4: Divide the intersecting straight weld seams according to the effective dividing points. According to the working range requirements of the welding robot's welding gun, remove segments with less than a certain number of point cloud points after segmentation, such as straight segments with fewer than 20 internal points. Record and save the start and end points of the effective straight weld seams. For the effective straight segments, change the start and end point information of each straight line. The direction vector of segments belonging to the same straight line remains unchanged.

[0059] Steps 3-5: In the remaining point cloud data after the straight line fitting is completed, according to the characteristics of the ship workpiece, each curved weld is obtained by the circular arc fitting method, and the start and end points, the middle point of the arc, the center coordinates, the center normal vector, the radius, the start angle and the end angle information of each curved weld are recorded.

[0060] The weld fitting and segmentation results in this embodiment are as follows: Figure 5 As shown;

[0061] The present invention fully considers the intersection of welds when fitting weld feature points to various types of welds. It obtains the point cloud belonging to each weld by a quadratic straight line fitting method, reducing misfitting. At the same time, based on the characteristics of ship workpieces, it obtains the curved weld information in the workpiece by a circular arc straight line fitting method after the straight line fitting is completed. This solution can quickly realize the extraction and fitting of complete weld information of complex workpieces, optimize the welding process, and shorten the operation time.

[0062] This solution also provides a point cloud-based system for extracting weld seams from complex workpieces, including the following modules:

[0063] Point cloud acquisition module: used to scan the workpiece to be identified and obtain point cloud data;

[0064] Weld feature point acquisition module: used to process the acquired point cloud data to obtain the weld feature points of the workpiece;

[0065] Weld seam extraction module: used to fit and obtain information on straight and curved weld seams based on the feature points of the workpiece weld seam.

[0066] This solution also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0067] Step 1: Scan the workpiece to be identified to obtain point cloud data;

[0068] Step 2: Process the obtained point cloud data to obtain the feature points of the workpiece weld;

[0069] Step 3: Based on the feature points of the workpiece weld, fit the information of each straight and curved weld.

[0070] This solution also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program, when executed by a processor, performs the following steps:

[0071] Step 1: Scan the workpiece to be identified to obtain point cloud data;

[0072] Step 2: Process the obtained point cloud data to obtain the feature points of the workpiece weld;

[0073] Step 3: Based on the feature points of the workpiece weld, fit the information of each straight and curved weld.

[0074] The embodiments described above are merely one implementation method of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for extracting weld seams from complex workpieces based on point clouds, characterized in that, Includes the following steps: Step 1: Scan the workpiece to be identified to obtain point cloud data; Step 2: Process the obtained point cloud data to obtain the feature points of the workpiece weld: Step 2-1: Use the random sample consensus algorithm to perform plane fitting on the workpiece point cloud data to obtain the point cloud data belonging to the base plate, and simultaneously separate the rib plate point cloud clusters. Step 2-2: The obtained rib point cloud clusters are separated using a clustering algorithm to obtain the point cloud data of each rib, thus achieving rib separation for complex workpieces. Step 2-2-1: During the seed queue growth process, accumulate the current seed point. The cumulative value of the angle between the normal vectors of the k nearest neighboring points and the k nearest neighboring points. First, determine the accumulated value at the current point. Does it exceed the angle accumulation threshold? ,like Use the original region-growing clustering; if The neighboring points of the current seed point are not added to the seed queue, and clustering is stopped; Step 2-2-2: After the current clustering is completed, repeat step 2-2-1 to cluster new clusters in the remaining point cloud until the initial normal vector angle threshold is reached. If the value cannot generate a new cluster, the angle accumulation threshold during the clustering process is adaptively adjusted based on the normal vector information of the neighboring points in the remaining point cloud. Use the adjusted angle to accumulate the threshold Cluster again until point cloud data of all curved ribs are obtained; in, The average absolute error of the change in the normal vector of each neighboring point of the seed point is calculated using the values ​​of each point within the neighborhood. normal vector mean of neighborhood normal vectors The modulus of the difference is calculated. Steps 2-3: Hash value encoding is performed on the obtained base plate point cloud and rib plate point cloud, and the nearest point search method is used to obtain the weld point cloud at the intersection of the base plate and rib plate. Step 3: Based on the feature points of the workpiece weld, fit the information of each straight and curved weld.

2. The method for extracting weld seams from complex workpieces based on point clouds according to claim 1, characterized in that, The specific steps in steps 2-3 for obtaining the weld point cloud at the intersection of the base plate and the rib plate are as follows: Step 2-3-1: Process the point clouds of the base plate and rib plate using hash value encoding: ； in, The three-dimensional coordinates of the point cloud; Step 2-3-2: Calculate the Euclidean distance between the base plate point cloud and the rib plate point cloud, and use the set distance threshold to select the base plate point cloud and rib plate point cloud with a distance less than the threshold as the weld feature point cloud at the intersection of the base plate and the rib plate.

3. The method for extracting weld seams from complex workpieces based on point clouds according to claim 1, characterized in that, The weld information fitting based on weld feature points in step 3 is specifically as follows: Step 3-1: For the extracted weld feature points, use the random consensus algorithm to perform a second straight line fitting, and separate the point cloud data belonging to different intersecting straight lines based on the average distance between the nearest points in the point cloud. Step 3-2: For each extracted straight line, according to the workpiece welding task requirements, the straight lines whose angle with the current straight line exceeds a certain value are recorded as intersecting straight lines and each intersecting straight line is recorded. Step 3-3: To divide the intersecting straight weld seams, the point closest to each intersecting straight line calculated and recorded in the previous straight line point cloud is used as a candidate dividing point, and the effective dividing point is selected according to the number of inner points within a certain range of the dividing point in the two intersecting straight lines. When dividing a line, among all the lines that intersect the current line selected based on the angle between the direction vectors of different lines, some lines may only intersect the current line in the extended direction and there is no actual intersection point. After calculating the closest point of each intersecting line, the number of points within a certain range of the dividing point between the two lines is counted to determine whether the two lines actually intersect. Steps 3-4: Divide the intersecting straight welds according to the effective dividing points. According to the working range requirements of the welding robot's welding gun, remove segments with less than a certain number of point cloud values ​​after segmentation. Record and save the start and end points of the effective straight welds. Steps 3-5: In the remaining point cloud data after the straight line fitting is completed, according to the characteristics of the ship workpiece, each curved weld is obtained by the circular arc fitting method, and the start and end points, the middle point of the arc, the center coordinates, the center normal vector, the radius, the start angle and the end angle information of each curved weld are recorded.

4. The method for extracting weld seams from complex workpieces based on point clouds according to claim 3, characterized in that, The separation of point cloud data belonging to different intersecting lines in step 3-1 specifically involves: Step 3-1-1: In the weld feature points, use the random consistency algorithm to fit the straight weld with a larger threshold. First, fit the longer weld. All points in the same direction of the straight line will be fitted as the interior points of the current straight line. Step 3-1-2: Set a threshold for selecting points within a straight line based on the average distance between the nearest neighbors of the weld feature points and the rib thickness. : ； in The average distance to the nearest neighbor. This is the proportionality coefficient; Step 3-1-3: Perform a second fitting on the straight line obtained in the first fitting in step 3-1-1: Select the segment with the most inliers in the first fitting, reduce the distance threshold for inlier selection in this segment, and perform line fitting again; When there are many welds, points that do not belong to the current straight line in the straight direction will be mistakenly fitted into the points inside the line. When fitting a second time, it is necessary to sort the points inside the current straight line first. The points inside the line obtained by fitting are unordered. The interior points are sorted by the projection length of the vector formed by the interior points of the line and the origin selected by the fitted line onto the line direction. Then, the distance between adjacent points after sorting is used as the basis for further sorting. With segmented threshold The line is divided into segments of varying sizes, and the segment with the most interior points is selected for a second line fitting.

5. A point cloud-based system for extracting weld seams from complex workpieces, used to implement the method described in claim 1, characterized in that, Includes the following modules: Point cloud acquisition module: used to scan the workpiece to be identified and obtain point cloud data; Weld feature point acquisition module: used to process the acquired point cloud data to obtain the weld feature points of the workpiece; Weld seam extraction module: used to fit and obtain information on straight and curved weld seams based on the feature points of the workpiece weld seam.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-4.

7. A computer-storable medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images