Welding seam identification method of non-planar structural member, medium and equipment

Abstract

The invention relates to a welding seam identification method for a non-planar structural member, and the method comprises the steps: collecting a workpiece surface point cloud, segmenting the workpiece surface point cloud to obtain a bottom plate region point cloud and a rib plate region point cloud, carrying out the plane fitting of the bottom plate region point cloud, obtaining a search vector and a plurality of plane equations, and dividing each plane equation to obtain a plane region and a curved surface region; and constructing a rib plate plane equation according to the search vector, projecting the rib plate region point cloud to the rib plate plane equation to obtain a bottom plate boundary point cloud, fitting a straight line for the bottom plate boundary point cloud in the plane region or / and performing vector analysis on the bottom plate boundary point cloud in the curved surface region to obtain welding seam points, and sequencing the welding seam points to obtain a welding seam track. The problems that in the prior art, in a complex non-planar structure, the welding seam position cannot be efficiently and accurately recognized, and consequently the welding efficiency and quality are low are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of automated welding, and in particular to a method, medium and equipment for identifying a weld of a non-planar structural part. Background Art

[0002] In the field of modern shipbuilding, small-scale production management methods and processes are widely used to speed up shipbuilding, improve shipbuilding quality and form large-scale production. In recent years, with the development of computer vision and automation technology, weld recognition has gradually shifted from manual to automated. The existing technology mainly adopts imported models or vision-based weld detection methods to achieve automatic recognition and extraction of welds by building a three-dimensional model of the workpiece or analyzing two-dimensional images. Specifically, the workpiece model is imported into the software for analysis and processing, and the weld is manually selected for welding; the laser scanning method usually uses a three-dimensional laser scanner or a 3D camera to obtain the three-dimensional point cloud data of the workpiece, and locates the spatial position of the weld through the corresponding visual algorithm; the machine vision method usually uses a camera to capture a two-dimensional image of the workpiece surface, and combines the image processing algorithm to identify the weld position.

[0003] In small-scale welding, most workpieces are currently planar workpieces, while manual welding is mainly used for non-planar workpieces. The operator observes the geometric shape of the workpiece based on experience and determines the weld position for welding. Although this method is simple, it is easily affected by human factors, is inefficient and lacks accuracy, especially in complex non-planar structures. In robot welding based on visual information, the complexity of visual recognition is further increased due to the shape diversity and position uncertainty of the welds of non-planar structures, making it difficult to cope with the diverse needs of weld recognition for small-scale non-planar structural parts of ships.

[0004] Therefore, how to improve the existing technology in complex non-planar structures, which cannot efficiently and accurately identify the weld position, resulting in low welding efficiency and quality, is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] Based on this, the purpose of this application is to provide a weld identification method, medium and equipment for non-planar structural parts to solve at least one technical problem mentioned in the above background technology.

[0006] In a first aspect, the present application provides a method for identifying a weld of a non-planar structural member, comprising:

[0007] Collect the workpiece surface point cloud and segment it into the base plate area point cloud and the rib plate area point cloud;

[0008] Fit a plane to the point cloud of the bottom plate area to obtain a search vector and several plane equations, and divide each plane equation to obtain a plane area and a curved surface area;

[0009] Construct the web plane equation according to the search vector, and project the point cloud of the web area onto the web plane equation to obtain the bottom plate boundary point cloud;

[0010] Fit a straight line to the bottom plate boundary point cloud in the plane area or / and perform vector analysis on the bottom plate boundary point cloud in the curved surface area to obtain the weld points, and sort the weld points to obtain the weld track.

[0011] Further, the steps of fitting a plane to the bottom plate area point cloud to obtain the search vector and several plane equations, and dividing each plane equation to obtain the plane area and the curved surface area include:

[0012] Perform plane fitting on the bottom plate area point cloud to obtain several plane equations and the plane normal vectors corresponding to each plane equation;

[0013] Obtain the angle between each plane normal vector and the z-axis of the robot base coordinate system, and judge whether each angle is less than the set angle threshold. If so, the plane normal vector corresponding to this angle is the horizontal vector. If not, the plane normal vector corresponding to this angle is the inclined vector, and the corresponding plane equation is the inclined plane equation;

[0014] Obtain the average value of each horizontal vector and normalize it to obtain the search vector;

[0015] Obtain the average distance between the inliers of each inclined plane equation and the corresponding inclined plane equation, and judge whether the average distance is greater than the distance threshold. If so, the corresponding plane equation is the plane area. If not, the corresponding plane equation is the curved surface area.

[0016] Further, the steps of obtaining several plane equations and the plane normal vectors corresponding to each plane equation include:

[0017] S211: Perform plane fitting on the bottom plate area point cloud to obtain the corresponding plane equation;

[0018] S212: Obtain the distance between each point cloud in the bottom plate area point cloud and the corresponding plane equation, and judge whether it is less than the distance threshold. If so, this point cloud is the inlier of the corresponding plane equation. If not, this point cloud is the outlier of the corresponding plane equation;

[0019] S213: Judge whether the number of outliers is greater than the set number threshold. If so, use the outliers of the plane equation as the updated bottom plate area point cloud, and return to step S211. If not, obtain all the plane equations and the plane normal vectors of each plane equation.

[0020] Further, the steps of constructing the web plane equation according to the search vector and projecting the point cloud of the web area onto the web plane equation to obtain the bottom plate boundary point cloud include:

[0021] Obtain the coordinate extreme values of the rib plate area points cloud, and construct the rib plate plane equation according to the coordinate extreme values and the search vector;

[0022] Project the rib plate area points cloud onto the rib plate plane equation to obtain the rib plate projected points cloud;

[0023] Calculate the centroid of the rib plate projected points cloud to obtain a number of centroid points, and construct the refined rib plate points cloud;

[0024] Taking each refined rib plate points cloud as the center point, obtain the centroid of all the bottom plate area points cloud within the set range, and obtain the coordinates of each bottom plate boundary point according to the x coordinate, y coordinate of each refined rib plate points cloud and the z coordinate of the corresponding centroid, so as to construct the bottom plate boundary points cloud.

[0025] Further, when the bottom plate boundary points cloud is located in the plane area, the steps of obtaining the weld points and sorting the weld points to obtain the weld track include:

[0026] S41: Fit a straight line to the bottom plate boundary points cloud to obtain the corresponding straight line equation;

[0027] S42: Obtain the distance between each point cloud in the bottom plate boundary points cloud and the corresponding straight line equation, and judge whether it is less than the distance threshold. If so, the point cloud is the in-point of the corresponding straight line equation. If not, the point cloud is the out-point of the corresponding straight line equation;

[0028] S43: Judge whether the number of out-points is greater than the set number threshold. If so, take the out-points of the straight line equation as the updated bottom plate boundary points cloud, and return to step S41. If not, obtain all the straight line equations and the corresponding in-points of each straight line equation.

[0029] S44: Conduct vector analysis on the corresponding in-points of each straight line equation to obtain the straight line endpoints;

[0030] S45: Taking any end point cloud in the bottom plate boundary points cloud as the starting point, obtain the distance between each straight line endpoint and the starting point, and sort the corresponding straight line endpoints in the set order according to the distance to obtain the sequential straight line endpoints;

[0031] S46: Sort the corresponding straight line equations according to the sequential straight line endpoints to obtain the sorted straight line equations;

[0032] S47: Obtain the intersection points between adjacent straight line equations in the sorted straight line equations, as well as the starting straight line endpoint and the final straight line endpoint in the sequential straight line endpoints, as the weld points, and construct the weld track according to the order of the weld points.

[0033] Further, the steps of conducting vector analysis on the corresponding in-points of each straight line equation to obtain the straight line endpoints include:

[0034] Traverse the points in each straight-line equation, obtain all the interior points within a set radius centered on the current interior point, and obtain the current local point cloud;

[0035] Obtain the point in the current local point cloud that is farthest from the current feature point to construct a first vector;

[0036] Traverse the current local point cloud, construct a second vector based on the current feature point and the current local points, and obtain the cosine value between the first vector and the second vector. Determine whether it is greater than the set cosine threshold. If not, keep it unchanged. If so, the current interior point is the endpoint of the straight line.

[0037] Furthermore, when the bottom plate boundary point cloud is located in the curved surface area, the steps of obtaining the weld points and sorting the weld points to obtain the weld track include:

[0038] S48: Conduct vector analysis on the bottom plate boundary point cloud to obtain the curve endpoints;

[0039] S49: Take any endpoint cloud in the bottom plate boundary point cloud as the starting point, obtain the distances between each curve endpoint and the starting point, and sort the corresponding curve endpoints in a set order according to the distances to obtain the sorted curve endpoints, which are the sequential curve endpoints;

[0040] S410: Fit a straight-line equation with the starting curve endpoint and the final curve endpoint, obtain the distances between each curve endpoint and the straight-line equation, and sort the corresponding curve endpoints in a set order according to the distances to obtain the sorted curve endpoints, which are the weld points, to construct the weld track

[0041] Furthermore, when the bottom plate boundary point cloud is partially located in the curved surface area and partially located in the plane area, the steps of obtaining the weld points and sorting the weld points to obtain the weld track include:

[0042] S411: Obtain the weld points in the plane area according to steps S41 - S47;

[0043] S412: Obtain the weld points in the curved surface area according to steps S48 - S410;

[0044] S413: Insert the weld points in the curved surface area into the weld points in the plane area through the B-spline planning method to obtain the sorted weld points to construct the weld track.

[0045] In a second aspect, the present application also provides a computer storage medium storing executable program code; the executable program code is used to execute the weld recognition method for non-planar structural members according to any item in the first aspect.

[0046] In a third aspect, the present application also provides a terminal device, including a memory and a processor; the memory stores program code executable by the processor; the program code is used to execute the weld seam recognition method for the non-planar structural member according to any one of the first aspect.

[0047] A method, medium, and device for recognizing weld seams of a non-planar structural member provided by the present invention collect the point cloud on the surface of a workpiece and segment it into the point cloud of the bottom plate area and the point cloud of the rib plate area, so as to accurately obtain the three-dimensional coordinate information on the surface of the workpiece, provide basic data for subsequent segmentation and analysis, and at the same time can adapt to workpieces of various complex shapes, improving the applicable range of the method. Then, the point cloud of the bottom plate area is fitted to a plane to obtain a search vector and several plane equations, and each plane equation is divided to obtain a plane area and a curved surface area. By dividing the range covered by each plane equation into a plane area and a curved surface area, it is helpful to adopt different processing methods for different areas subsequently, improving the flexibility and accuracy of processing. Then, a rib plate plane equation is constructed according to the search vector, and the point cloud of the rib plate area is projected onto the rib plate plane equation to obtain the bottom plate boundary point cloud, providing key data for subsequent extraction of weld points. Finally, a straight line is fitted to the bottom plate boundary point cloud in the plane area or / and vector analysis is performed on the bottom plate boundary point cloud in the curved surface area to obtain weld points, and the weld points are sorted to obtain a weld track. By adopting different point cloud processing methods for the bottom plate boundary point cloud in different areas, the recognition accuracy of weld points is improved, thereby efficiently and accurately identifying the weld position. It solves the problems in the prior art that in a complex non-planar structure, the weld position cannot be efficiently and accurately identified, resulting in low welding efficiency and quality. Description of the Drawings

[0048] Figure 1 is a flowchart of the method for recognizing weld seams of a non-planar structural member according to an embodiment of the present invention;

[0049] Figure 2 is a schematic diagram of the point cloud on the surface of a workpiece according to an embodiment of the present invention;

[0050] Figure 3 is a schematic diagram of the point cloud of the bottom plate area and the point cloud of the rib plate area according to an embodiment of the present invention;

[0051] Figure 4 is a schematic diagram of the projected point cloud of the rib plate according to an embodiment of the present invention;

[0052] Figure 5 is a schematic diagram of the bottom plate boundary point cloud according to an embodiment of the present invention;

[0053] Figure 6 is a schematic diagram of weld points according to an embodiment of the present invention. Detailed Embodiments

[0054] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0055] It should be noted that if there are directional indications involved in the embodiments of the present invention, such as up, down, left, right, front, back..., then the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture. If the specific posture changes, the directional indications will also change accordingly. In addition, if there are descriptions such as "first, second", "S1, S2", "step one, step two" in the embodiments of the present invention, such descriptions are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that all those that do not violate the invention points under the inventive concept of the invention should be included in the protection scope of the present invention.

[0056] As Figure 1 shown, the present invention provides a method for identifying weld seams of non-planar structural parts:

[0057] S1: Collect the point cloud on the surface of the workpiece, and segment it to obtain the point cloud of the bottom plate area and the point cloud of the rib plate area;

[0058] Specifically, optionally but not limited to, obtain the surface image of the workpiece by shooting with a collection device to obtain the point cloud data of the workpiece, and obtain the point cloud on the surface of the workpiece as Figure 2 shown, so as to preprocess the point cloud on the surface of the workpiece to segment and obtain the point cloud of the bottom plate area and the point cloud of the rib plate area as Figure 3 shown. More specifically, the collection device can be optionally but not limited to common scanning devices such as 3D line scan cameras, 3D structured light cameras, TOF cameras, laser scanners, multi-view stereo vision systems, grating projection measuring instruments, etc. that can be used to obtain three-dimensional data on the surface of an object.

[0059] Preferably, in order to collect the complete point cloud on the surface of the workpiece, optionally but not limited to, provide a collection device, which may include a ground rail cantilever, a robot, a collection device, a welding platform, and a ground rail. Install the ground rail cantilever on the ground rail, hang the robot upside down on the ground rail cantilever, the robot grabs the collection device, detachably fix the workpiece on the welding platform, drive the ground rail cantilever to move uniformly through the ground rail, and the robot on the ground rail cantilever grabs the collection device to surround the surface of the workpiece to collect the complete point cloud on the surface of the workpiece.

[0060] Further preferably, the steps of obtaining the point cloud of the workpiece surface and segmenting to obtain the point cloud of the bottom plate area and the point cloud of the rib plate area may include:

[0061] S11: Collect the point cloud of the workpiece surface;

[0062] S12: Perform a pass-through filter on the points on the workpiece surface to obtain the initial point cloud of the bottom plate area and the initial point cloud of the rib plate area;

[0063] S13: Perform downsampling and statistical filtering on the initial point cloud of the bottom plate area and the initial point cloud of the rib plate area to obtain the optimized point cloud of the bottom plate area and the point cloud of the rib plate area.

[0064] Specifically, the specific parameters of methods such as pass-through filtering, downsampling, and statistical filtering are arbitrarily set by those skilled in the art.

[0065] S2: Fit a plane to the point cloud of the bottom plate area to obtain a search vector and several plane equations, and divide each plane equation to obtain a plane area and a curved surface area;

[0066] Specifically, optionally but not limited to, using the RANSAC plane fitting algorithm to fit a plane to the point cloud of the bottom plate area obtained in step S1 to obtain the corresponding plane equation and the plane normal vector of the plane equation, and then obtaining the distance between each point cloud in the point cloud of the bottom plate area and the plane equation, and determining whether it is greater than the threshold. If it is less, the corresponding point cloud is an inlier of the plane equation. If it is not less, the corresponding point cloud is an outlier of the plane equation. Repeat the above steps for the outliers to obtain several plane equations and the plane normal vectors of each plane equation. Calculate the average value of the plane normal vectors to obtain the search vector, and divide the area covered by each plane equation into a plane area and a curved surface area according to the angle between each plane equation and the Z axis of the robot coordinate system.

[0067] Preferably, the steps of fitting a plane to the point cloud of the bottom plate area to obtain a search vector and several plane equations, and dividing each plane equation into a plane area and a curved surface area may include:

[0068] S21: Fit a plane to the point cloud of the bottom plate area to obtain several plane equations and the corresponding plane normal vectors of each plane equation;

[0069] S211: Fit a plane to the point cloud of the bottom plate area to obtain the corresponding plane equation;

[0070] S212: Obtain the distance between each point cloud in the point cloud of the bottom plate area and the corresponding plane equation, and determine whether it is less than the distance threshold. If so, the point cloud is an inlier of the corresponding plane equation. If not, the point cloud is an outlier of the corresponding plane equation;

[0071] S213: Determine whether the number of outlier points is greater than the set number threshold. If so, use the outlier points of the plane equation as the updated floor area point cloud and return to step S211. If not, obtain all the plane equations and the plane normal vectors of each plane equation.

[0072] Specifically, it is optional but not limited to setting a distance threshold and a number threshold. Then, obtain the distances between each point cloud in the floor area point cloud and the corresponding plane equation. Divide the floor area point cloud into inlier points and outlier points according to the distance threshold, and determine whether the number of outlier points is greater than the set number threshold. If so, it means that there are still other planes, and the above steps need to be repeated to continue plane fitting for the remaining point cloud to extract all planes until the number of remaining outlier points is not greater than the set number threshold, indicating that all planes and their corresponding plane normal vectors have been extracted at this time, and the plane fitting is completed.

[0073] S22: Obtain the angles between each plane normal vector and the z-axis of the robot base coordinate system, and determine whether each angle is less than the set angle threshold. If so, the plane normal vector corresponding to this angle is a horizontal vector. If not, the plane normal vector corresponding to this angle is an inclined vector, and the corresponding plane equation is an inclined plane equation.

[0074] Specifically, it is optional but not limited to setting an angle threshold, obtaining the angles between each plane normal vector and the z-axis of the robot base coordinate system, and determining whether each angle is less than the set angle threshold. If so, it means that the plane corresponding to this angle is closer to the horizontal direction, so the plane normal vector corresponding to this angle is a horizontal vector. If not, the plane corresponding to this angle has a greater inclination degree relative to the horizontal direction, so the plane normal vector corresponding to this angle is an inclined vector, and the corresponding plane equation is an inclined plane equation.

[0075] S23: Obtain the average value of each horizontal vector and normalize it to obtain a search vector.

[0076] Specifically, it is optional but not limited to obtaining the average value of each horizontal vector and normalizing the average value to obtain a search vector.

[0077] Preferably, since both the floor area point cloud and the rib plate area point cloud are located in the positive direction of the z-axis of the robot base coordinate system, after obtaining the search vector, it is also necessary to judge the direction of the search vector. When its Z value is less than 0, it means that the direction of the search vector is opposite to the area where the floor area point cloud and the rib plate area point cloud are located, and the search vector needs to be reversed.

[0078] S24: Obtain the average distance between the inlier points of each inclined plane equation and the corresponding inclined plane equation, and determine whether the average distance is greater than the distance threshold. If so, the corresponding plane equation is a plane area. If not, the corresponding plane equation is a curved surface area.

[0079] Specifically, since the plane fitting is directly performed on the point cloud in the bottom plate area in step S21, a corresponding plane equation can be obtained for both the inclined plane and the curved surface. Therefore, a distance threshold can be set, and the average distance between the inliers of each inclined plane equation and the corresponding inclined plane equation can be obtained to determine whether the average distance is greater than the distance threshold. If so, it indicates that the distance between each point cloud and the plane equation is relatively close, and the area covered by the corresponding plane equation is a plane area. If not, it indicates that the distance between each point cloud and the plane equation is relatively far, the coordinates are relatively scattered, and the area covered by the corresponding plane equation is a curved surface area.

[0080] S3: Construct a rib plane equation according to the search vector, and project the point cloud in the rib area onto the rib plane equation to obtain the bottom plate boundary point cloud;

[0081] Specifically, it is optional but not limited to constructing a rib plane equation according to the point cloud in the rib area and the search vector obtained in step S2, and projecting the point cloud in the rib area onto the rib plane equation to obtain the bottom plate boundary point cloud

[0082] S31: Obtain the coordinate extreme values of the point cloud in the rib area to construct a rib plane equation according to the coordinate extreme values and the search vector;

[0083] Specifically, it is optional but not limited to obtaining the maximum value of each point cloud coordinate in the point cloud in the rib area as the coordinate extreme value, so as to construct a rib plane equation as shown in Equation 3-1 according to the coordinate extreme value and the search vector:

[0084] AX + BY + CZ + D = 0 3-1

[0085] Among them, (A, B, C) is the search vector, that is, the normal vector of the rib plane equation.

[0086] S32: Project the point cloud in the rib area onto the rib plane equation to obtain the rib projection point cloud;

[0087] Specifically, it is optional but not limited to projecting the point cloud in the rib area onto the rib plane equation according to the search vector to obtain the rib projection point cloud as Figure 4 shown.

[0088] Preferably, since the rib projection point cloud also includes several noise points, it is also necessary to perform Euclidean clustering on the rib projection point cloud, and retain the largest clustered point cloud as the optimized rib projection point cloud to remove the noise points.

[0089] S33: Calculate the centroid of the rib projection point cloud to obtain several centroid points to construct a refined rib point cloud;

[0090] Specifically, but not limited to, it is optional to set a segmentation distance. According to the segmentation distance, the rib projection point cloud is segmented into several local point clouds, and the average value of the point cloud coordinates in each local point cloud is obtained to get the centroid points corresponding to each local point cloud, so as to construct a refined rib point cloud according to all the centroid points; the segmentation distance can be arbitrarily set by those skilled in the art.

[0091] S34: Taking each refined rib point cloud as the center point, obtain the centroid of all the bottom plate area point clouds within the set range, so as to obtain the coordinates of each bottom plate boundary point according to the x coordinate, y coordinate of each refined rib point cloud and the z coordinate of the corresponding centroid, and construct the bottom plate boundary point cloud.

[0092] Specifically, but not limited to, taking each point in the refined rib point cloud as the center point, constructing a cuboid with a length of L, a width of W, and an infinite height, then obtaining the coordinate mean value of all the bottom plate area point clouds within the volume range of each cuboid to get the centroid points corresponding to each cuboid, obtaining the z coordinate of the centroid point corresponding to each cuboid and the x coordinate, y coordinate of the corresponding refined rib point cloud, to get the bottom plate boundary points corresponding to each cuboid, so as to construct the bottom plate boundary point cloud as Figure 5 shown.

[0093] S4: Fit a straight line to the bottom plate boundary point cloud in the plane area or / and perform vector analysis on the bottom plate boundary point cloud in the curved surface area to obtain the weld points, and sort the weld points to obtain the weld track;

[0094] Specifically, but not limited to, fitting a straight line to the bottom plate boundary point cloud in the plane area to obtain several straight line equations, so as to obtain the endpoints and intersection points of each straight line equation, to obtain the weld points according to the endpoints and intersection points, and sort them to obtain the weld track, or / and perform vector analysis on the bottom plate boundary point cloud in the curved surface area to obtain the starting point and ending point of the weld, and fit a straight line according to the starting point and ending point of the weld, to calculate the distance from each point in the bottom plate boundary point cloud in the curved surface area to the straight line, and sort the point cloud from small to large according to the distance to obtain the weld track; or, insert the sorted points into the weld points in the plane area after B-spline planning to form a complete weld track

[0095] Preferably, the steps of obtaining the weld points and sorting the weld points to obtain the weld track may optionally include:

[0096] When the bottom plate boundary point cloud is located in the plane area, the steps of obtaining the weld points and sorting the weld points to obtain the weld track may optionally include:

[0097] S41: Fit a straight line to the bottom plate boundary point cloud to obtain the corresponding straight line equation;

[0098] S42: Obtain the distance between each point cloud in the bottom plate boundary point cloud and the corresponding straight line equation, and determine whether it is less than the distance threshold. If so, the point cloud is an inlier of the corresponding straight line equation; if not, the point cloud is an outlier of the corresponding straight line equation.

[0099] S43: Determine whether the number of outliers is greater than the set number threshold. If so, use the outliers of the straight line equation as the updated bottom plate boundary point cloud, and return to step S41. If not, obtain all the straight line equations and the corresponding inliers of each straight line equation.

[0100] Specifically, it is optional but not limited to setting the distance threshold and the number threshold, and then obtain the distance between each point cloud in the bottom plate boundary point cloud and the corresponding straight line equation. Divide the bottom plate boundary point cloud into inliers and outliers according to the distance threshold, and determine whether the number of outliers is greater than the set number threshold. If so, it means that there are still other straight lines, and the above steps need to be repeated to continue fitting straight lines to the remaining point cloud to extract all the straight lines until the number of remaining outliers is not greater than the set number threshold, indicating that all the straight line equations and their inliers have been extracted at this time, and the straight line fitting is completed.

[0101] Preferably, there may be noise points among the inliers of each straight line equation at this time. Therefore, it is optional to project the inliers of each straight line equation onto the corresponding straight line equation, and then remove the noise points in the inliers through the statistical filtering method.

[0102] Further preferably, in order to improve the subsequent data processing efficiency, after step S43, it is optional to downsample the inliers of each straight line equation according to the projection parameters of each straight line equation to obtain the optimized inliers of the straight line equation:

[0103] S43': Obtain the projection parameters of each straight line equation to downsample the inliers of the corresponding straight line equation according to the projection parameters to obtain the optimized inliers of the straight line equation.

[0104] Specifically, it is optional but not limited to setting the sampling radius, and then obtain any point P0 on the straight line equation and the direction vector L of the straight line equation to calculate the projection parameters of each straight line equation according to Equation 4-1:

[0105] t = (Pi - L) * L / L * L 4-1

[0106] where Pi is the i-th inlier corresponding to the straight line equation, 0 < i ≤ n, n is the number of inliers, (Pi - L) is the difference vector between point Pi and the direction vector L, (Pi - L) * L is the projection length of the difference vector on the vector L, and L * L is the square of the modulus length of the direction vector L.

[0107] Thus, the projection parameter t corresponding to the points inside each straight-line equation is obtained. Among the inliers corresponding to each straight-line equation, the two endpoints corresponding to the maximum value tmax and the minimum value tmin of the projection parameter t are obtained. The inliers of the straight-line equation within the sampling radius range of each endpoint are retained, and the inliers of the remaining optimized straight-line equations are downsampled to obtain the inliers of the optimized straight-line equations.

[0108] S44: Perform vector analysis on the corresponding inliers of each straight-line equation to obtain the straight-line endpoints;

[0109] Specifically, the steps of performing vector analysis on the corresponding inliers of each straight-line equation to obtain the straight-line endpoints, which are the weld points, may optionally include:

[0110] S441: Traverse the inliers of each straight-line equation to obtain all the inliers within a set radius centered on the current inlier, and obtain the current local point cloud;

[0111] S442: Obtain the point in the current local point cloud that is farthest from the current feature point to construct the first vector;

[0112] S443: Traverse the current local point cloud, construct the second vector according to the current feature point and the current local point, and obtain the cosine value between the first vector and the second vector. Determine whether it is greater than the set cosine threshold. If not, keep it unchanged. If so, the current inlier is the straight-line endpoint.

[0113] Specifically, it is optional but not limited to setting the radius distance R and the cosine threshold. Then traverse the inliers of each straight-line equation to obtain all the inliers within the set radius centered on the current inlier P as the current local point cloud. Subsequently, obtain the distances between all the point clouds in the current local point cloud and the current feature point, and obtain the point with the maximum distance value as point B to construct the first vector PB. Then traverse the current local point cloud again, construct the second vector according to the current feature point P and the current local point C PC and obtain the cosine value between the first vector PB and the second vector PC to determine whether it is greater than the cosine threshold. If not, keep it unchanged. If so, the current inlier P is the straight-line endpoint. Preferably, the cosine threshold is set to 0.

[0114] Further preferably, since there may be redundant endpoints in the point cloud data due to measurement errors or noise, a filtering distance threshold may be optionally set. Obtain the distances between each pair of straight-line endpoints, and determine whether they are less than the filtering distance threshold. If so, randomly remove one of them. If not, keep it unchanged.

[0115] S45: Using any endpoint cloud in the bottom plate boundary point cloud as the starting point, obtain the distances between each straight-line endpoint and the starting point, and sort the corresponding straight-line endpoints in a set order according to the distances to obtain the ordered straight-line endpoints;

[0116] S46: Sort the corresponding line equations according to the sequential line endpoints to obtain the sorted line equations.

[0117] S47: Obtain the intersections between adjacent line equations in the sorted line equations, as well as the starting line endpoint and the final line endpoint among the sequential line endpoints, as weld points, and construct a weld track according to the order of the weld points.

[0118] When the bottom plate boundary point cloud is located in the curved surface area, the steps of obtaining weld points and sorting the weld points to obtain a weld track may optionally include:

[0119] S48: Perform vector analysis on the bottom plate boundary point cloud to obtain curve endpoints.

[0120] S49: Take any endpoint cloud in the bottom plate boundary point cloud as the starting point, obtain the distances between each curve endpoint and the starting point, and sort the corresponding curve endpoints in a set order according to the distances to obtain the sorted curve endpoints, which are sequential curve endpoints.

[0121] S410: Fit a line equation with the starting curve endpoint and the final curve endpoint, obtain the distances between each curve endpoint and the line equation, and sort the corresponding curve endpoints in a set order according to the distances to obtain the sorted curve endpoints, which are weld points, to construct a weld track.

[0122] When the bottom plate boundary point cloud is partially located in the curved surface area and partially located in the plane area, the steps of obtaining weld points and sorting the weld points to obtain a weld track may optionally include:

[0123] S411: Obtain the weld points in the plane area according to steps S41 - S47.

[0124] S412: Obtain the weld points in the curved surface area according to steps S48 - S410.

[0125] S413: Insert the weld points in the curved surface area into the weld points in the plane area through the B-spline planning method to obtain the sorted weld points to construct a weld track.

[0126] After obtaining the weld track points, all the weld track points can be sent to the welding robot, enabling the robot to weld the weld according to the corresponding weld track points.

[0127] In this embodiment, a method for identifying weld seams of a non-planar structural member of the present invention is presented. By collecting the point cloud on the surface of the workpiece and segmenting to obtain the point cloud of the bottom plate area and the point cloud of the rib plate area, the three-dimensional coordinate information on the surface of the workpiece can be accurately obtained, providing basic data for subsequent segmentation and analysis. At the same time, it can adapt to workpieces of various complex shapes, improving the applicable range of the method. Then, a plane is fitted to the point cloud of the bottom plate area to obtain a search vector and several plane equations, and each plane equation is divided to obtain a plane area and a curved surface area. By dividing the range covered by each plane equation into a plane area and a curved surface area, it helps to adopt different processing methods for different areas in the subsequent process, improving the flexibility and accuracy of the processing. Then, a rib plate plane equation is constructed according to the search vector, and the point cloud of the rib plate area is projected onto the rib plate plane equation to obtain the bottom plate boundary point cloud, providing key data for subsequent extraction of weld points. Finally, a straight line is fitted to the bottom plate boundary point cloud in the plane area and / or vector analysis is performed on the bottom plate boundary point cloud in the curved surface area to obtain weld points, and the weld points are sorted to obtain a weld track. By adopting different point cloud processing methods for the bottom plate boundary point cloud in different areas, the recognition accuracy of weld points is improved, thereby efficiently and accurately identifying the weld position. This solves the problems in the prior art that in a complex non-planar structure, the weld position cannot be efficiently and accurately identified, resulting in low welding efficiency and quality, etc.

[0128] On the other hand, the present invention also provides a computer storage medium storing executable program code; the executable program code is used to execute the above-mentioned weld seam identification method for any non-planar structural member.

[0129] On the other hand, the present invention also provides a terminal device including a memory and a processor; the memory stores program code executable by the processor; the program code is used to execute the above-mentioned weld seam identification method for any non-planar structural member.

[0130] Exemplarily, the program code can be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the program code in the terminal device.

[0131] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the terminal device may further include input / output devices, network access devices, a bus, etc.

[0132] The processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0133] The memory may be an internal storage unit of the terminal device, such as a hard disk or memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a SmartMedia Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory may also include both an internal storage unit and an external storage device of the terminal device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0134] The above computer storage medium and terminal device are created based on the above method for identifying welds of non-planar structural members. Their technical functions and beneficial effects are not elaborated here. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0135] The above embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent should be subject to the appended claims.

Claims

1. A method for identifying welds of non-planar structural parts, characterized in that: include: Collect the workpiece surface point cloud and segment it into the base plate area point cloud and the rib plate area point cloud; Fit a plane to the point cloud of the bottom plate area to obtain a search vector and several plane equations, and divide each plane equation to obtain a plane area and a curved surface area; The plane equation of the rib plate is constructed according to the search vector, and the point cloud of the rib plate area is projected onto the plane equation of the rib plate to obtain the boundary point cloud of the bottom plate; A straight line is fitted to the bottom plate boundary point cloud in the plane area or / and a vector analysis is performed on the bottom plate boundary point cloud in the curved surface area to obtain weld points, and the weld points are sorted to obtain a weld trajectory.

2. The method according to claim 1, characterized in that The steps of fitting a plane to the bottom plate area point cloud to obtain a search vector and a plurality of plane equations, and dividing each plane equation to obtain a plane area and a curved surface area include: Perform plane fitting on the bottom plate area point cloud to obtain several plane equations and the plane normal vectors corresponding to each plane equation; Obtain the angle between each plane normal vector and the z-axis of the robot base coordinate system, and determine whether each angle is less than the set angle threshold. If so, the plane normal vector corresponding to the angle is a horizontal vector. If not, the plane normal vector corresponding to the angle is an inclined vector, and the corresponding plane equation is the inclined plane equation. Get the average value of each horizontal vector and normalize it to get the search vector; The average distance between the inner point of each inclined plane equation and the corresponding inclined plane equation is obtained, and it is determined whether the average distance is greater than the distance threshold. If so, the corresponding plane equation is a plane area, and if not, the corresponding plane equation is a curved surface area.

3. The method according to claim 2, characterized in that The steps of obtaining a plurality of plane equations and the plane normal vectors corresponding to the plane equations include: S211: performing plane fitting on the bottom plate area point cloud to obtain the corresponding plane equation; S212: Obtain the distance between each point cloud in the bottom plate area point cloud and the corresponding plane equation, and determine whether it is less than a distance threshold. If so, the point cloud is an inner point of the corresponding plane equation; if not, the point cloud is an outer point of the corresponding plane equation; S213: Determine whether the number of external points is greater than the set threshold value. If so, use the external points of the plane equation as the updated bottom plate area point cloud and return to step S211. If not, obtain all plane equations and the plane normal vectors of each plane equation.

4. The method according to claim 1, characterized in that: The steps of constructing the rib plate plane equation according to the search vector and projecting the rib plate area point cloud onto the rib plate plane equation to obtain the bottom plate boundary point cloud include: Obtaining the coordinate extreme value of the point cloud in the rib plate area, so as to construct the rib plate plane equation according to the coordinate extreme value and the search vector; Project the point cloud of the rib plate area onto the rib plate plane equation to obtain the rib plate projection point cloud; Calculate the centroid of the rib plate projection point cloud and obtain several centroid points to construct a refined rib plate point cloud; Taking each refined stiffener point cloud as the center point, the centroid of all bottom plate area point clouds within the set range is obtained, so as to obtain the coordinates of each bottom plate boundary point according to the x-coordinate, y-coordinate of each refined stiffener point cloud and the z-coordinate of the corresponding centroid, so as to construct the bottom plate boundary point cloud.

5. The method according to claim 1, characterized in that When the bottom plate boundary point cloud is located in a plane area, the steps of obtaining weld points and sorting the weld points to obtain weld trajectories include: S41: fitting a straight line to the bottom plate boundary point cloud to obtain a corresponding straight line equation; S42: Obtain the distance between each point cloud in the bottom plate boundary point cloud and the corresponding straight line equation, and determine whether it is less than a distance threshold. If so, the point cloud is an inner point of the corresponding straight line equation; if not, the point cloud is an outer point of the corresponding straight line equation; S43: Determine whether the number of external points is greater than the set threshold. If so, use the external points of the straight line equation as the updated bottom plate boundary point cloud and return to step S41. If not, obtain all straight line equations and the corresponding internal points of each straight line equation. S44: Perform vector analysis on the corresponding interior points of each straight line equation to obtain the straight line endpoints; S45: taking any endpoint point cloud in the bottom plate boundary point cloud as the starting point, obtaining the distance between each straight line endpoint and the starting point, and sorting the corresponding straight line endpoints in a set order according to the distance to obtain sequential straight line endpoints; S46: sorting the corresponding straight line equations according to the sequential straight line endpoints to obtain sorted straight line equations; S47: Obtain the intersection points between adjacent straight line equations in the sorted straight line equations, and the starting straight line endpoint and the final straight line endpoint in the sequential straight line endpoints as weld points, and construct a weld trajectory according to the order of the weld points.

6. The method according to claim 5, characterized in that The steps of performing vector analysis on the corresponding interior points of each straight line equation to obtain the straight line endpoints include: Traverse the inner points of each line equation, obtain all inner points within the set radius with the current inner point as the center, and obtain the current local point cloud; Obtain the point farthest from the current feature point in the current local point cloud to construct the first vector; Traverse the current local point cloud, construct the second vector according to the current feature point and the current local point, and obtain the cosine value between the first vector and the second vector to determine whether it is greater than the set cosine threshold. If not, it remains unchanged. If yes, the current inner point is the endpoint of the straight line.

7. The method according to claim 5, characterized in that When the bottom plate boundary point cloud is located in the curved surface area, the steps of obtaining weld points and sorting the weld points to obtain weld trajectories include: S48: Perform vector analysis on the bottom plate boundary point cloud to obtain the curve endpoints; S49: obtaining any endpoint point cloud in the bottom plate boundary point cloud as the starting point, obtaining the distance between each curve endpoint and the starting point, and sorting the corresponding curve endpoints in a set order according to the distance to obtain sorted curve endpoints, which are sequential curve endpoints; S410: Fitting a straight line equation with the starting curve endpoint and the final curve endpoint, obtaining the distance between each curve endpoint and the straight line equation, and sorting the corresponding curve endpoints in a set order according to the distance to obtain the sorted curve endpoints as weld points to construct a weld trajectory.

8. The method according to claim 7, characterized in that When the bottom plate boundary point cloud is partially located in the curved surface area and partially located in the flat surface area, the steps of obtaining weld points and sorting the weld points to obtain weld trajectories include: S411: Obtaining the weld points in the plane area according to steps S41-S47; S412: Obtaining weld points in the curved area according to steps S48-S410; S413: inserting the weld points in the curved area into the weld points in the planar area by using a b-spline programming method to obtain sorted weld points to construct a weld trajectory.

9. A computer storage medium, characterized in that An executable program code is stored; the executable program code is used to execute the weld identification method for a non-planar structural part according to any one of claims 1 to 8.

10. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code executable by the processor; the program code is used to execute the weld identification method for a non-planar structural part according to any one of claims 1 to 8.

Images