Weld seam recognition method, device, welding robot, and storage medium

Abstract

The application provides a welding seam identification method and device, a welding robot and a storage medium. The welding seam identification method comprises: acquiring three-dimensional point cloud data of a right-angle workpiece; if it is detected that there is an intersection straight line between multiple planes based on the three-dimensional point cloud data, then multiple boundary points having a neighboring relationship are extracted from the three-dimensional point cloud data for the planes having the intersection straight line; the foot points of the multiple boundary points having the neighboring relationship on the corresponding intersection straight line are determined respectively; based on the end points corresponding to the maximum foot point distance on the intersection straight line, a target point closest to the end points is searched from the three-dimensional point cloud data; and the end points are corrected based on the target point, and a corrected point obtained by the correction is identified as a welding seam feature point for the right-angle workpiece. The application can improve the accuracy of welding seam identification for the right-angle workpiece, and improve the efficiency of welding seam identification for the right-angle workpiece.

Description

Technical Field

[0001] This application relates to the field of welding technology, specifically to a weld seam identification method, device, welding robot, and storage medium. Background Technology

[0002] In robotic welding technology, identifying the weld seam is crucial for welding right-angled workpieces. Currently, line laser scanning robotic welding and human-taught robotic welding are the main methods. However, line laser scanning robotic welding, which relies on a 2D camera to extract the weld seam, is prone to misalignment and inaccurate location. Human-taught robotic welding, on the other hand, is labor-intensive and inefficient due to its reliance on manual operation. Therefore, current methods for weld seam identification on right-angled workpieces cannot simultaneously achieve both efficiency and accuracy. Summary of the Invention

[0003] One objective of this application is to provide a weld seam identification method, apparatus, welding robot, and storage medium that improves the accuracy and efficiency of weld seam identification for right-angled workpieces, at least to a certain extent.

[0004] According to one aspect of the embodiments of this application, a weld identification method is provided, including:

[0005] Obtain the 3D point cloud data of the right-angled workpiece;

[0006] If, based on the three-dimensional point cloud data, intersecting lines are detected between multiple planes, then for the planes with intersecting lines, multiple boundary points with adjacent relationships are extracted from the three-dimensional point cloud data; the multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes.

[0007] For the multiple boundary points that are adjacent to each other, determine the foot of the perpendicular from each boundary point to the corresponding intersecting line;

[0008] Based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines, the target point closest to the endpoint is found from the three-dimensional point cloud data; the maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines;

[0009] The endpoints are corrected based on the target point, and the corrected points are identified as weld feature points for the right-angled workpiece; the weld feature points are feature points at the weld of the right-angled workpiece.

[0010] According to one aspect of the embodiments of this application, a weld seam identification device is provided, comprising:

[0011] The acquisition module is used to acquire the 3D point cloud data of the right-angled workpiece;

[0012] The boundary point extraction module is used to extract multiple boundary points with adjacent relationships from the three-dimensional point cloud data if intersecting lines are detected between multiple planes based on the three-dimensional point cloud data; the multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes.

[0013] The perpendicular foot determination module is used to determine the perpendicular foot of each of the multiple boundary points that have an adjacent relationship on the corresponding intersecting lines.

[0014] The target point search module is used to search for the target point closest to the endpoint in the three-dimensional point cloud data based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines; the maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines;

[0015] The weld seam recognition module is used to correct the endpoint based on the target point and identify the corrected point as a weld seam feature point for the right-angled workpiece; the weld seam feature point is a feature point at the weld seam of the right-angled workpiece.

[0016] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0017] From the target points, obtain the three points that are closest to each other;

[0018] Based on the minimum bounding sphere, sphere fitting is performed on the three points that are closest to each other to obtain sphere data;

[0019] Extract the center of the encircling sphere from the sphere data;

[0020] The center of the surrounding sphere and the target point are used as the weld feature points for the right-angled workpiece.

[0021] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0022] The three-dimensional point cloud data is segmented into planes to obtain a first plane, a second plane, and a third plane. The first plane is the plane corresponding to the right-angled workpiece and parallel to the XY plane. The second plane is the plane corresponding to the right-angled workpiece and parallel to the YZ plane. The third plane is the plane corresponding to the right-angled workpiece and parallel to the XZ plane. The XY plane is the plane formed by the X-axis and the Y-axis. The YZ plane is the plane formed by the Y-axis and the Z-axis. The XZ plane is the plane formed by the X-axis and the Z-axis.

[0023] For the first plane, the second plane, and the third plane, detect whether there are intersecting straight lines between any two planes;

[0024] If there are intersecting lines between every two planes, then extract the first boundary point of the first plane, the second boundary point of the second plane, and the third boundary point of the third plane from the three-dimensional point cloud data.

[0025] Using the first boundary point, the second boundary point, and the third boundary point as reference points, the distances between the reference points and non-reference points are detected to obtain the distance between each reference point and the non-reference point; the non-reference points are the boundary points other than the reference points among the first boundary point, the second boundary point, and the third boundary point.

[0026] The system acquires reference points whose distance is less than a preset distance threshold and non-reference points whose distance is less than the preset distance threshold; the preset distance threshold is a pre-set distance indicating the distance between the non-reference points and the reference points.

[0027] The reference points whose distance is less than a preset distance threshold and the non-reference points whose distance is less than the preset distance threshold are used as multiple boundary points that have an adjacent relationship.

[0028] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0029] Determine the distance between every two feet of the intersecting lines;

[0030] The maximum perpendicular distance is obtained by comparing the distances between every two perpendicular feet on the intersecting lines.

[0031] Obtain the endpoint corresponding to the maximum perpendicular distance on the intersecting lines;

[0032] The Euclidean distance between each point in the three-dimensional point cloud data and the endpoint is calculated to obtain the distance between each point and the endpoint.

[0033] The point in the 3D point cloud data that is furthest from the endpoint is taken as the target point.

[0034] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0035] Obtain the original three-dimensional point cloud data for the right-angled workpiece;

[0036] The original three-dimensional point cloud data is transformed to obtain the actual three-dimensional point cloud data in the coordinate system of the welding robot base.

[0037] The actual 3D point cloud data is clustered to obtain the 3D point cloud data.

[0038] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0039] Obtain the value range of the actual 3D point cloud data in a preset dimension;

[0040] Iterate through each point in the actual 3D point cloud data and determine the value of the point in the preset dimension; the preset dimension is a pre-defined dimension used to indicate the direction towards the ground.

[0041] Get all points whose values ​​are not within the range of the given values;

[0042] Filter out all points whose values ​​are not within the range to obtain real 3D point cloud data;

[0043] The real 3D point cloud data is sampled to obtain sampled 3D point cloud data;

[0044] Clustering is performed on the sampled 3D point cloud data to obtain spatial noise points and target 3D point cloud data, and the target 3D point cloud data is used as the 3D point cloud data.

[0045] In some embodiments of this application, based on the above technical solutions, the weld identification device is configured as follows:

[0046] Based on the posture matrix of the center point of the welding robot's end tool and the robot's hand-eye matrix, the original three-dimensional point cloud data is transformed to obtain the actual three-dimensional point cloud data in the welding robot's base coordinate system.

[0047] According to one aspect of the embodiments of this application, a welding robot is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the welding robot to implement the methods provided in the various optional implementations described above.

[0048] According to one aspect of the embodiments of this application, a computer program medium is provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the methods provided in the various optional implementations described above.

[0049] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations described above.

[0050] In the technical solution provided in this application embodiment, for the three-dimensional point cloud data of a right-angled workpiece, planes with intersecting straight lines are detected, and boundary points adjacent to other planes are extracted from the boundary points of the planes. Based on the perpendicular feet of these boundary points on the intersecting straight lines, the two farthest target perpendicular feet are obtained. Target points corresponding to the target perpendicular feet are obtained from the three-dimensional point cloud data. Based on the target points, the endpoints corresponding to the target perpendicular feet are corrected to obtain the weld feature points of the right-angled workpiece. On the one hand, by processing the three-dimensional data, the characteristics of the workpiece in three-dimensional space can be reflected, thereby improving the accuracy of weld feature point recognition compared to the extraction method using a two-dimensional camera. On the other hand, processing the three-dimensional data in the above manner improves the efficiency of weld feature point recognition compared to manual teaching that relies on manual operation.

[0051] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0052] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0053] The above and other objectives, features and advantages of this application will become more apparent from a detailed description of exemplary embodiments thereof with reference to the accompanying drawings.

[0054] Figure 1 A schematic flowchart of a weld identification method according to Embodiment 1 of this application is shown.

[0055] Figure 2 A schematic flowchart of a weld identification method according to Embodiment 2 of this application is shown.

[0056] Figure 3 A schematic diagram showing the relationship between the three planes involved in Embodiment 2 of this application is shown.

[0057] Figure 4 A schematic diagram illustrating the relationship between three intersecting straight lines according to Embodiment 2 of this application is shown.

[0058] Figure 5 A schematic diagram showing the relationship between corresponding boundary points of each plane according to Embodiment 2 of this application is provided.

[0059] Figure 6 A schematic diagram illustrating the relationship between adjacent boundary points according to Embodiment 2 of this application is shown.

[0060] Figure 7 A schematic diagram showing the relationship between the various weld feature points according to Embodiment 2 of this application is shown.

[0061] Figure 8A flowchart illustrating the steps of a specific weld identification process according to Embodiment 2 of this application is shown.

[0062] Figure 9 A schematic diagram of the weld identification device according to Embodiment 3 of this application is shown.

[0063] Figure 10 A schematic diagram of the structure of a welding robot according to Embodiment 4 of this application is shown. Detailed Implementation

[0064] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided to make the description of this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0065] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more exemplary embodiments. Numerous specific details are provided in the following description to give a full understanding of exemplary embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced with one or more specific details omitted, or other methods, components, steps, etc., can be employed. In other instances, well-known structures, methods, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0066] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0067] Figure 1 A schematic flowchart of a weld identification method according to Embodiment 1 of this application is shown. The weld identification method includes:

[0068] Step S101: Obtain the 3D point cloud data of the right-angled workpiece;

[0069] The three-dimensional point cloud data of a right-angled workpiece can be obtained by taking pictures of the right-angled workpiece with a three-dimensional camera, which can be installed at the end of the welding torch of a welding robot.

[0070] As an optional implementation, the original three-dimensional point cloud data of the right-angled workpiece captured by a three-dimensional camera is used as the three-dimensional point cloud data to quickly obtain the three-dimensional point cloud data.

[0071] As an optional implementation, in order to make the three-dimensional point cloud data more accurate, ground point cloud and spatial noise can be filtered out to obtain the three-dimensional point cloud data, thereby further improving the accuracy of weld feature point identification.

[0072] As an optional implementation, in order to make the weld identification process fast and the identification results accurate, the point cloud can be sampled after filtering out ground point cloud and spatial noise, and a portion of the point cloud data can be used as the three-dimensional point cloud data. After processing, the weld feature points can be obtained.

[0073] Step S102: If intersecting lines are detected between multiple planes based on 3D point cloud data, then for the planes with intersecting lines, extract multiple boundary points with adjacent relationships from the 3D point cloud data; multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes.

[0074] As an optional implementation, the multiple planes include three planes with three intersecting straight lines. For a right-angled workpiece, the three planes intersect pairwise, and boundary points of each plane exist near the intersection lines. The boundary points of the two planes on either side of the intersection lines are adjacent. To obtain the adjacent boundary points of two planes, a specific distance range for indicating adjacency can be preset. If the distance between two boundary points is within this range, the two boundary points are determined to be adjacent. It should be noted that when extracting boundary points with an adjacent relationship, the distance between the boundary points of the two planes is judged, and the corresponding adjacent boundary points are obtained between the two planes; for boundary points within the same plane, even if the distance between two boundary points is within this range, they will not be considered as boundary points with an adjacent relationship.

[0075] As an optional implementation, if intersecting lines are detected between multiple planes based on 3D point cloud data, then for the planes with intersecting lines, multiple boundary points with adjacent relationships are extracted from the 3D point cloud data, including: performing plane segmentation on the 3D point cloud data to obtain a first plane, a second plane, and a third plane; the first plane is the plane corresponding to the right-angled workpiece and parallel to the XY plane, the second plane is the plane corresponding to the right-angled workpiece and parallel to the YZ plane, and the third plane is the plane corresponding to the right-angled workpiece and parallel to the XZ plane; the XY plane is the plane formed by the X-axis and Y-axis, the YZ plane is the plane formed by the Y-axis and Z-axis, and the XZ plane is the plane formed by the X-axis and Z-axis; for the first plane, the second plane, and the third plane, it is detected whether there are intersecting lines between every two planes; if each If there are intersecting lines between two planes, the first boundary point of the first plane, the second boundary point of the second plane, and the third boundary point of the third plane are extracted from the 3D point cloud data. Using the first boundary point, the second boundary point, and the third boundary point as reference points, the distance between the reference points and non-reference points is detected to obtain the distance between each reference point and a non-reference point. Non-reference points are the boundary points other than the reference points among the first, second, and third boundary points. Reference points and non-reference points whose distances are less than a preset distance threshold are obtained. The preset distance threshold is a pre-set indicator of the distance between non-reference points and reference points that are adjacent. The reference points and non-reference points whose distances are less than the preset distance threshold are regarded as multiple boundary points with adjacent relationships.

[0076] The first boundary point is a boundary point on the first plane, the second boundary point is a boundary point on the second plane, and the third boundary point is a boundary point on the third plane. The reference point is a boundary point used as a reference to detect distances between itself and other boundary points.

[0077] Specifically, the distance between the reference point and the non-reference points can be calculated by using the first boundary point as the reference point and the second and third boundary points as non-reference points. Similarly, the distance between the reference point and the non-reference points can be calculated by using the third boundary point as the reference point and the first and second boundary points as non-reference points.

[0078] Step S103: For multiple boundary points that are adjacent to each other, determine the foot of the perpendicular from each boundary point to the corresponding intersecting line.

[0079] The intersecting line between two adjacent boundary points is the line that intersects between those two boundary points.

[0080] Step S104: Based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines, find the target point closest to the endpoint from the 3D point cloud data; the maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines.

[0081] The point on the intersecting line where the two perpendiculars with the maximum perpendicular distance lie is the endpoint. The target point is the point in the 3D point cloud data that is closest to the endpoint.

[0082] As an optional implementation, based on the endpoint corresponding to the maximum perpendicular distance on intersecting lines, the target point closest to the endpoint is found from the 3D point cloud data, including: determining the distance between every two perpendiculars on the intersecting lines; comparing the distances between every two perpendiculars on the intersecting lines to obtain the maximum perpendicular distance; obtaining the endpoint corresponding to the maximum perpendicular distance on the intersecting lines; calculating the Euclidean distance between each point in the 3D point cloud data and the endpoint to obtain the distance between each point and the endpoint; and selecting the point in the 3D point cloud data with the largest distance to the endpoint as the target point.

[0083] Step S105: Correct the endpoints based on the target points, and identify the corrected points as weld feature points for right-angled workpieces; weld feature points are feature points at the weld of right-angled workpieces.

[0084] As an optional implementation, the endpoints are corrected based on the target points, and the corrected points are identified as weld feature points for right-angled workpieces. This includes: obtaining the three points closest to each other from the target points; performing sphere fitting on the three points closest to each other based on the minimum bounding sphere to obtain sphere data; extracting the center of the bounding sphere from the sphere data; and using the center of the bounding sphere and the target points as weld feature points for right-angled workpieces.

[0085] Specifically, the target points can include two target points corresponding to the endpoints of the first intersecting line, two target points corresponding to the endpoints of the second intersecting line, and two target points corresponding to the endpoints of the third intersecting line. The first intersecting line is the line between the first and second planes. The second intersecting line is the line between the second and third planes. The third intersecting line is the line between the first and third planes. The distance between every three target points can be detected, and the three closest target points are obtained. The distance between the three target points can be detected as follows: calculate the distance between every two target points, and use the sum of the distances between every two target points corresponding to the three target points as the distance between the three target points. The three closest points are located near the intersection of the three planes, while the other three points are far away from the intersection.

[0086] Can the center of the surrounding sphere reflect the characteristics of the weld seam in a right-angled workpiece? By using the target point and the center of the surrounding sphere as weld feature points, the weld feature points can be made more complete.

[0087] As an optional implementation, the endpoint is corrected based on the target point, and the corrected point is identified as a weld feature point for the right-angled workpiece. This includes: using the target point as a weld feature point for the right-angled workpiece to complete the correction of the endpoint.

[0088] As an optional implementation, after correcting the endpoints based on the target points and identifying the corrected points as weld feature points for the right-angled workpiece, the method further includes: welding the right-angled workpiece according to the weld feature points of the right-angled workpiece.

[0089] In this embodiment, for the 3D point cloud data of a right-angled workpiece, planes with intersecting straight lines are detected, and boundary points adjacent to other planes are extracted from the boundary points of the planes. Based on the perpendicular feet of these boundary points on the intersecting straight lines, the two farthest target perpendicular feet are obtained. Target points corresponding to the target perpendicular feet are obtained from the 3D point cloud data. Based on the target points, the endpoints corresponding to the target perpendicular feet are corrected to obtain the weld feature points of the right-angled workpiece. On the one hand, by processing the 3D data, the characteristics of the workpiece in three-dimensional space can be reflected, thereby improving the accuracy of weld feature point recognition compared to the extraction method using a 2D camera. On the other hand, processing the 3D data in the above manner improves the efficiency of weld feature point recognition compared to manual teaching that relies on manual operation.

[0090] Figure 2 A schematic flowchart of a weld identification method according to Embodiment 2 of this application is shown. The weld identification method includes:

[0091] Step S201: Obtain the original 3D point cloud data for the right-angled workpiece;

[0092] This implementation describes how to obtain more accurate 3D point cloud data so that the accuracy of the identified weld feature points is higher than that of weld identification using the original 3D point cloud data.

[0093] Step S202: Perform coordinate transformation on the original 3D point cloud data to obtain the actual 3D point cloud data in the coordinate system of the welding robot base;

[0094] As an optional implementation, coordinate transformation is performed on the original three-dimensional point cloud data to obtain the actual three-dimensional point cloud data in the welding robot base coordinate system. This includes: performing coordinate transformation on the original three-dimensional point cloud data based on the posture matrix of the center point of the welding robot end tool and the robot hand-eye matrix to obtain the actual three-dimensional point cloud data in the welding robot base coordinate system.

[0095] As an alternative implementation, the tool center point is the end point of the welding robot's welding torch.

[0096] Step S203: Cluster the actual 3D point cloud data to obtain 3D point cloud data;

[0097] As an optional implementation, clustering actual 3D point cloud data to obtain 3D point cloud data includes: obtaining the value range of the actual 3D point cloud data in a preset dimension; traversing each point in the actual 3D point cloud data and determining the value of the point in the preset dimension; the preset dimension is a pre-defined dimension used to indicate the direction towards the ground; obtaining all points whose values ​​are not within the value range; filtering out all points whose values ​​are not within the value range to obtain real 3D point cloud data; sampling the real 3D point cloud data to obtain sampled 3D point cloud data; clustering the sampled 3D point cloud data to obtain spatial noise points and target 3D point cloud data, and using the target 3D point cloud data as the 3D point cloud data.

[0098] True 3D point cloud data is 3D point cloud data with ground point cloud data removed. Sampled 3D point cloud data is a portion of true 3D point cloud data that has been sampled. Target 3D point cloud data is point cloud data from sampled 3D point cloud data, excluding spatial noise points.

[0099] Using the above method, ground point clouds and spatial noise points can be removed, resulting in more accurate 3D point cloud data of the workpiece. Furthermore, sampling reduces the computational cost of weld seam recognition, improving its efficiency compared to processing all points.

[0100] Step S204: If intersecting lines are detected between multiple planes based on 3D point cloud data, then for the planes with intersecting lines, extract multiple boundary points with adjacent relationships from the 3D point cloud data; multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes.

[0101] Step S205: For multiple boundary points that are adjacent to each other, determine the foot of the perpendicular from each boundary point to the corresponding intersecting line;

[0102] Step S206: Based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines, find the target point closest to the endpoint from the 3D point cloud data; the maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines.

[0103] Step S207: Correct the endpoints based on the target points, and identify the corrected points as weld feature points for right-angled workpieces; weld feature points are feature points at the weld of right-angled workpieces.

[0104] The following is for reference Figures 3 to 8The process of weld seam identification for a right-angled workpiece in a specific scenario is described. Figure 8 This is a flowchart of the steps involved in the weld identification process.

[0105] Acquiring original 3D point cloud data of right-angled workpieces: A 3D data camera is used to acquire point cloud data of the right-angled workpieces (pcd0).

[0106] The original 3D point cloud data is transformed into the robot base coordinate system: pcd0 is transformed into the welding robot base coordinate system using the toolPos pose matrix of the welding robot's end effector TCP and the robot's hand-eye matrix handEye, resulting in the actual point cloud pcd1 of the right-angled workpiece. The transformation relationship is as follows:

[0107] pcd1=transform(pcd0,toolPos*handEye).

[0108] Ground point cloud filtering: The ground point cloud in pcd1 that is not related to the right-angled workpiece is filtered out by a filter to obtain the real point cloud pcd2 of the right-angled workpiece.

[0109] Sampling of point cloud of right-angled workpiece: Uniform sampling of point cloud pcd2 of right-angled workpiece to obtain point cloud pcd3. This process can reduce the number of actual point clouds, thereby reducing the computing power for extracting weld seams of right-angled workpieces and improving the efficiency of weld seam extraction.

[0110] Clustering of point clouds of right-angled workpieces: Clustering and segmenting of the actual point cloud pcd3 of the right-angled workpieces, removing spatial noise points other than the actual point cloud of the right-angled workpieces, and retaining only the point cloud pcd4 related to the right-angle weld, so as to avoid interference with the extraction of the right-angle weld.

[0111] The point cloud of the right-angled workpiece is segmented into three independent point cloud planes: the actual point cloud pcd4 of the right-angled workpiece is segmented into point cloud planes plane1, plane2, and plane3. The relationship between plane1, plane2, and plane3 is as follows: Figure 3 As shown.

[0112] To obtain the intersecting lines between pairwise point clouds of a right-angled workpiece: traverse all segmented plane point clouds (plane1, plane2, and plane3), and determine whether there are intersecting lines between each pair. If so, calculate the corresponding line equations L1, L2, and L3. The relationship between lines L1, L2, and L3 is as follows: Figure 4 As shown.

[0113] Boundary point extraction is performed on three independent point cloud planes: the boundary points b1, b2, and b3 corresponding to the point clouds plane1, plane2, and plane3 are calculated respectively. The relationship between boundary points b1, b2, and b3 is as follows: Figure 5 As shown.

[0114] To obtain the point clouds of adjacent points between each pair of boundary points: traverse all boundary points b1, b2, and b3, and calculate the point clouds p1, p2, and p3 of each pair of adjacent boundary points. The relationships between the point clouds p1, p2, and p3 of each pair of adjacent boundary points are as follows: Figure 6 As shown.

[0115] Find the feet of the perpendiculars of adjacent points to the intersecting lines in the point cloud plane: Find the feet of the perpendiculars of adjacent boundary points p1, p2 and p3 to their corresponding intersecting lines L1, L2 and L3, respectively.

[0116] To find the endpoints of the perpendicular on the straight line and their correction points: Calculate the points pt1, pt2, pt3, pt4, pt5, and pt6 with the largest Euclidean distance between the feet of the perpendiculars on the straight line. These points are the endpoints of the weld. The Euclidean distance is calculated as follows:

[0117] d=sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2).

[0118] In the actual point cloud pcd4 of the right-angled workpiece, find the points pt1', pt2', pt3', pt4', pt5' and pt6' that have the closest Euclidean distance to the endpoints pt1, pt2, pt3, pt4, pt5 and pt6 respectively, correct the endpoints, and then use the minimum bounding sphere method to perform sphere fitting on pt4', pt5' and pt6' and find their sphere center cir0.

[0119] Output the final weld feature points: The final output includes the actual points pt1', pt2', pt3' and the perpendicular foot cir0 after the right-angle workpiece is corrected. The relationship between pt1', pt2', pt3', pt4', pt5', pt6' and the center cir0 of the sphere is as follows: Figure 7 As shown.

[0120] Using the above method can improve the accuracy and efficiency of identifying weld seams in right-angled workpieces.

[0121] Figure 9 A weld identification device according to Embodiment 3 of this application is shown. The device includes:

[0122] The acquisition module 301 is used to acquire the three-dimensional point cloud data of the right-angled workpiece;

[0123] The boundary point extraction module 302 is used to extract multiple boundary points with adjacent relationships from the 3D point cloud data if intersecting lines are detected between multiple planes based on 3D point cloud data; the multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes.

[0124] The perpendicular foot determination module 303 is used to determine the perpendicular foot of each boundary point on the corresponding intersecting line for multiple boundary points that have an adjacent relationship.

[0125] The target point search module 304 is used to find the target point closest to the endpoint in the 3D point cloud data based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines; the maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines.

[0126] The weld seam recognition module 305 is used to correct the endpoints based on the target point and identify the corrected points as weld seam feature points for right-angled workpieces; the weld seam feature points are feature points at the weld seam of right-angled workpieces.

[0127] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0128] From the target point, obtain the three points that are closest to each other;

[0129] Based on the minimum bounding sphere, sphere data is obtained by fitting a sphere to the three points that are closest to each other;

[0130] Extract the center of the bounding sphere from the sphere data;

[0131] The center of the surrounding sphere and the target point are used as the weld feature points for right-angled workpieces.

[0132] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0133] The 3D point cloud data is segmented into three planes to obtain a first plane, a second plane, and a third plane. The first plane is the plane parallel to the XY plane corresponding to the right-angled workpiece, the second plane is the plane parallel to the YZ plane corresponding to the right-angled workpiece, and the third plane is the plane parallel to the XZ plane corresponding to the right-angled workpiece. The XY plane is the plane formed by the X-axis and the Y-axis, the YZ plane is the plane formed by the Y-axis and the Z-axis, and the XZ plane is the plane formed by the X-axis and the Z-axis.

[0134] For the first plane, the second plane, and the third plane, detect whether there are intersecting straight lines between any two planes;

[0135] If there are intersecting lines between every two planes, then extract the first boundary point of the first plane, the second boundary point of the second plane, and the third boundary point of the third plane from the 3D point cloud data.

[0136] Using the first boundary point, the second boundary point, and the third boundary point as reference points, the distances between the reference points and non-reference points are detected to obtain the distance between each reference point and a non-reference point. Non-reference points are the boundary points other than the reference points among the first, second, and third boundary points.

[0137] Acquire reference points that are less than a preset distance threshold and non-reference points that are less than a preset distance threshold; the preset distance threshold is a pre-defined indicator of the distance between non-reference points and adjacent reference points;

[0138] Reference points whose distance is less than a preset distance threshold and non-reference points whose distance is less than a preset distance threshold are considered as multiple boundary points with an adjacent relationship.

[0139] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0140] Determine the distance between the feet of every two perpendiculars on the intersecting lines;

[0141] Compare the distances between every two feet of the perpendiculars on the intersecting lines to obtain the maximum perpendicular distance;

[0142] Find the endpoints corresponding to the maximum perpendicular distance on intersecting lines;

[0143] The Euclidean distance between each point and the endpoint in the 3D point cloud data is calculated to obtain the distance between each point and the endpoint.

[0144] The point in the 3D point cloud data that is furthest from the endpoint is taken as the target point.

[0145] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0146] Obtain the original 3D point cloud data for the right-angled workpiece;

[0147] The original 3D point cloud data is transformed to obtain the actual 3D point cloud data in the coordinate system of the welding robot base.

[0148] Clustering is performed on actual 3D point cloud data to obtain 3D point cloud data.

[0149] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0150] Obtain the value range of actual 3D point cloud data in a preset dimension;

[0151] Iterate through each point in the actual 3D point cloud data and determine the value of the point in the preset dimension; the preset dimension is a pre-defined dimension used to indicate the direction towards the ground.

[0152] Get all points whose values ​​are outside the range of possible values;

[0153] All points whose values ​​are outside the range are filtered out to obtain the true 3D point cloud data;

[0154] Sampled 3D point cloud data is obtained by sampling real 3D point cloud data.

[0155] Clustering is performed on the sampled 3D point cloud data to obtain spatial noise points and target 3D point cloud data, and the target 3D point cloud data is used as the 3D point cloud data.

[0156] In one exemplary embodiment of this application, the weld identification device is configured as follows:

[0157] Based on the posture matrix of the center point of the welding robot's end tool and the robot's hand-eye matrix, coordinate transformation is performed on the original 3D point cloud data to obtain the actual 3D point cloud data in the welding robot's base coordinate system.

[0158] The following is for reference. Figure 10 To describe the welding robot 40 according to Embodiment 4 of this application. Figure 10 The welding robot 40 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0159] like Figure 10 As shown, the welding robot 40 is represented in the form of a general-purpose computing device. The components of the welding robot 40 may include, but are not limited to: at least one processing unit 410, at least one storage unit 420, and a bus 430 connecting different system components (including storage unit 420 and processing unit 410).

[0160] The storage unit stores program code, which can be executed by the processing unit 410 to perform the steps described in the explanatory section of this specification, according to various exemplary embodiments of the present application. For example, the processing unit 410 can perform actions such as... Figure 1 The steps shown are as follows.

[0161] Storage unit 420 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 4201 and / or cache memory 4202, and may further include a read-only memory (ROM) 4203.

[0162] Storage unit 420 may also include a program / utility 4204 having a set (at least one) program module 4205, such program module 4205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0163] Bus 430 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0164] The welding robot 40 can also communicate with one or more devices that enable users to interact with it, and / or with any device that enables it to communicate with one or more other computing devices (e.g., routers, modems, etc.). This communication can be achieved through an input / output (I / O) interface 450, which is connected to a display unit 440. Furthermore, the welding robot 40 can also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 460. As shown, the network adapter 460 communicates with other modules of the welding robot 40 via a bus 430. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with the welding robot 40, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0165] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (including a welding robot) to execute the method according to the embodiments of this application.

[0166] In an exemplary embodiment of this application, a computer-readable storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a computer's processor, cause the computer to perform the methods described in the above method embodiments.

[0167] According to one embodiment of this application, a program product for implementing the methods in the above-described method embodiments is also provided. This program product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a welding robot. However, the program product of this application is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0168] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0169] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0170] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0171] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as JAVA and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0172] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0173] Furthermore, although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0174] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (including a welding robot) to execute the method according to the embodiments of this application.

[0175] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the appended claims.

Claims

1. A method for weld identification, characterized in that, include: Obtain the 3D point cloud data of the right-angled workpiece; If, based on the three-dimensional point cloud data, intersecting lines are detected between multiple planes, then for the planes with intersecting lines, multiple boundary points with adjacent relationships are extracted from the three-dimensional point cloud data; the multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes. For the multiple boundary points that are adjacent to each other, determine the foot of the perpendicular from each boundary point to the corresponding intersecting line; Based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines, find the target point closest to the endpoint from the three-dimensional point cloud data; The maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines; From the target point, obtain the three points that are closest to each other; based on the minimum bounding sphere, perform sphere fitting on the three points that are closest to each other to obtain sphere data; extract the center of the bounding sphere from the sphere data; use the center of the bounding sphere and the target point as the weld feature points for the right-angled workpiece; the weld feature points are the feature points at the weld of the right-angled workpiece.

2. The method according to claim 1, characterized in that, If, based on the 3D point cloud data, intersecting lines are detected between multiple planes, then for the planes with intersecting lines, multiple boundary points with adjacent relationships are extracted from the 3D point cloud data, including: The three-dimensional point cloud data is segmented into planes to obtain a first plane, a second plane, and a third plane. The first plane is the plane corresponding to the right-angled workpiece and parallel to the XY plane. The second plane is the plane corresponding to the right-angled workpiece and parallel to the YZ plane. The third plane is the plane corresponding to the right-angled workpiece and parallel to the XZ plane. The XY plane is the plane formed by the X-axis and the Y-axis. The YZ plane is the plane formed by the Y-axis and the Z-axis. The XZ plane is the plane formed by the X-axis and the Z-axis. For the first plane, the second plane, and the third plane, detect whether there are intersecting straight lines between any two planes; If there are intersecting lines between every two planes, then extract the first boundary point of the first plane, the second boundary point of the second plane, and the third boundary point of the third plane from the three-dimensional point cloud data. Using the first boundary point, the second boundary point, and the third boundary point as reference points, the distances between the reference points and non-reference points are detected to obtain the distance between each reference point and the non-reference point; the non-reference points are the boundary points other than the reference points among the first boundary point, the second boundary point, and the third boundary point. The system acquires reference points whose distance is less than a preset distance threshold and non-reference points whose distance is less than the preset distance threshold; the preset distance threshold is a pre-set distance indicating the distance between the non-reference points and the reference points. The reference points whose distance is less than a preset distance threshold and the non-reference points whose distance is less than the preset distance threshold are used as multiple boundary points that have an adjacent relationship.

3. The method according to claim 1, characterized in that, Based on the endpoint corresponding to the maximum perpendicular distance on the intersecting lines, the target point closest to the endpoint is found from the 3D point cloud data, including: Determine the distance between every two feet of the intersecting lines; The maximum perpendicular distance is obtained by comparing the distances between every two perpendicular feet on the intersecting lines. Obtain the endpoint corresponding to the maximum perpendicular distance on the intersecting lines; The Euclidean distance between each point in the three-dimensional point cloud data and the endpoint is calculated to obtain the distance between each point and the endpoint. The point in the 3D point cloud data that has the smallest distance to the endpoint is taken as the target point.

4. The method according to claim 1, characterized in that, Obtain the 3D point cloud data of the right-angled workpiece, including: Obtain the original three-dimensional point cloud data for the right-angled workpiece; The original three-dimensional point cloud data is transformed to obtain the actual three-dimensional point cloud data in the coordinate system of the welding robot base. The actual 3D point cloud data is clustered to obtain the 3D point cloud data.

5. The method according to claim 4, characterized in that, Clustering the actual 3D point cloud data to obtain the 3D point cloud data includes: Obtain the value range of the actual 3D point cloud data in a preset dimension; Iterate through each point in the actual 3D point cloud data and determine the value of the point in the preset dimension; the preset dimension is a pre-defined dimension used to indicate the direction towards the ground. Get all points whose values ​​are not within the range of the given values; Filter out all points whose values ​​are not within the range to obtain real 3D point cloud data; The real 3D point cloud data is sampled to obtain sampled 3D point cloud data; Clustering is performed on the sampled 3D point cloud data to obtain spatial noise points and target 3D point cloud data, and the target 3D point cloud data is used as the 3D point cloud data.

6. The method according to claim 4, characterized in that, The original 3D point cloud data is transformed to obtain the actual 3D point cloud data in the base coordinate system of the welding robot, including: Based on the posture matrix of the center point of the welding robot's end tool and the robot's hand-eye matrix, the original three-dimensional point cloud data is transformed to obtain the actual three-dimensional point cloud data in the welding robot's base coordinate system.

7. A weld seam identification device, characterized in that, include: The acquisition module is used to acquire the 3D point cloud data of the right-angled workpiece; The boundary point extraction module is used to extract multiple boundary points with adjacent relationships from the three-dimensional point cloud data if intersecting lines are detected between multiple planes based on the three-dimensional point cloud data; the multiple boundary points with adjacent relationships are multiple boundary points that are adjacent to each other between the boundary points of two planes. The perpendicular foot determination module is used to determine the perpendicular foot of each of the multiple boundary points that have an adjacent relationship on the corresponding intersecting lines. The target point search module is used to search for the target point closest to the endpoint in the three-dimensional point cloud data based on the endpoint corresponding to the maximum perpendicular distance on the intersecting straight line. The maximum perpendicular distance is the maximum distance between any two perpendiculars on the intersecting lines; The weld seam recognition module is used to obtain the three closest points to each other from the target point; perform sphere fitting on the three closest points based on the minimum bounding sphere to obtain sphere data; extract the center of the bounding sphere from the sphere data; and use the center of the bounding sphere and the target point as weld seam feature points for the right-angled workpiece; the weld seam feature points are feature points at the weld seam of the right-angled workpiece.

8. A welding robot, characterized in that, include: One or more processors; A storage device for storing one or more programs that, when executed by one or more processors, cause the welding robot to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores computer-readable instructions that, when executed by the computer's processor, cause the computer to perform the method of any one of claims 1 to 6.

Images