Welding seam positioning method based on surface structure matching

Abstract

The invention provides a welding seam positioning method based on surface structure matching. The method comprises the following steps: acquiring reference surface structure information and reference welding seam information of a to-be-detected welding seam in a target workpiece point cloud; and performing surface structure detection on the target workpiece point cloud, and performing surface structure filtering based on the reference surface structure information. And traversing each input welding seam obtained by combining the filtered surface structures, and filtering the input welding seam based on the reference welding seam information of the welding seam to be detected and the welding seam type of the input welding seam. Wherein when repeated linear welding seams in the input welding seams are filtered, filtering is carried out based on the relative position relation of the repeated linear welding seams. According to the scheme, the detection effect is remarkably improved by using surface structure information matching, the system can process all potential welding seams in the point cloud at the same time, and the data utilization rate and the detection efficiency are greatly improved. In addition, for repeated linear welding seams, the repeated linear welding seams can be distinguished through the relative position relation, and the application flexibility in a complex welding scene is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional point cloud processing, in particular to a welding seam positioning method based on surface structure matching. BACKGROUND

[0002] A modern intelligent welding system mainly consists of two core modules, a mechanical arm and a visual detection system. The visual system acquires workpiece images in real time through an industrial camera, accurately identifying the welding seam position information. The mechanical arm system has a dual function, adjusting the camera pose according to the visual feedback, and performing the final welding task. In the visual detection link, the system generally adopts a welding seam positioning method based on prior knowledge. The specific implementation process is as follows: first, three-dimensional feature extraction is performed on the two point cloud data at the start and end points of the welding seam, and the planes and cylindrical surfaces are identified; then, through spatial geometric relationship analysis, all possible surface structures are obtained; finally, the surface structure filtering is performed in combination with the welding seam number and the corresponding welding seam prior information, and the intersection point of the surface structure intersection line is taken as the start and end position coordinates of the welding seam path.

[0003] The method adopted in the prior art has obvious efficiency bottleneck. In the detection process, the system relies on the prior information of a specific welding seam for surface structure filtering, which makes the system only detect a single specific welding seam in a single detection, seriously restricting the overall detection efficiency.

[0004] In addition, the existing method cannot distinguish repeated straight-line welds with completely consistent surface structure information. When processing repeated straight-line welds with completely consistent surface structure characteristics, the system cannot effectively distinguish them. This is because the system completely relies on the same geometric parameters as prior information, making it impossible to identify repeated welds with different spatial positions but completely consistent geometric characteristics during the detection process. This technical defect seriously restricts the application effect of the system in complex welding scenarios. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a welding seam positioning method based on surface structure matching, which can improve the detection efficiency and distinguish repeated straight-line welds.

[0006] In a first aspect, the present application provides a welding seam positioning method based on surface structure matching, which comprises:

[0007] Obtaining a target workpiece point cloud, and obtaining reference surface structure information and reference welding seam information of a welding seam to be detected in the target workpiece point cloud;

[0008] Determining repeated straight-line welds and their corresponding relative position relationships based on the reference welding seam in the reference welding seam information;

[0009] The target workpiece point cloud is subjected to surface structure detection, and the detected surface structure is filtered based on the reference surface structure in the reference surface structure information to obtain the filtered surface structure.

[0010] Each input weld obtained by traversing the filtered surface structure combination is determined, the weld type of the input weld is determined, and the input weld is filtered based on the reference weld information of the weld to be detected and the weld type of the input weld.

[0011] Specifically, when filtering for repeating straight welds in the input weld seam, the filtering is based on the relative positional relationship of the repeating straight welds.

[0012] In an optional implementation, the step of determining the repeating straight weld and its corresponding relative positional relationship based on the reference weld in the reference weld information includes:

[0013] For the current reference straight weld in the reference weld information, find the reference straight weld in the reference weld information that has the same and parallel surface structure information as the endpoint of the current reference straight weld, and use it as the repeating straight weld.

[0014] Calculate the relative positional relationships of repeating straight welds.

[0015] In an optional implementation, the step of calculating the relative positional relationship of repeating straight welds includes:

[0016] For each of the repeated straight welds, the camera coordinate TCP is connected to the endpoint of the repeated straight weld, and a reference surface is constructed based on the connection and the direction vector of the repeated straight weld.

[0017] Obtain the positional relationship between other repeating straight welds and the reference surface, and obtain the relative positional relationship of the repeating straight welds based on the positional relationship of the other repeating straight welds.

[0018] In an optional implementation, the detected surface structure type is a three-plane structure, a two-plane structure, a three-cylinder structure, or a two-cylinder structure.

[0019] The step of filtering the detected surface structure based on the reference surface structure in the reference surface structure information includes:

[0020] For each detected surface structure, a reference surface structure belonging to the same surface structure type as the detected surface structure is determined based on the surface structure type of each reference surface structure in the reference surface structure information;

[0021] The reference surface structures belonging to the same surface structure type are matched with the surface structure, and surface structure filtering is performed according to the filtering conditions corresponding to the surface structure type.

[0022] In an optional implementation, the weld type of the weld to be inspected and the input weld is a straight weld type;

[0023] The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes:

[0024] For each of the input welds, obtain the surface structure corresponding to the two endpoints of the input weld, perform the operation of projecting the endpoints of the input welds onto the surface structure, and perform input weld filtering and input weld endpoint merging based on the projection result;

[0025] The direction of the weld intersection line is calculated using the retained weld start and weld end points, and the angle between the weld intersection line direction and the weld direction of the reference weld of the weld to be inspected is calculated. Input welds corresponding to weld start and weld end points with an angle greater than a preset angle are filtered out.

[0026] The system detects whether there is a three-plane structure in the point cloud of the target workpiece that overlaps with the retained input weld. If so, the overlapping input weld is filtered out.

[0027] If there are repeated straight welds in the reference welds corresponding to the weld to be detected, the relative positional relationship between the retained input welds is detected, and filtering is performed based on the relative positional relationship and the relative positional relationship between the reference weld and its repeated straight welds.

[0028] If the number of retained input welds is greater than 1, calculate the weld length of each input weld, obtain the difference between the weld length and the length of the reference weld of the weld to be inspected, and retain the input weld with the smallest difference.

[0029] In an optional implementation, the steps of performing the operation of projecting the endpoints of the input weld onto the surface structure, and performing input weld filtering and input weld endpoint merging based on the projection results, include:

[0030] The endpoint of the intersection line of the surface structure corresponding to the end point of the input weld is projected onto the extension line of the intersection line in the surface structure corresponding to the start point of the weld to obtain the projection distance. Input welds corresponding to the start point and end point of the weld with projection distance greater than the preset distance are filtered out.

[0031] The retained weld start and end points are clustered, and weld start and end points belonging to the same cluster are merged to obtain the merged input weld.

[0032] In an optional implementation, the step of detecting whether there is a three-plane structure in the target workpiece point cloud that overlaps with the retained input weld seam includes:

[0033] For each retained input weld, calculate the angle between the weld direction of the input weld and the intersection line of each three-plane structure detected in the point cloud of the target workpiece, and filter out the three-plane structures with an angle smaller than the first preset angle.

[0034] Calculate the angle between the plane normal of the input weld and the plane normal of each selected three-plane structure, and select the three-plane structure with an angle smaller than the second preset angle;

[0035] Calculate the projection distance from the common endpoint of the intersection line in each of the selected three-plane structures to the input weld, and select the three-plane structures whose projection distance is less than the preset distance;

[0036] Determine the vertical plane in the selected three-plane structure, and project the endpoint of the input weld onto the vertical plane to obtain the projection vector;

[0037] Based on the projection vectors of the two endpoints of the input weld, it is determined whether the two endpoints are on the same side of the vertical plane. If they are not on the same side of the vertical plane, it is determined that the input weld overlaps with the three-plane structure.

[0038] In an optional implementation, the step of filtering based on the relative positional relationship between the retained input welds and the relative positional relationship between the reference weld and its repeating straight welds includes:

[0039] Determine whether there is collinearity among the reference welds of the repeating straight welds of the weld to be inspected. If there is collinearity, classify the collinear reference welds into the same reference weld group, and generate the relative positional relationship between the reference welds and their repeating straight welds based on the relative positional relationship between the reference welds and their repeating straight welds.

[0040] Obtain the relative positional relationship between the retained input welds, classify the collinear input welds into the same input weld group, and obtain the relative positional relationship between the input weld groups;

[0041] If the input weld group and the reference weld group have the same array, match the input weld group and the reference weld group.

[0042] Calculate the length difference between the corresponding weld in the matched input weld group and the reference weld group, retain the input weld with the smallest difference, and filter out the other input welds.

[0043] In an optional implementation, the weld type of the weld to be inspected and the input weld is a full-circle weld type;

[0044] The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes:

[0045] The radius and center of the reference weld are obtained based on the reference weld information of the weld to be inspected;

[0046] Obtain the surface structure corresponding to the endpoints of the input weld;

[0047] Traverse all the biplane cylindrical structures in the obtained surface structure and retain the biplane cylindrical structure whose radius is closest to the reference weld.

[0048] The center, radius, and axis of the retained two-sided cylindrical structure are detected, and the output point of the retained input weld is obtained based on the center, radius, and axis.

[0049] In an optional implementation, the weld type of the weld to be inspected and the input weld is a notched circular weld type;

[0050] The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes:

[0051] Obtain the starting point surface structure and ending point surface structure corresponding to the starting point and ending point of each input weld;

[0052] Combine all starting surface structures and ending surface structures, and calculate the difference between the cylindrical radius of the starting surface structure and the cylindrical radius of the ending surface structure in each combination, filtering out combinations whose difference is greater than the preset difference;

[0053] For each retained combination, calculate the distance between the center of the cylindrical surface of the starting surface structure and the center of the cylindrical surface of the ending surface structure in each combination, and filter out combinations whose distance is greater than the preset distance;

[0054] Based on the surface structure types of the starting and ending surface structures in each retained combination, the combinations are filtered.

[0055] Select the face structure with the most cylindrical points from the filtered combinations, and use the information of this face structure as the retained input weld circle information;

[0056] The surface structure information of the reference weld in the reference weld information of the weld to be inspected is obtained, and the output point and notch point of the retained input weld are obtained based on the surface structure information.

[0057] This invention provides a weld location method based on surface structure matching. It acquires a point cloud of a target workpiece and obtains reference surface structure information and reference weld information for the weld to be detected within the point cloud. Based on the reference weld information, it determines repeating straight welds and their corresponding relative positional relationships. Surface structure detection is performed on the target workpiece point cloud, and the detected surface structures are filtered based on the reference surface structure information. Each input weld obtained from the filtered surface structure combinations is traversed to determine the weld type of the input weld. The input welds are then filtered based on the reference weld information and the weld type. Specifically, when filtering repeating straight welds among the input welds, the filtering is based on the relative positional relationships of the repeating straight welds.

[0058] In this solution, the detection effect is significantly improved by using surface structure information matching. The system can simultaneously process all potential welds in the point cloud, greatly improving data utilization and detection efficiency. In addition, for repetitive straight welds, the system can distinguish them by their relative positional relationship, increasing the flexibility of application in complex welding scenarios. Attached Figure Description

[0059] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 Schematic diagrams of various types of surface structures;

[0061] Figure 2 Schematic diagrams of various types of welds;

[0062] Figure 3 This is a schematic diagram of the main surface structure and auxiliary surface structure of the weld.

[0063] Figure 4 A schematic diagram of a repeating straight weld.

[0064] Figure 5 A flowchart illustrating the weld positioning method based on surface structure matching provided in an embodiment of the present invention;

[0065] Figure 6 This is a logical diagram of the initialization process in an embodiment of the present invention;

[0066] Figure 7 This is a logic diagram of detecting repetitive straight welds in an embodiment of the present invention;

[0067] Figure 8 Schematic diagrams of non-collinear and collinear weld seams;

[0068] Figure 9 This is a logic diagram illustrating the detection of the relative positional relationship of repeating straight welds in an embodiment of the present invention;

[0069] Figure 10 This is a schematic diagram of the surface structure filtering logic in an embodiment of the present invention;

[0070] Figure 11 This is a schematic diagram of the overall logic of weld seam filtering in an embodiment of the present invention;

[0071] Figure 12 This is a logic diagram of the straight weld seam filtering in an embodiment of the present invention;

[0072] Figure 13 This is a schematic diagram illustrating the logic for detecting whether a weld is cut off by a three-plane structure in an embodiment of the present invention.

[0073] Figure 14 This is a logic diagram of the repeated straight weld seam filtering in an embodiment of the present invention;

[0074] Figure 15 This is a logic diagram of the full-circle weld seam filtering in an embodiment of the present invention;

[0075] Figure 16 This is a logic diagram of the notched circular weld seam filtering in an embodiment of the present invention. Detailed Implementation

[0076] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0077] The following section first explains the definitions of some of the terms and concepts involved in the embodiments of this invention.

[0078] Surface structure: refers to a surface structure composed of the intersection of cylindrical surfaces and planes, or planes and planes, such as... Figure 1 As shown.

[0079] For ease of description, the planes and cylinders that make up the cylindrical structure are named plane ①, plane ②, and cylinder, respectively. The normal vector of plane ① is parallel to the axis of the cylinder, and the normal vector of plane ② is perpendicular to the axis of the cylinder.

[0080] Weld type: such as Figure 2As shown, welds can be classified into three types: straight welds, full-circle welds, and notched circular welds. Inspecting straight welds requires examining the cylindrical or planar structures corresponding to the two endpoints of the weld. Inspecting full-circle welds requires examining any segment of the cylindrical structure on the full circle. Inspecting notched circular welds requires examining the two cylindrical structures corresponding to the endpoints of the notch. For planar structures, the plane to be inspected must have a sufficient area; for cylindrical structures, the cylinder must have a sufficiently large central angle.

[0081] Main surface structure and auxiliary surface structure: such as Figure 3 As shown, the main surface structure refers to the primary form of the surface structure corresponding to the weld endpoint, including three-plane structures, three-cylinder structures, etc. The auxiliary surface structure refers to the surface structure that may correspond to the weld endpoint; the intersection line of the auxiliary surface structures should include the weld. Generally, auxiliary surface structures exist only when the main surface structure is a three-plane structure, and the auxiliary surface structure is a two-plane structure derived from the three-plane structure. Two-plane structures do not have corresponding auxiliary structures.

[0082] Repeating straight welds and relative position relationship groups: When parallel straight welds have the same endpoint surface structure information, the two straight welds are repeating straight welds in a single workpiece, and an additional relative position relationship group is needed to distinguish them. The relative position relationship can be represented by irrelevant or self (-1), upper side (0), lower side (1), left side (2), right side (3), etc.

[0083] Figure 4 There are four parallel straight welds, each with a different set of relative positions. Welds 1, 2, and 3 have a three-sided structure at one end and a two-sided structure at the other. Weld 4 has two-sided structures at both ends. Welds 1, 2, 3, and 4 are all parallel to each other. For weld 1, welds 2 and 3 are repeating straight welds, with weld 3 to the right of weld 1 and weld 2 to the bottom of weld 1. Therefore, the relative positions of welds 1 to 4 can be represented as -1, 1, 3, -1. For weld 2, welds 1 and 3 are repeating straight welds, with welds 1 and 3 to the top of weld 2. Therefore, the relative positions can be represented as 0, -1, 0, -1. For weld 3, welds 1 and 2 are repeating straight welds, with weld 1 to the left of weld 3 and weld 2 to the bottom of weld 3. Therefore, the relative positions can be represented as 2, 1, -1, -1. Weld 4 is unique, as both ends have a two-sided structure. The starting face structure is consistent with the starting auxiliary face structure of welds 1, 2, and 3, while the ending face structure is consistent with the ending main face structure of the other welds. Therefore, for weld 4, welds 1, 2, and 3 are all repeating straight welds, with welds 1 and 3 on the upper side of weld 4 and weld 2 on the left side of weld 4. The relative relationship can be represented as 0, 2, 0, -1.

[0084] Please see Figure 5The above is a flowchart of a weld positioning method based on surface structure matching provided in an embodiment of the present invention. This weld positioning method can be executed by a weld positioning device based on surface structure matching. This device can be implemented by software and / or hardware and can be configured in an electronic device, such as a computer, server, tablet computer, etc. The detailed steps of this weld positioning method based on surface structure matching are described below.

[0085] S11, acquire the point cloud of the target workpiece, and acquire the reference surface structure information and reference weld information of the weld to be inspected in the point cloud of the target workpiece.

[0086] S12, determine the repeating straight weld and its corresponding relative position relationship based on the reference weld in the reference weld information.

[0087] S13, perform surface structure detection on the point cloud of the target workpiece, and filter the detected surface structure based on the reference surface structure in the reference surface structure information to obtain the filtered surface structure.

[0088] S14. Traverse each input weld obtained from the filtered surface structure combination, determine the weld type of the input weld, and filter the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld. When filtering the repeated straight welds in the input weld, the filtering is based on the relative position relationship of the repeated straight welds.

[0089] In this embodiment, the target workpiece point cloud is a three-dimensional point cloud obtained by photographing the target workpiece. When performing a weld operation on the target workpiece, it is necessary to determine the specific weld information of the weld to be inspected in the target workpiece point cloud, including information indicating the direction and length of the weld.

[0090] Based on the modeling method for the target workpiece, the reference surface structure information and reference weld information of the weld to be inspected can be obtained based on the modeling information. The reference weld information is the information of the reference weld structure corresponding to the weld to be inspected in the modeling, and the reference surface structure information is the information of the surface structure to which the reference weld belongs.

[0091] Since there are differences between the information obtained from modeling and the information obtained from actual shooting, the solution in this embodiment aims to determine the weld information corresponding to the weld to be inspected in the point cloud of the target workpiece based on the reference surface structure information and reference weld information of the weld to be inspected obtained from modeling.

[0092] In this embodiment, after obtaining the reference surface structure and reference weld information of the weld to be inspected, preprocessing can be performed on the reference surface structure and reference weld information, which can also be understood as initialization processing.

[0093] Please refer to the following:Figure 6 The input is a weld database (reference weld information) and a list of reference surface structures (reference surface structure information). Preprocessing may include obtaining the surface structure information corresponding to all welds to be inspected from the reference surface structure information. For example, it includes the main reference surface structure and auxiliary reference surface structure information corresponding to the endpoints of all welds to be inspected, including the intersection direction of the reference surface structure, the plane normal, etc. (the cylindrical surface structure information also includes the cylindrical surface center radius, the relationship between plane ② and the cylindrical surface).

[0094] Furthermore, obtaining specific information from the reference weld information allows for the determination of repeating straight welds and their corresponding relative positional relationships based on the reference welds within the reference weld information. Specifically, this can be achieved in the following ways:

[0095] For the current reference straight weld in the reference weld information, find the reference straight weld in the reference weld information that has the same surface structure information as the endpoint of the current reference straight weld and is parallel to it, and use it as the repeating straight weld; calculate the relative positional relationship of the repeating straight weld.

[0096] Please refer to the following: Figure 7 In the repetitive straight weld inspection process, the reference weld information includes reference straight welds, based on a list of reference straight welds and a list of camera coordinates used when the weld was captured. Each reference straight weld is iterated over, starting with i = 0, and processing is performed on the i-th reference straight weld. If i is less than the total number of reference straight welds, reference straight welds with the same endpoint surface structure as the i-th reference straight weld are filtered out, as are reference straight welds parallel to the i-th reference straight weld. Then, the relative positional relationships between the filtered repetitive straight welds are obtained.

[0097] In the process of calculating relative positional relationships, these relationships can be divided into two types: collinear and non-collinear. Figure 8 As shown, specifically, it can be achieved in the following ways:

[0098] For each repeating straight weld, connect the camera coordinate TCP and the endpoint of the repeating straight weld, and construct a reference plane based on the connection and the direction vector of the repeating straight weld; obtain the positional relationship between other repeating straight welds and the reference plane, and obtain the relative positional relationship of the repeating straight weld based on the positional relationship of other repeating straight welds.

[0099] Please refer to the following: Figure 9 Connect the camera coordinates TCP used during shooting to the endpoint of the current repeating straight weld (the i-th repeating straight weld), and use this connection and the weld direction vector to form a reference surface, and calculate the plane normal of the reference surface.

[0100] Determine whether the endpoint of each other repeating straight weld is on the positive direction, negative direction, or reference plane of the reference plane, and thus determine whether each other repeating straight weld is on the "upper side", "lower side", or collinear with the current repeating straight weld.

[0101] If other repeating straight welds are collinear with the current repeating straight weld, the endpoint of the current repeating straight weld is used as the starting point, and the endpoints of the other collinear repeating straight welds are used as the ending points. It is then determined whether all the constructed reference directions are the same as the weld direction. If they are the same, the repeating straight weld is determined to be "to the left" of the current reference straight weld; otherwise, the repeating straight weld is determined to be "to the right" of the current reference straight weld. If all the constructed reference directions are opposite to each other, then the other repeating straight welds overlap with the current repeating straight weld, indicating an error in the reference weld data. In this case, manual intervention is required for correction.

[0102] By using the above methods, all repeating straight welds and their corresponding relative positions can be obtained.

[0103] Based on the above, the surface structure search, surface structure filtering, and weld seam filtering in the target workpiece point cloud are then executed in sequence to finally determine the weld seam corresponding to the weld seam to be detected.

[0104] The surface structure search includes planar search and cylindrical search. The surface structure is determined based on the searched planes and cylinders. After determining all surface structures in the target workpiece point cloud, the surface structure is filtered based on the reference surface structure information of the weld to be inspected.

[0105] The detected surface structure types are three-plane structure, two-plane structure, three-cylinder structure, or two-cylinder structure.

[0106] When filtering the detected surface structure based on the reference surface structure information, the following method can be used:

[0107] For each detected surface structure, a reference surface structure belonging to the same surface structure type as the surface structure is determined based on the surface structure type of each reference surface structure in the reference surface structure information; the reference surface structures belonging to the same surface structure type are matched with the surface structure, and surface structure filtering is performed according to the filtering conditions corresponding to the surface structure type.

[0108] Please refer to the following: Figure 10The system performs filtering based on the surface structure type of the reference surface structure, including three-plane structure filtering, two-plane structure filtering, three-cylinder structure filtering, and two-cylinder structure filtering. Filtering conditions include the concavity / convexity of the surface structure, the direction of the intersection line, the direction of the plane normal, the length of the intersection line, the axial direction of the cylinder, the distance from the endpoint of the intersection line to the reference plane, the center and radius of the cylinder, and the relative positional relationship between plane ② and the cylinder in the cylindrical structure. Surface structure filtering is performed using corresponding filtering conditions based on the different surface structure types.

[0109] The remaining surface structure after filtering can be understood as a surface structure that matches the reference surface structure in the reference surface structure information.

[0110] The remaining surface structures after filtering are combined to obtain the input weld (one weld corresponds to two endpoints, one endpoint corresponds to one surface structure, and two surface structures are combined to obtain one weld). The weld corresponding to the weld to be detected exists in these input welds, and the weld corresponding to the weld to be detected needs to be determined from the input welds.

[0111] Please refer to the following: Figure 11 In the overall weld filtering process, the reference welds determined in the initialization process can be traversed. It is determined whether the reference surface structures corresponding to the two endpoints of each reference weld have a matching surface structure among the remaining surface structures after filtering. If not, the next reference weld can be checked. If a matching surface structure exists, the input weld can be filtered based on the current reference weld. During filtering, different filtering methods are used based on the weld type.

[0112] In this embodiment, different filtering methods are used for different weld types when performing weld filtering. The weld types include straight weld type, full circle weld type and notched circle weld type.

[0113] In this embodiment, when both the weld to be detected and the input weld are of the straight weld type, the input weld is filtered based on the reference weld information of the weld to be detected and the weld type of the input weld. This can be achieved in the following way:

[0114] For each input weld, obtain the surface structure corresponding to the two endpoints of the input weld, perform the operation of projecting the endpoints of the input weld onto the surface structure, and perform input weld filtering and input weld endpoint merging based on the projection results;

[0115] The direction of the weld intersection line is calculated using the retained weld start and weld end points. The angle between the weld intersection line direction and the weld direction of the reference weld of the weld to be inspected is calculated. Input welds corresponding to weld start and weld end points with an angle greater than the preset angle are filtered out.

[0116] The system detects whether there are three-plane structures in the point cloud of the target workpiece that overlap with the retained input weld. If such structures exist, the overlapping input welds are filtered out.

[0117] If there are repeated straight welds in the reference weld corresponding to the weld to be detected, the relative positional relationship between the input welds that are detected and retained is filtered based on the relative positional relationship and the relative positional relationship between the reference weld and its repeated straight welds.

[0118] If the number of retained input welds is greater than 1, calculate the weld length of each input weld, obtain the difference between the weld length and the length of the reference weld of the weld to be inspected, and retain the input weld with the smallest difference.

[0119] Please refer to the following: Figure 12 This is the overall process for filtering straight weld seams. For each input weld seam, the endpoints of the surface structures corresponding to the weld seam start and end points are identified using the weld seam direction. Then, weld seam projection filtering and close-endpoint merging are performed. Specifically, the endpoints of the input weld seams are projected onto the surface structures, and based on the projection results, input weld seam filtering and endpoint merging are performed. This can be achieved in the following way:

[0120] Project the endpoint of the intersection line of the surface structure corresponding to the end point of the input weld onto the extension line of the intersection line in the surface structure corresponding to the start point of the weld to obtain the projection distance. Filter the input welds corresponding to the start point and end point of the weld that have a projection distance greater than a preset distance. Perform clustering processing on the retained start point and end point of the weld, and merge the start point and end point of the weld that belong to the same cluster to obtain the merged input weld.

[0121] Considering the slight errors that may occur when the welding system photographs the same weld endpoint from different angles, this embodiment uses clustering to eliminate interfering detection points in order to improve detection accuracy.

[0122] Based on this, filtering based on weld angle is performed. For the filtered input weld, it is determined whether there is a three-sided truncation. Specifically, the step of detecting whether there is a three-sided planar structure overlapping the retained input weld in the target workpiece point cloud can be implemented in the following way:

[0123] For each retained input weld, calculate the angle between the weld direction of the input weld and the intersection line of each three-plane structure detected in the point cloud of the target workpiece, and filter out the three-plane structures with an angle smaller than the first preset angle.

[0124] Calculate the angle between the plane normal of the input weld and the plane normal of each selected three-plane structure, and select the three-plane structure with an angle smaller than the second preset angle;

[0125] Calculate the projection distance from the common endpoint of the intersection line in each of the selected three-plane structures to the input weld, and select the three-plane structures whose projection distance is less than the preset distance;

[0126] Determine the vertical plane in the selected three-plane structure, project the endpoints of the input weld onto the vertical plane, and obtain the projection vector;

[0127] The two endpoints of the input weld are determined by the projection vectors of their respective endpoints. If they are not on the same side of the vertical plane, the input weld is determined to overlap with the three-plane structure.

[0128] Please refer to the following: Figure 13 The information used includes all detected three-plane structures and straight welds in the input weld (including the weld number, endpoint coordinates, plane normal, and weld reference direction of all detected straight welds).

[0129] The loop variable i = 0 represents the i-th straight weld, and obtains the intersection direction and plane normal of the straight weld. Iterate through all welds to filter matching three-plane structures, calculate the angle between the weld direction and the intersection line of the three-plane structure, and filter out three-plane structures with an angle less than a threshold.

[0130] Calculate the angle between the weld plane normal and the plane normal of the three-plane structure, and filter out the three-plane structures whose angle with the weld plane normal is less than a threshold.

[0131] Calculate the projected distance from the common endpoint of the three intersecting lines of the three-plane structure to the straight weld seam, and filter out the three-plane structures whose projected distance is less than the threshold.

[0132] Compare the planar normals corresponding to the straight weld with the planar normals of the three-plane structure, and select the vertical plane that is approximately perpendicular to the straight weld. Project the endpoints of the straight weld onto this vertical plane to obtain projection points. Connect the projection points and the weld endpoints to obtain the projection vector. Select the three-plane structures whose projection vector length is greater than a threshold.

[0133] Next, calculate the angle between the projection vector and the plane normal vector. If the projection vectors of the two weld endpoints are in the same direction, then the two endpoints of the weld are on the same side of the three-plane structure, meaning the straight weld is not truncated by the three-plane structure. Otherwise, the two endpoints of the weld are on opposite sides of the three-plane structure, and the weld is truncated by the three-plane structure, which can be understood as the straight weld overlapping with the three-plane structure. In this case, the straight weld needs to be filtered out.

[0134] For the input welds that have been filtered as described above, repeat the weld filtering and weld length filtering operations to retain the unique input weld result.

[0135] Specifically, in the repeated weld filtering process, the relative positional relationship between the retained input welds is detected, and filtering is performed based on the relative positional relationship and the relative positional relationship between the reference weld and its repeated straight welds. This step can be implemented in the following way:

[0136] Determine whether there is collinearity among the reference welds of the repeating straight welds of the weld to be inspected. If there is collinearity, classify the collinear reference welds into the same reference weld group, and generate the relative positional relationship between the reference welds and their repeating straight welds based on the relative positional relationship between the reference welds and their repeating straight welds.

[0137] Obtain the relative positional relationship between the retained input welds, classify the collinear input welds into the same input weld group, and obtain the relative positional relationship between the input weld groups;

[0138] If the input weld group and the reference weld group have the same array, match the input weld group and the reference weld group, calculate the length difference of the corresponding weld in the matched input weld group and the reference weld group, retain the input weld with the smallest difference, and filter out the other input welds.

[0139] Please refer to the following: Figure 14 When performing duplicate weld filtering, the input information includes multiple detected input welds (straight welds) and information on all reference welds (reference lines) that are duplicated with the currently detected straight weld (including reference weld length and reference line relative position relationship group).

[0140] First, determine if there are collinear reference lines in the relative position relationship group. If collinear reference lines exist, classify them into the same reference weld group (reference line group), and generate corresponding relative position relationship groups between reference line groups and between collinear reference lines based on the relative position relationship group. Otherwise, each input reference line corresponds to a reference line group, and the relative position relationship group of the reference line group is consistent with the relative position relationship group of the reference lines.

[0141] The input straight weld seams to be filtered are processed, and the relative positional relationships between the input straight lines are calculated to obtain groups of relative positional relationships of the input straight lines. Collinear input straight lines are grouped into the same input weld seam group (input straight line group). From each input straight line group, one input straight line is randomly selected, and the relative positional relationships between the input straight line groups are calculated.

[0142] The system checks if the number of input line groups and reference line groups are the same. If they are not the same, the number of non-collinear input lines is not equal to the number of non-collinear lines in the reference group, making it impossible to use the relative positions of the reference lines for filtering duplicate line welds, thus ending the duplicate line filtering process. If they are the same, the system uses the relative position relationship between the reference line group and the input line group to match the input line groups one by one.

[0143] Then, iterate through all reference line groups, determining if any reference lines are collinear. If collinear reference lines exist, they need to be matched with the input lines. Specifically, iterate through all input lines in the input line group, retrieve the collinear input line relative position relationship group corresponding to the input line group from the relative position relationship group, and filter out all candidate line groups that have the same relative position relationship as the collinear reference line relative position relationship group. If the number of candidate line groups is greater than 1, calculate the length difference of the corresponding weld seam between the candidate line group and the reference line group, select the candidate line group with the smallest difference as the output result of the reference line group, output all input lines corresponding to the reference lines, and filter out all remaining input lines that do not meet the conditions; otherwise, the collinear reference line group has no matching input lines, and the repeated line filtering process ends.

[0144] In a reference line group where no collinear reference lines exist, there is only one reference line. Iterate through all input lines in the corresponding input line group, selecting the weld line with the smallest length difference from the reference line as the output of that reference line group. Output all input lines corresponding to the reference lines, filtering out all remaining input lines that do not meet the conditions.

[0145] After filtering out duplicate straight welds using the above method, if the number of retained input welds is not unique, the weld length of the retained input welds can be calculated, and the difference between the length of the retained input welds and the length of the reference weld can be calculated. The input weld with the smallest difference is retained, and this input weld is the weld corresponding to the weld to be detected in the point cloud of the target workpiece.

[0146] Furthermore, when both the weld to be inspected and the input weld are of the full-circle weld type, filtering the input weld based on the reference weld information of the weld to be inspected and the weld type of the input weld can be achieved in the following way:

[0147] Based on the reference weld information of the weld to be inspected, obtain the radius and center of the reference weld; obtain the surface structure corresponding to the endpoint of the input weld; traverse all the biplane cylindrical structures in the obtained surface structure, and retain the biplane cylindrical structure whose radius is closest to that of the reference weld; detect the center, radius and axis of the retained biplane cylindrical structure, and obtain the output point of the retained input weld based on the center, radius and axis.

[0148] Please refer to the following: Figure 15 In this embodiment, the weld filtering process for the full-circle weld type includes input information such as the surface structure corresponding to the weld start point and the surface structure corresponding to the weld end point.

[0149] The radius and center of the reference circular weld in the reference weld are identified. From the filtered surface structures, the two-dimensional cylindrical structure that is closest to the reference center and radius is selected. The center, radius, and axial direction of the selected two-dimensional cylindrical structure are detected, and then the output point of the input circular weld is calculated.

[0150] The output point refers to the coordinate point on the whole circle. Generally, the whole circle can be divided into 4 or 5 points. The specific number of points can be set according to the requirements.

[0151] Furthermore, when both the weld to be inspected and the input weld are of the notched circular weld type, filtering the input weld based on the reference weld information of the weld to be inspected and the weld type of the input weld can be achieved in the following way:

[0152] Obtain the starting point surface structure and ending point surface structure corresponding to the starting point and ending point of each input weld;

[0153] Combine all starting surface structures and ending surface structures, and calculate the difference between the cylindrical radius of the starting surface structure and the cylindrical radius of the ending surface structure in each combination, filtering out combinations whose difference is greater than the preset difference;

[0154] For each retained combination, calculate the distance between the center of the cylindrical surface of the starting surface structure and the center of the cylindrical surface of the ending surface structure in each combination, and filter out combinations whose distance is greater than the preset distance;

[0155] Based on the surface structure types of the starting and ending surface structures in each retained combination, the combinations are filtered.

[0156] Select the face structure with the most cylindrical points from the filtered combinations, and use the information of this face structure as the retained input weld circle information;

[0157] The surface structure information of the reference weld in the reference weld information of the weld to be inspected is obtained, and the output point and notch point of the retained input weld are obtained based on the surface structure information.

[0158] Please refer to the following: Figure 16 Identify the intersection points of all input surface structures (i.e., the surface structures retained after filtering) that correspond to the endpoints of the weld (i.e., the input weld). Arrange and combine all input start-point and end-point surface structures to form a combination of a start-point surface structure and an end-point surface structure.

[0159] Then, radius matching filtering is used to traverse all combinations of starting surface structures and ending surface structures, calculate the difference between the cylindrical radius of the starting surface structure and the cylindrical radius of the ending surface structure. If the difference is less than the threshold, the corresponding starting and ending combination is retained; otherwise, the corresponding combination is filtered out.

[0160] Then, center-matching filtering is used to traverse all combinations of starting surface structures and ending surface structures, calculate the distance between the center of the cylindrical surface of the starting surface structure and the center of the cylindrical surface of the ending surface structure. If the distance is less than the threshold, the corresponding starting and ending combination is retained; otherwise, the corresponding combination is filtered out.

[0161] Determine the surface structure type of the starting and ending surface structures. If both the starting and ending surface structures are three-sided cylindrical structures, select the combination with the smallest central angle from the current starting and ending combination; otherwise, select the combination with the largest central angle.

[0162] From the selected combination of starting and ending surface structures, the surface structure with the most cylindrical points is selected as the representative surface structure of the notch circle. The corresponding center, radius, and axial direction are obtained from this surface structure and used as the retained input notch circle weld information.

[0163] The intersection point of the corresponding weld endpoints in the starting and ending surface structure combination is used as the input notch point of the notch circle weld. The output point of the notch circle is calculated using the center, radius, and axis.

[0164] The weld location method based on surface structure matching provided in this embodiment utilizes the overall framework of surface structure matching based on reference surface structures, and proposes an effective weld identification system. It makes full use of prior information such as surface structure lists and weld databases, and achieves efficient weld location through steps such as planar cylindrical surface detection, surface structure detection, surface structure filtering, and weld filtering.

[0165] By calculating the relative positional relationship between repeating straight welds, the system can distinguish between them. This eliminates the need for manual identification of weld numbers or approximate locations, improving system usability and expanding its application scenarios.

[0166] By utilizing the cylindrical center radius of the circular arc weld and the direction, length, and relative position of the straight weld, weld seams are filtered out, further filtering out surface structures that may interfere with identification, thereby improving the ease of use and accuracy of the system.

[0167] This approach reduces reliance on manual intervention. The system fully utilizes information from the reference surface structure list for surface structure filtering, improving its ability to filter similar surface structures. Simultaneously, a weld filtering step further filters out erroneous welds caused by repetitive surface structures, enabling the system to identify such structures. These two points lower the system's requirements for image capture pose, reduce reliance on manual adjustments, and ensure stable operation and weld detection even without manual specification of weld numbers.

[0168] Furthermore, the system exhibits high accuracy in weld seam positioning. By utilizing the relative positions of the center point and straight weld seams, it distinguishes and locates repetitive weld seams, enhancing not only recognition capabilities but also providing the system with a degree of error correction. As the amount of input data increases, the system can provide more accurate detection results and attempt to correct erroneous results, significantly improving weld seam positioning accuracy.

[0169] In addition, the system has high detection efficiency. It uses surface structure filtering and weld seam filtering to distinguish between repeated surface structures and repeated straight weld seams, enabling the system to detect multiple weld seams in the same point cloud, which significantly improves the utilization efficiency of point cloud data and the overall detection efficiency of the system.

[0170] In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0171] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Furthermore, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0173] It should be noted that if the functionality is implemented as a software module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0174] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0175] The above are merely embodiments of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A weld positioning method based on surface structure matching, characterized in that, The method includes: Acquire the point cloud of the target workpiece, and acquire the reference surface structure information and reference weld information of the weld to be detected in the point cloud of the target workpiece; Based on the reference welds in the aforementioned reference weld information, determine the repeating straight welds and their corresponding relative positional relationships; The target workpiece point cloud is subjected to surface structure detection, and the detected surface structure is filtered based on the reference surface structure in the reference surface structure information to obtain the filtered surface structure. Each input weld obtained by traversing the filtered surface structure combination is determined, the weld type of the input weld is determined, and the input weld is filtered based on the reference weld information of the weld to be detected and the weld type of the input weld. Specifically, when filtering for repeating straight welds in the input weld seam, the filtering is based on the relative positional relationship of the repeating straight welds.

2. The weld positioning method based on surface structure matching according to claim 1, characterized in that, The step of determining the repeating straight weld and its corresponding relative positional relationship based on the reference weld in the reference weld information includes: For the current reference straight weld in the reference weld information, find the reference straight weld in the reference weld information that has the same and parallel surface structure information as the endpoint of the current reference straight weld, and use it as the repeating straight weld. Calculate the relative positional relationships of repeating straight welds.

3. The weld positioning method based on surface structure matching according to claim 2, characterized in that, The step of calculating the relative positional relationship of repeated straight welds includes: For each of the repeated straight welds, the camera coordinate TCP is connected to the endpoint of the repeated straight weld, and a reference surface is constructed based on the connection and the direction vector of the repeated straight weld. Obtain the positional relationship between other repeating straight welds and the reference surface, and obtain the relative positional relationship of the repeating straight welds based on the positional relationship of the other repeating straight welds.

4. The weld positioning method based on surface structure matching according to claim 1, characterized in that, The detected surface structure types are three-plane structure, two-plane structure, three-cylinder structure, or two-cylinder structure; The step of filtering the detected surface structure based on the reference surface structure in the reference surface structure information includes: For each detected surface structure, a reference surface structure belonging to the same surface structure type as the detected surface structure is determined based on the surface structure type of each reference surface structure in the reference surface structure information; The reference surface structures belonging to the same surface structure type are matched with the surface structure, and surface structure filtering is performed according to the filtering conditions corresponding to the surface structure type.

5. The weld positioning method based on surface structure matching according to claim 1, characterized in that, The weld type of the weld to be inspected and the input weld is a straight weld type; The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes: For each of the input welds, obtain the surface structure corresponding to the two endpoints of the input weld, perform the operation of projecting the endpoints of the input welds onto the surface structure, and perform input weld filtering and input weld endpoint merging based on the projection result; The direction of the weld intersection line is calculated using the retained weld start and weld end points, and the angle between the weld intersection line direction and the weld direction of the reference weld of the weld to be inspected is calculated. Input welds corresponding to weld start and weld end points with an angle greater than a preset angle are filtered out. The system detects whether there is a three-plane structure in the point cloud of the target workpiece that overlaps with the retained input weld. If so, the overlapping input weld is filtered out. If there are repeated straight welds in the reference welds corresponding to the weld to be detected, the relative positional relationship between the retained input welds is detected, and filtering is performed based on the relative positional relationship and the relative positional relationship between the reference weld and its repeated straight welds. If the number of retained input welds is greater than 1, calculate the weld length of each input weld, obtain the difference between the weld length and the length of the reference weld of the weld to be inspected, and retain the input weld with the smallest difference.

6. The weld positioning method based on surface structure matching according to claim 5, characterized in that, The steps of performing the operation of projecting the endpoints of the input weld onto the surface structure, and performing input weld filtering and endpoint merging based on the projection results, include: The endpoint of the intersection line of the surface structure corresponding to the end point of the input weld is projected onto the extension line of the intersection line in the surface structure corresponding to the start point of the weld to obtain the projection distance. Input welds corresponding to the start point and end point of the weld with projection distance greater than the preset distance are filtered out. The retained weld start and end points are clustered, and weld start and end points belonging to the same cluster are merged to obtain the merged input weld.

7. The weld positioning method based on surface structure matching according to claim 5, characterized in that, The step of detecting whether there is a three-plane structure in the point cloud of the target workpiece that overlaps with the retained input weld includes: For each retained input weld, calculate the angle between the weld direction of the input weld and the intersection line of each three-plane structure detected in the point cloud of the target workpiece, and filter out the three-plane structures with an angle smaller than the first preset angle. Calculate the angle between the plane normal of the input weld and the plane normal of each selected three-plane structure, and select the three-plane structure with an angle smaller than the second preset angle; Calculate the projection distance from the common endpoint of the intersection line in each of the selected three-plane structures to the input weld, and select the three-plane structures whose projection distance is less than the preset distance; Determine the vertical plane in the selected three-plane structure, and project the endpoint of the input weld onto the vertical plane to obtain the projection vector; Based on the projection vectors of the two endpoints of the input weld, it is determined whether the two endpoints are on the same side of the vertical plane. If they are not on the same side of the vertical plane, it is determined that the input weld overlaps with the three-plane structure.

8. The weld positioning method based on surface structure matching according to claim 5, characterized in that, The step of filtering based on the relative positional relationship between the input welds retained by the detection, and the relative positional relationship between the reference weld and its repeating straight weld, includes: Determine whether there is collinearity among the reference welds of the repeating straight welds of the weld to be inspected. If there is collinearity, classify the collinear reference welds into the same reference weld group, and generate the relative positional relationship between the reference welds and their repeating straight welds based on the relative positional relationship between the reference welds and their repeating straight welds. Obtain the relative positional relationship between the retained input welds, classify the collinear input welds into the same input weld group, and obtain the relative positional relationship between the input weld groups; If the input weld group and the reference weld group have the same array, match the input weld group and the reference weld group. Calculate the length difference between the corresponding weld in the matched input weld group and the reference weld group, retain the input weld with the smallest difference, and filter out the other input welds.

9. The weld positioning method based on surface structure matching according to claim 1, characterized in that, The weld type of the weld to be inspected and the input weld is a full-circle weld type; The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes: The radius and center of the reference weld are obtained based on the reference weld information of the weld to be inspected; Obtain the surface structure corresponding to the endpoints of the input weld; Traverse all the biplane cylindrical structures in the obtained surface structure and retain the biplane cylindrical structure whose radius is closest to the reference weld. The center, radius, and axis of the retained two-sided cylindrical structure are detected, and the output point of the retained input weld is obtained based on the center, radius, and axis.

10. The weld positioning method based on surface structure matching according to claim 1, characterized in that, The weld type of the weld to be inspected and the input weld is a notched circular weld type; The step of filtering the input weld based on the reference weld information of the weld to be detected and the weld type of the input weld includes: Obtain the starting point surface structure and ending point surface structure corresponding to the starting point and ending point of each input weld; Combine all starting surface structures and ending surface structures, and calculate the difference between the cylindrical radius of the starting surface structure and the cylindrical radius of the ending surface structure in each combination, filtering out combinations whose difference is greater than the preset difference; For each retained combination, calculate the distance between the center of the cylindrical surface of the starting surface structure and the center of the cylindrical surface of the ending surface structure in each combination, and filter out combinations whose distance is greater than the preset distance; Based on the surface structure types of the starting and ending surface structures in each retained combination, the combinations are filtered. Select the face structure with the most cylindrical points from the filtered combinations, and use the information of this face structure as the retained input weld circle information; The surface structure information of the reference weld in the reference weld information of the weld to be inspected is obtained, and the output point and notch point of the retained input weld are obtained based on the surface structure information.

Images