Improved three-sided structural weld seam finding method, device and electronic equipment

By collecting and filtering point clouds of target objects, detecting and preserving the three-sided structure of concave structures, the positioning difficulties of existing weld seam detection methods when the workpiece rotates and its size changes are solved, achieving high-accuracy and adaptable weld seam detection.

CN118734495BActive Publication Date: 2024-11-15FAIR INNOVATION (SUZHOU) ROBOTIC SYSTEM CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411230820.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-11-15
Estimated Expiration
2044-09-04

AI Technical Summary

Technical Problem

Existing weld location methods struggle to accurately pinpoint weld locations when workpieces rotate, translate, or undergo dimensional changes, and are prone to identifying incorrect weld endpoints when point cloud quality is poor.

Method used

By collecting point clouds of the target object, detecting three-sided structures composed of multiple planes, filtering out three-sided structures with intersecting lines, cutting off parts that are not in the same quadrant, retaining the three-sided structures with concave structures, and matching them with pre-stored weld information to find weld information.

Benefits of technology

It improves the accuracy and adaptability of weld inspection, enabling accurate positioning of welds when workpiece size and orientation change, and reducing the impact of point cloud registration errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118734495B_ABST
    Figure CN118734495B_ABST
Patent Text Reader

Abstract

The present application provides an improved three-sided structure weld search method, device and electronic device, which collects and obtains a point cloud of a target object, and obtains multiple planes based on point cloud detection. The multiple planes constitute multiple three-sided structures, and each three-sided structure includes three planes. Screen out three-sided structures with intersection lines between any two planes. Determine the quadrant in which the shooting point is located in space, and cut off the parts of the screened three-sided structures that are located in quadrants other than this quadrant. Detect the three-sided structures that are concave structures among the retained three-sided structures, and for the three-sided structures that meet the preset requirements, match the intersection lines in the three-sided structures with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information. This solution improves the accuracy of the detection results by cutting off the interference of parts in other quadrants and detecting the three-sided structures that meet the concave structure, and matches with the pre-stored weld information to determine the positioning weld, which is more adaptable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of automatic welding, and in particular to an improved three-sided structure weld seam finding method, device and electronic equipment. Background Art

[0002] At present, in the field of welding technology, advanced technologies such as robotic arms and three-dimensional point clouds can be used to achieve automated, accurate and efficient weld extraction and tracking, thereby solving the problems and shortcomings of traditional welding methods.

[0003] In the existing weld search methods, the search is generally based on the direction vector and distance of the plane. However, in the existing search methods, if the workpiece to be welded rotates, translates, etc., it is difficult to locate the exact weld position, and the adaptability is weak. When the weld search based on plane distance changes size, the wrong position will often be found. In addition, in the existing methods, only the intersection of the two surfaces is used to determine the endpoints of the weld. When the point cloud quality of the two surfaces is poor, there may be internal missing surfaces, holes, reflective interference surfaces, etc., resulting in inaccurate endpoints. Summary of the invention

[0004] The objects of the present invention include, for example, providing an improved three-sided structure weld finding method, device and electronic equipment, which can improve the accuracy of weld detection and have stronger adaptability.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides an improved three-sided structural weld search method, the method comprising:

[0007] Acquire a point cloud of the target object, and acquire a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes;

[0008] Screen out three-sided structures with intersection lines between any two planes, and obtain the intersection lines between any two planes in the screened three-sided structures;

[0009] Determine the quadrant in which the shooting point is located in space, and cut off the parts of the screened three-sided structure located in quadrants other than the quadrant, so as to retain the three-sided structure located in the quadrant;

[0010] Detecting the three-surface structures that are concave structures among the retained three-surface structures, and detecting whether the intersection lines of the three-surface structures that are concave structures meet preset requirements;

[0011] For a three-sided structure that meets preset requirements, the intersection lines in the three-sided structure are matched with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information.

[0012] In an optional embodiment, the step of screening out three-surface structures having an intersection line between any two planes includes:

[0013] For any two planes in the three-plane structure, construct an intersection line equation according to the plane equations of the two planes;

[0014] Projecting each point on the plane onto the intersection line equation to obtain a projection point, and obtaining a projection distance corresponding to each point;

[0015] Target points on each of the planes whose projection distance is less than a set distance are determined. If the number of the target points is greater than or equal to a preset number, an intersection line exists between any two of the planes.

[0016] In an optional embodiment, the step of cutting off the parts of the screened three-sided structure located in quadrants other than the quadrant includes:

[0017] For each plane in the screened three-plane structure, detecting a first relative position of each point on the plane relative to the other two planes;

[0018] Determining a second relative position of the shooting point relative to the other two planes;

[0019] Delete the points where the first relative position is inconsistent with the second relative position.

[0020] In an optional implementation, the step of determining the second relative position of the shooting point relative to the other two planes includes:

[0021] Calculate the centroid point based on the points on each of the planes;

[0022] Calculating the direction vector between the shooting point and each of the centroid points;

[0023] The dot product between the direction vector and the plane is calculated, and a second relative position of the shooting point with respect to the other two planes is obtained according to the dot product.

[0024] In an optional implementation manner, the step of detecting the three-sided structures that are concave structures among the retained three-sided structures includes:

[0025] For any two planes in the retained three-plane structure, obtain the centroid points of each of the planes, and obtain the direction vectors from each of the centroid points to the shooting point;

[0026] Constructing an auxiliary plane according to the intersection line between the two planes, and determining the intersection lines between the auxiliary plane and each of the planes;

[0027] Determine the auxiliary points on each of the intersection lines, and calculate the direction vectors between the auxiliary points;

[0028] Whether the two planes are concave structures is detected according to the normal vectors of the two planes and the direction vector between the auxiliary points.

[0029] In an optional embodiment, the step of detecting whether the two planes are concave structures according to the normal vectors of the two planes and the direction vector between the auxiliary points comprises:

[0030] Determine a starting auxiliary point and an ending auxiliary point of a direction vector between the two auxiliary points;

[0031] Obtaining a first angle between a normal vector of a plane to which the starting auxiliary point belongs and the direction vector, and a second angle between a normal vector of a plane to which the terminating auxiliary point belongs and the direction vector;

[0032] If the first angle is an obtuse angle and the second angle is an acute angle, it is determined that the two planes are concave structures.

[0033] In an optional implementation manner, the step of detecting whether the intersection lines of the three-surface structure belonging to the concave structure meet preset requirements includes:

[0034] For a three-sided structure belonging to a concave structure, calculating the angle between every two intersection lines of the three-sided structure;

[0035] It is detected whether the angle between each two intersection lines is greater than or equal to a preset angle. If it is greater than or equal to the preset angle, it is determined that the preset requirement is met.

[0036] In an optional embodiment, the step of matching the intersection line in the three-surface structure with the pre-stored weld information of the target object to find the intersection line corresponding to the weld information includes:

[0037] Obtaining a weld direction vector from the pre-stored weld information of the target object, and obtaining an intersection direction vector of each intersection line in the three-surface structure;

[0038] Calculating the angle between each of the intersection direction vectors and the weld direction vector;

[0039] If there is at least one of the angles or 180 degrees minus the angle that is smaller than the preset angle, the intersection line corresponding to the minimum equivalent angle is obtained as the intersection line corresponding to the weld information, wherein the equivalent angle is the angle or 180 minus the angle.

[0040] In a second aspect, the present invention provides an improved three-sided structure weld finding device, the device comprising:

[0041] An acquisition module, used for acquiring a point cloud of a target object, and acquiring a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes;

[0042] A screening module, used for screening out three-sided structures with intersection lines between any two planes, and obtaining the intersection lines between any two planes in the screened three-sided structures;

[0043] A cutting module, used for determining the quadrant where the shooting point is located in space, and cutting off the parts of the screened three-surface structure located in quadrants other than the quadrant, so as to retain the three-surface structure located in the quadrant;

[0044] A detection module is used to detect the three-surface structures that are concave structures among the three-surface structures that are retained, and to detect whether the intersection lines of the three-surface structures that are concave structures meet preset requirements;

[0045] The search module is used to match the intersection lines in the three-sided structure that meets the preset requirements with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information.

[0046] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the steps of the method described in any one of the aforementioned implementation modes are implemented.

[0047] The beneficial effects of the embodiments of the present invention include, for example:

[0048] The present application provides an improved three-sided structure weld search method, device and electronic device, which collect and obtain a point cloud of a target object, and obtain multiple planes based on point cloud detection. The multiple planes constitute multiple three-sided structures, and each three-sided structure includes three planes. The three-sided structures in which there are intersection lines between any two planes are screened out, and the intersection lines between any two planes in the screened three-sided structures are obtained. The quadrant in which the shooting point is located in space is determined, and the parts of the screened three-sided structure located in quadrants other than the quadrant are cut off to retain the three-sided structure located in the quadrant. The three-sided structures that are concave structures in the retained three-sided structures are detected, and it is detected whether the intersection lines of the three-sided structures that are concave structures meet the preset requirements. For the three-sided structures that meet the preset requirements, the intersection lines in the three-sided structures are matched with the weld information of the pre-stored target object to find the intersection lines corresponding to the weld information.

[0049] This solution improves the accuracy of the detection result by removing interference from parts in other quadrants and detecting the three-sided structure that conforms to the concave structure, and matches it with the pre-stored weld information to determine the positioning weld, which is more adaptable. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A flowchart of an improved three-sided structure weld search method provided in an embodiment of the present application;

[0052] Figure 2 for Figure 1 A flowchart of the sub-steps included in S12;

[0053] Figure 3 for Figure 1 A flowchart of the sub-steps included in S13;

[0054] Figure 4 A schematic diagram of the three-side structure before the excision process provided in the embodiment of the present application;

[0055] Figure 5 A schematic diagram of the three-side structure after the resection process provided in the embodiment of the present application;

[0056] Figure 6 for Figure 3 A flowchart of the sub-steps included in S132;

[0057] Figure 7 for Figure 1 A flowchart of the sub-steps included in S14;

[0058] Figure 8 It is a schematic diagram of two planes and an auxiliary plane in an embodiment of the present application;

[0059] Fig. 9 for Figure 7 A flowchart of the sub-steps included in S144;

[0060] Fig.10 Schematic diagram of two planes of the concave structure in the embodiment of the present application;

[0061] Fig.11 Schematic diagram of two planes of the convex structure in the embodiment of the present application;

[0062] Fig.12for Figure 1 Another flow chart of the sub-steps included in S14;

[0063] Fig.13 Schematic diagram of two three-sided structures in the embodiment of the present application;

[0064] Fig.14 for Figure 1 A flowchart of the sub-steps included in S15;

[0065] Fig.15 Schematic diagram of two repeated three-sided structures in the embodiment of the present application;

[0066] Fig.16 A functional module block diagram of an improved three-sided structure weld finding device provided in an embodiment of the present application;

[0067] Fig.17 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0069] It should be mentioned before discussing the exemplary embodiments in more detail that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but can also have additional steps not included in the accompanying drawings. The process can correspond to methods, functions, procedures, subroutines, subprograms, etc. In addition, the embodiments in the present invention and the features in the embodiments can be combined with each other without conflict.

[0070] The term “including” and its variations used in the present invention are open inclusions, that is, “including but not limited to.” The term “based on” means “based at least in part on.” The term “one embodiment” means “at least one embodiment.”

[0071] It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish the corresponding contents, and are not used to limit the order or interdependence.

[0072] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0073] Figure 1 The flowchart of the improved three-sided structure weld seam search method provided by the embodiment of the present invention can realize the search of the weld seam of the three-sided structure in the workpiece. The method can be executed by an improved three-sided structure weld seam search device, which can be implemented in the form of hardware and / or software, and the improved three-sided structure weld seam search device can be configured in electronic devices such as computers, servers, and tablet computers.

[0074] Specifically, see Figure 1 The improved three-sided structure weld seam search method provided in this embodiment includes the following steps:

[0075] S11, acquiring a point cloud of a target object, and acquiring a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes.

[0076] S12, screening out three-surface structures having intersection lines between any two planes, and obtaining the intersection lines between any two planes in the screened out three-surface structures.

[0077] S13, determining the quadrant where the shooting point is located in the space, and removing the parts of the screened three-surface structure located in quadrants other than the quadrant, so as to retain the three-surface structure located in the quadrant.

[0078] S14, detecting the three-surface structures that are concave structures among the retained three-surface structures, and detecting whether the intersection lines of the three-surface structures that are concave structures meet preset requirements.

[0079] S15, for the three-sided structure that meets the preset requirements, matching the intersection lines in the three-sided structure with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information.

[0080] In this embodiment, the target object may be a workpiece, and an image of the target object may be obtained by photographing with a camera installed on the robotic arm, and a point cloud of the target object may be obtained based on the photographed image.

[0081] In this embodiment, the weld seam search is mainly performed on a three-sided structure in a workpiece. The three-sided structure is similar to a corner structure, and the weld seams to be searched are weld seams formed by the intersecting surfaces of the three-sided structure.

[0082] Based on the collected point cloud of the target object, plane detection is performed to obtain multiple planes. Plane detection can use, for example, a Hough plane detection algorithm, a RANSAC plane detection algorithm, etc. In this embodiment, a Hough plane detection algorithm can be used. The detected multiple planes can form multiple three-plane structures.

[0083] Among the multiple three-sided structures, some of them have intersection lines between any two planes, while some of them may have planes without real intersection lines. Therefore, firstly, the three-sided structures with intersection lines between any two planes are screened out. For the screened three-sided structures, the real intersection lines between any two planes can be obtained, which are the intersection line segments formed by the actual intersection between the two planes.

[0084] When using a camera to shoot a target object, it is inevitable to shoot unnecessary extended surfaces, which will affect the subsequent detection and judgment of the three-sided structure. Therefore, it is necessary to cut off the unnecessary extended surfaces and only retain the required three-sided structure.

[0085] When shooting, the part of the target object facing the camera is generally shot from the camera's shooting angle. If the three-dimensional space is divided into multiple quadrants, the shooting point is located in one of the quadrants, where the shooting point is the location of the camera. The three-sided structure of the required part is the part in the same quadrant as the shooting point. Therefore, the part of the three-sided structure located in the quadrant other than the quadrant where the shooting point is located can be cut off, so as to retain the three-sided structure located in the same quadrant as the shooting point.

[0086] In order to facilitate the welding of the weld by the welding tool, the required three-sided structure is a concave structure. The concave structure here is relative to the shooting point, that is, in the direction toward the shooting point, the weld is surrounded by the surrounding plane.

[0087] In addition, considering that the point cloud may be affected by the lighting when shooting, a reflective plane may be generated, and the reflective plane may also form a three-sided structure together with other planes, but the welds in such three-sided structures are not real welds or welds that do not meet the requirements. Therefore, in this embodiment, it is also possible to detect whether the intersection lines of the three-sided structure meet the preset requirements. In this way, the three-sided structure can be further screened.

[0088] In this embodiment, the weld information of the target object is pre-stored in the database, and the weld information includes the weld number, the starting coordinates of the weld, the end coordinates of the weld, etc., and the weld information is saved in the obj file format. The saved starting coordinates and end coordinates of the weld are the position information in the simulation space, which can be understood as the position information under the coordinate system constructed in the modeling software. The position information of the target object actually photographed is the position information under the robot base coordinate system. Therefore, after reading the starting coordinates and end coordinates of the weld in the database, the starting coordinates and end coordinates of the weld are converted into coordinates under the robot base coordinate system.

[0089] On the basis of the three-sided structure obtained by the above screening, the intersection line in the three-sided structure can be matched with the weld start point coordinates and weld end point coordinates in the pre-stored weld information, so as to find the intersection line in the three-sided structure that matches the weld start point coordinates and weld end point coordinates, and the intersection line is the weld actually required to be welded.

[0090] The improved three-sided structure weld search method provided in this embodiment improves the accuracy of the detection results by removing the interference of parts in other quadrants and detecting the three-sided structure that conforms to the concave structure, and matches it with the pre-stored weld information to determine the positioning weld, which is more adaptable.

[0091] See also Figure 2 In this embodiment, the above screening of the three-surface structure with an intersection line between any two planes can be achieved by the following method:

[0092] S121, for any two planes in the three-plane structure, construct an intersection line equation according to the plane equations of the two planes.

[0093] S122, projecting each point on the plane onto the intersection line equation to obtain a projection point, and obtaining a projection distance corresponding to each point.

[0094] S123, determining target points on each of the planes whose projection distance is less than a set distance, and if the number of the target points is greater than or equal to a preset number, there is an intersection line between any two planes.

[0095] In this embodiment, the main purpose here is to screen out three-sided structures where there is an intersection line between any two planes. When screening, it is determined whether there is an intersection line between every two planes. If there is an intersection line between any two planes, the corresponding three-sided structure can be retained, otherwise, the corresponding three-sided structure is deleted.

[0096] Specifically, for every two arbitrary planes, the plane equations of the two planes are combined, and a point on the intersection line is determined by assignment. The cross product of the normal vectors of the two planes is calculated, and the cross product is the direction vector of the intersection line. In this way, the intersection line equation can be constructed based on a point on the determined intersection line and the direction vector of the intersection line.

[0097] For any of the two planes, projecting the points on the plane onto the intersection equation is actually projecting onto the intersection line corresponding to the intersection equation, so that the projection point on the intersection line can be obtained. The distance between each point and its corresponding projection point is the projection distance corresponding to the point.

[0098] For each point on the plane, if its projection distance is less than the set distance, the point can be used as a target point, indicating that the point is near the intersection line. If the number of target points on the plane is greater than or equal to the preset number, it is determined that the plane has a real intersection line on the intersection line.

[0099] For the other plane of the two planes, the same judgment method as above is adopted. If both planes have a real intersection line on the intersection line, it is determined that there is an intersection line between the two planes.

[0100] Through the above method, three-surface structures with intersection lines between any two planes can be screened out.

[0101] For the selected three-sided structure, the intersection line between any two planes in the three-sided structure can be obtained, and the intersection line is the actual intersection line segment. In any two planes, the projection points obtained by projecting the points on each plane to the intersection line can respectively form two line segments, and the two line segments may have overlapping parts. The endpoints of each line segment are obtained respectively, so that four endpoints can be obtained. Compare the sizes of the four endpoints to determine the smallest endpoint and the largest endpoint. The line segment formed by the smallest endpoint and the largest endpoint is the intersection line segment formed by the actual intersection of the two planes.

[0102] After the three-sided structure is screened out based on the above mechanism and the intersection line segments between any two planes are determined, some extended surfaces may be photographed when photographing the target object. These extended surfaces will affect the subsequent detection and judgment of the three-sided structure. Therefore, please refer to Figure 3 In this embodiment, the three-side structure can be cut off in the following manner:

[0103] S131, for each plane in the screened three-plane structure, detecting a first relative position of each point on the plane relative to the other two planes.

[0104] S132: Determine a second relative position of the shooting point relative to the other two planes.

[0105] S133: Delete points where the first relative position is inconsistent with the second relative position.

[0106] Figure 4 The schematic diagram of the photographed three-sided structure is schematically shown, and the three-sided structure includes plane A, plane B and plane C, and it is assumed that the shooting point is point P in the figure. Assuming that the three-dimensional space is divided into 8 quadrants, wherein the shooting point is located in quadrant I, then what is actually needed is the part of the three-sided structure in quadrant I, and the parts located in other quadrants need to be cut off.

[0107] Specifically, for each plane in the three-plane structure, the relative position of the plane with the other two planes can be determined, and the relative position of the shooting point with the other two planes can be determined. The points on the plane whose relative positions are consistent with the relative positions of the shooting point are retained, and the inconsistent points are deleted, so as to achieve the resection processing of the plane. The other two planes are resected in the same way. In this way, the resection processing of the three planes can be completed to obtain the part located in the same quadrant as the shooting point. For example, the three-plane structure after the resection processing can be as follows Figure 5 as shown in .

[0108] See also Figure 6 , when determining the second relative position of the shooting point relative to the other two planes, it can be achieved in the following ways:

[0109] S1321, calculating the centroid point based on the points on each of the planes.

[0110] S1322: Calculate the direction vector between the shooting point and each of the centroid points.

[0111] S1323: Calculate the dot product between the direction vector and the plane, and obtain a second relative position of the shooting point with respect to the other two planes according to the dot product.

[0112] In this embodiment, it is assumed that the three-plane structure includes plane A, plane B and plane C. First, the centroid of each plane is calculated. The direction vector between the shooting point and each centroid is obtained, and the dot product between the direction vector and the normal vector of the plane is calculated.

[0113] For example, when resecting plane A, the direction vector between the shooting point and the centroid of plane B can be calculated, and the dot product between the direction vector and the normal vector of plane B can be calculated. If the dot product is greater than 0, it means that the shooting point is in the direction of the normal vector, otherwise, it means that the shooting point is in the opposite direction of the normal vector. The directional relationship between the shooting point and the normal vector is the second relative position between the shooting point and plane B. Similarly, by calculating the direction vector between the shooting point and the centroid of plane C, the second relative position between the shooting point and plane C can be determined.

[0114] In addition, for each point on plane C, the method for determining the first relative position between each point and plane B and plane C is the same as the above method, which will not be repeated here.

[0115] Traverse each point pa_i in plane A. If the first relative position between it and plane B and plane C is the same as the second relative position between the shooting point and plane B and plane C, for example, they are both in the negative half space of the two planes, then point pa_i can be retained and saved in the point cloud of the new plane A'. Otherwise, point pa_i is deleted.

[0116] Similarly, perform the resection process on plane B, traverse each point pb_i in plane B, and determine its first relative position relative to plane A and plane C, and add the points that meet the conditions to the new plane B' point cloud. Check each point pc_i in plane C, and determine its position relative to plane A and plane B, and add the points that meet the conditions to the new plane C' point cloud.

[0117] After obtaining the point clouds of the new planes A', B', and C' in the above manner, it is possible to detect whether the number of points in each new point cloud is greater than the set number. If it is greater than the set number, it is determined to be effective resection. In the case of effective resection, the new three-sided structure obtained is a three-sided structure composed of the point clouds of the new planes A', B', and C'. If it is determined to be invalid resection, it means that the obtained three-sided structure does not meet the requirements, and the corresponding three-sided structure can be directly deleted.

[0118] Since each plane in the three-sided structure is cut, the endpoints of the intersection line segments formed by the planes may change. Therefore, for the three-sided structure after the cut processing, the endpoints of the intersection line between any two planes can be recalculated. Finally, the three-sided structure after the cut processing and the adjustment of the intersection line endpoints is obtained.

[0119] When taking into account the target object, the camera is usually placed in the quadrant of the three-sided structure where the weld is located to ensure the integrity of the shooting and the high quality of the point cloud. Therefore, the three-sided structure actually required should be a concave structure. Based on this, please refer to Figure 7 In this embodiment, the three-sided structures that are concave structures among the three-sided structures that are retained can be detected by the following method:

[0120] S141, for any two planes in the retained three-plane structure, obtain the centroid points of each of the planes, and obtain the direction vector from each of the centroid points to the shooting point.

[0121] S142, constructing an auxiliary plane according to the intersection line between the two planes, and determining the intersection lines between the auxiliary plane and each of the planes.

[0122] S143, determining the auxiliary points on each of the intersection lines, and calculating the direction vectors between the auxiliary points.

[0123] S144: Detect whether the two planes are a concave structure according to the normal vectors of the two planes and the direction vector between the auxiliary points.

[0124] In this embodiment, it is assumed that two planes in the three-plane structure are as follows Figure 8 As shown in , it includes plane A and plane B, wherein the arrow on plane A indicates the normal vector of plane A, and the arrow on plane B indicates the normal vector of plane B. There is an intersection line between plane A and plane B. Point CC indicates the shooting point.

[0125] First, the direction vectors from the centroid of plane A and plane B to the shooting point can be calculated respectively. If the normal vector of the plane is inconsistent with the direction vector, the normal vector of the plane can be negated to ensure that the normal vector of each plane is adjusted to the direction of the quadrant where the shooting point is located.

[0126] Take the midpoint D on the intersection line, use the direction vector of the intersection line as the normal vector, and construct an auxiliary plane C through point D. The auxiliary plane C has intersection lines with both plane A and plane B.

[0127] Take an auxiliary point on each of the two intersecting lines, such as Figure 8 Points E and F in the middle, and the distances between the midpoint D and points E and F are equal. Calculate the direction vector from point E to point F. Combine the normal vectors of the two planes and the direction vector between the auxiliary points to determine whether the two planes are concave structures. For details, please refer to Fig. 9 , this step can be achieved by:

[0128] S1441, determining a starting auxiliary point and an ending auxiliary point of a direction vector between the two auxiliary points.

[0129] S1442, obtaining a first angle between the normal vector of the plane to which the starting auxiliary point belongs and the direction vector, and a second angle between the normal vector of the plane to which the terminating auxiliary point belongs and the direction vector.

[0130] S1443: If the first angle is an obtuse angle and the second angle is an acute angle, determine that the two planes are a concave structure.

[0131] In this embodiment, if auxiliary point E points to auxiliary point F to construct a direction vector, the starting auxiliary point is point E and the ending auxiliary point is point F (this example will be used for explanation later). If auxiliary point F points to auxiliary point E to construct a direction vector, the starting auxiliary point is point F and the ending auxiliary point is point E.

[0132] like Fig.10As shown in , if the two planes are concave structures, the angle α between the normal vector N_a of plane A and the direction vector concavity_dir is an acute angle, about 45°. The angle β between the normal vector N_b of plane B and the direction vector concavity_dir is an obtuse angle, about 135°.

[0133] like Fig.11 As shown in , if the two planes are convex structures, the angle α between the normal vector N_a of plane A and the direction vector concavity_dir is an obtuse angle of about 135°. The angle β between the normal vector N_b of plane B and the direction vector concavity_dir is an acute angle of about 45°.

[0134] Based on the three-sided structure of the concave structure detected by the above method, considering that in actual shooting, due to the shooting angle, lighting and the metal material of the workpiece, a plane with a certain degree of integrity may be generated due to reflection, and this plane may form a three-sided structure with the two surrounding planes. However, this three-sided structure does not meet the specifications, and the intersection lines between its planes often do not meet the relevant specifications, which makes it inconvenient for welding tools to penetrate into the weld position for welding.

[0135] Based on the above considerations, please refer to Fig.12 In this embodiment, it is also possible to detect whether the intersection lines of the three-surface structures belonging to the concave structure meet the preset requirements in the following manner:

[0136] S145, for the three-sided structure that is a concave structure, calculating the angle between every two intersection lines in the three-sided structure.

[0137] S146, detecting whether the angle between every two intersection lines is greater than or equal to a preset angle, and if so, determining that the preset requirement is met.

[0138] like Fig.13 As shown in , plane 1, plane 2 and plane 3 form a three-plane structure, and plane 2, plane 3 and plane 4 form a three-plane structure. Among them, plane 4 is a reflective surface formed by reflection.

[0139] For each three-sided structure, the angle between every two intersection lines in the three-sided structure is calculated. If the angle between every two intersection lines is greater than or equal to the preset angle, it can be determined that the three-sided structure meets the preset requirements. Otherwise, the three-sided structure does not meet the preset requirements.

[0140] For example, Fig.13 The three-sided structure composed of midplane 2, plane 3 and plane 4 has a smaller angle between intersection lines ③ and ④, which is likely to be smaller than the preset angle. Therefore, it can be determined that the three-sided structure does not meet the preset requirements and can be deleted.

[0141] After filtering out the three-sided structure as above, you can match the intersection lines in the three-sided structure with the weld information of the pre-stored target object in the following ways. Please refer to Fig.14 :

[0142] S151, obtaining a weld direction vector in the pre-stored weld information of the target object, and obtaining an intersection direction vector of each intersection line in the three-surface structure.

[0143] S152, calculating the angle between each of the intersection direction vectors and the weld direction vector.

[0144] S153, if there is at least one of the angles or 180 degrees minus the angle that is smaller than the preset angle, obtain the intersection line corresponding to the minimum equivalent angle as the intersection line corresponding to the weld information, wherein the equivalent angle is the angle or 180 minus the angle.

[0145] In this embodiment, the endpoint coordinates of the weld can be read from the database, and then the weld direction vector d of the weld can be obtained. In addition, the intersection direction vectors of each intersection line in the three-sided structure are obtained, and are assumed to be d1, d2, and d3. The angles between d and d1, d2, and d3 are calculated respectively, and are recorded as a1, a2, and a3.

[0146] First, it can be determined whether the angles a1, a2, and a3 are greater than 90 degrees. If they are greater than 90 degrees, the angle is subtracted from 180 degrees. Then it is determined whether each angle or 180 degrees minus the angle is less than the preset angle. If there is an angle or 180 degrees minus the angle is less than the preset angle, it is determined that the three-sided structure can be retained, otherwise, the three-sided structure can be deleted.

[0147] For the three-sided structure retained at the end, the angle between the direction vector of each intersection line and the direction vector of the weld has been calculated above. Considering the possible reverse problem, when the angle is greater than 90 degrees, the angle can be subtracted from 180 degrees. The angle that is less than 90 degrees or greater than 90 degrees is subtracted from 180 degrees as the equivalent angle. Find the minimum equivalent angle, and the intersection line corresponding to the minimum equivalent angle is the actual weld required for welding. The information of the intersection line, such as the two end points of the intersection line, can be output for subsequent welding.

[0148] In addition, in this embodiment, it is considered that some workpieces may have repeated structures, such as Fig.15As shown in the figure, the three-sided structure No. ① is exactly the same as the three-sided structure No. ②, and it is judged that the three-sided structure No. ① and the three-sided structure No. ② are repeated three-sided structures. At this time, the three-sided structure where the desired weld is located can be selected by the principle of proximity or distance. The distance dis from the shooting point to the intersection of the three-sided structure is used as the basis for judgment. If the target is to weld a weld in the three-sided structure No. ②, the three-sided structure with the largest dis is selected. If the target is to weld a weld in the three-sided structure No. ①, the three-sided structure with the smallest dis is selected.

[0149] The improved three-sided structure weld search method provided in this embodiment can improve the accuracy of weld position search. When there is any translation error in the point cloud registration, the improved algorithm provided in this embodiment can also find the correct location of the weld. In plane detection, Hough plane detection is used to detect all three-sided structures with real intersection lines based on the input point cloud itself, which is not affected by the registration. This step provides a preliminary, multi-candidate weld position set. Through multi-level filtering, screening and geometric feature matching, the impact of the low accuracy of point cloud registration is effectively reduced, and the accuracy and robustness of weld position search are improved. The comprehensive application of step-by-step optimization and multiple steps enables the algorithm to maintain a high weld positioning accuracy under large registration error conditions.

[0150] Regardless of how the size and posture of the workpiece change, the target weld is formed by the intersection of two planes in the three-sided structure, and the structure is fixed. Hough plane detection can accurately identify all three-sided structures with real intersection lines, ensuring that the basic data for weld search is accurate. According to the coordinates of the shooting point of the camera posture, each three-sided structure is divided and adjusted to adapt to changes in the size, rotation, translation, etc. of the workpiece. By judging the concavity and convexity of the three-sided structure to screen qualified weld structures, the adaptability of the algorithm is enhanced to ensure that the correct weld position can still be found under different size and position changes. By ensuring that the angle between the two intersection lines meets the threshold condition, the three-sided structure generated by the reflective plane is removed, which improves the robustness of the algorithm to reflective interference on different workpiece surfaces.

[0151] The method of determining the weld endpoint based on point cloud segmentation and three-surface intersection based on the posture of the shooting point effectively solves the shortcoming of only relying on the intersection of two surfaces to determine the weld endpoint, especially when the point cloud quality is poor, and improves the accuracy of weld endpoint determination.

[0152] Based on the same inventive concept, please refer to Fig.16, shows a schematic diagram of the functional modules of the improved three-sided structure weld search device provided by an embodiment of the present invention. This embodiment can divide the functional modules of the improved three-sided structure weld search device according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0153] For example, when each functional module is divided into corresponding functional modules, Fig.16 The improved three-sided structure weld seam finding device shown is only a schematic diagram of the device. The improved three-sided structure weld seam finding device may include a collection module, a screening module, a cutting module, a detection module and a finding module. The functions of each functional module of the improved three-sided structure weld seam finding device are described in detail below.

[0154] An acquisition module, used for acquiring a point cloud of a target object, and acquiring a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes;

[0155] A screening module, used for screening out three-sided structures with intersection lines between any two planes, and obtaining the intersection lines between any two planes in the screened three-sided structures;

[0156] A cutting module, used for determining the quadrant where the shooting point is located in space, and cutting off the parts of the screened three-surface structure located in quadrants other than the quadrant, so as to retain the three-surface structure located in the quadrant;

[0157] A detection module is used to detect the three-surface structures that are concave structures among the three-surface structures that are retained, and to detect whether the intersection lines of the three-surface structures that are concave structures meet preset requirements;

[0158] The search module is used to match the intersection lines in the three-sided structure that meets the preset requirements with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information.

[0159] It can be understood that the above-mentioned acquisition module, screening module, excision module, detection module and search module can be used to execute the above-mentioned S11 to S15. The detailed implementation method of the acquisition module, screening module, excision module, detection module and search module can refer to the relevant contents of the above-mentioned S11 to S15.

[0160] Optionally, the above screening module can be used to:

[0161] For any two planes in the three-plane structure, construct an intersection line equation according to the plane equations of the two planes;

[0162] Projecting each point on the plane onto the intersection line equation to obtain a projection point, and obtaining a projection distance corresponding to each point;

[0163] Target points on each of the planes whose projection distance is less than a set distance are determined. If the number of the target points is greater than or equal to a preset number, an intersection line exists between any two of the planes.

[0164] Optionally, the above-mentioned resection module can be used to:

[0165] For each plane in the screened three-plane structure, detecting a first relative position of each point on the plane relative to the other two planes;

[0166] Determining a second relative position of the shooting point relative to the other two planes;

[0167] Delete the points where the first relative position is inconsistent with the second relative position.

[0168] In a possible implementation, the above-mentioned resection module may be used to:

[0169] Calculate the centroid point based on the points on each of the planes;

[0170] Calculating the direction vector between the shooting point and each of the centroid points;

[0171] The dot product between the direction vector and the plane is calculated, and a second relative position of the shooting point with respect to the other two planes is obtained according to the dot product.

[0172] Optionally, the above detection module can be used for:

[0173] For any two planes in the retained three-plane structure, obtain the centroid points of each of the planes, and obtain the direction vectors from each of the centroid points to the shooting point;

[0174] Constructing an auxiliary plane according to the intersection line between the two planes, and determining the intersection lines between the auxiliary plane and each of the planes;

[0175] Determine the auxiliary points on each of the intersection lines, and calculate the direction vectors between the auxiliary points;

[0176] Whether the two planes are concave structures is detected according to the normal vectors of the two planes and the direction vector between the auxiliary points.

[0177] Optionally, the above detection module can be used for:

[0178] Determine a starting auxiliary point and an ending auxiliary point of a direction vector between the two auxiliary points;

[0179] Obtaining a first angle between a normal vector of a plane to which the starting auxiliary point belongs and the direction vector, and a second angle between a normal vector of a plane to which the terminating auxiliary point belongs and the direction vector;

[0180] If the first angle is an obtuse angle and the second angle is an acute angle, it is determined that the two planes are concave structures.

[0181] Optionally, the above detection module can also be used for:

[0182] For a three-sided structure belonging to a concave structure, calculating the angle between every two intersection lines of the three-sided structure;

[0183] It is detected whether the angle between each two intersection lines is greater than or equal to a preset angle. If it is greater than or equal to the preset angle, it is determined that the preset requirement is met.

[0184] Optionally, the above search module can be used to:

[0185] Obtaining a weld direction vector from the pre-stored weld information of the target object, and obtaining an intersection direction vector of each intersection line in the three-surface structure;

[0186] Calculating the angle between each of the intersection direction vectors and the weld direction vector;

[0187] If there is at least one of the angles or 180 degrees minus the angle that is smaller than the preset angle, the intersection line corresponding to the minimum equivalent angle is obtained as the intersection line corresponding to the weld information, wherein the equivalent angle is the angle or 180 minus the angle.

[0188] Fig.17 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may be a computer device, a server, a notebook computer, etc. Fig.17 As shown, the electronic device includes one or more processors and a memory; the processor in the electronic device may be one or more, Fig.17 A processor is taken as an example; the memory is used to store one or more programs; the one or more programs are executed by one or more processors, so that the one or more processors implement the improved three-sided structural weld search method as any one of the embodiments of the present invention.

[0189] The electronic device may further include: an input device and an output device.

[0190] The processor, memory, input device and output device in the electronic device can be connected through a bus or other means. Fig.17The example of connecting through bus is taken in the following.

[0191] The memory in the electronic device is a computer-readable storage medium and can be used to store one or more programs, which can be software programs, computer executable programs and modules, such as program instructions / modules corresponding to the improved three-sided structural weld search method provided in the embodiment of the present invention. The processor executes various functional applications and data processing of the terminal device by running the software programs, instructions and modules stored in the memory, that is, the improved three-sided structural weld search method in the above method embodiment is implemented.

[0192] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0193] In some examples, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0194] The input device can be used to receive input digital or character information and generate key signal input related to user settings and function control of the device. The output device can include display devices such as display screens.

[0195] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the program can also be used to execute the improved three-sided structure weld search method provided by any embodiment of the present invention.

[0196] The computer-readable storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (Random Access Memory, RAM), a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ReadOnly Memory, EPROM), a flash memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0197] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to: electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0198] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the above.

[0199] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0200] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An improved method for finding welds on three-sided structures. It is characterized in that The method comprises: Acquire a point cloud of the target object, and acquire a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes; Screen out three-sided structures with intersection lines between any two planes, and obtain the intersection lines between any two planes in the screened three-sided structures; Determine the quadrant in which the shooting point is located in space, and cut off the parts of the screened three-sided structure located in quadrants other than the quadrant, so as to retain the three-sided structure located in the quadrant; Detecting the three-surface structures that are concave structures among the retained three-surface structures, and detecting whether the intersection lines of the three-surface structures that are concave structures meet preset requirements; For a three-sided structure that meets preset requirements, the intersection lines in the three-sided structure are matched with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information; The step of detecting the three-sided structures that are concave structures among the retained three-sided structures comprises: For any two planes in the retained three-sided structure, the centroid points of each plane are obtained, and the direction vectors from each centroid point to the shooting point are obtained; an auxiliary plane is constructed according to the intersection line between the two planes, and the intersection lines of the auxiliary plane and each plane are determined; the auxiliary points on each intersection line are determined, and the direction vectors between the auxiliary points are calculated; according to the normal vectors of the two planes and the direction vectors between the auxiliary points, whether the two planes are concave structures is detected; The step of detecting whether the intersection lines of the three-surface structures belonging to the concave structure meet the preset requirements includes: For a three-sided structure that is a concave structure, the angle between every two intersection lines in the three-sided structure is calculated; whether the angle between every two intersection lines is greater than or equal to a preset angle is detected, and if it is greater than or equal to the preset angle, it is determined that the preset requirements are met.

2. According to the improved three-sided structural weld finding method of claim 1, It is characterized in that The step of screening out the three-surface structure having an intersection line between any two planes comprises: For any two planes in the three-plane structure, construct an intersection line equation according to the plane equations of the two planes; Projecting the points on each of the planes onto the intersection line equation to obtain projection points, and obtaining projection distances corresponding to each of the points, wherein each of the points is a point on each of the planes; Target points on each of the planes whose projection distance is less than a set distance are determined. If the number of the target points is greater than or equal to a preset number, an intersection line exists between any two of the planes.

3. The improved three-sided structural weld finding method according to claim 1, It is characterized in that The step of removing the parts of the screened three-sided structure located in other quadrants except the quadrant comprises: For each plane in the screened three-plane structure, detecting a first relative position of each point on the plane relative to the other two planes; Determining a second relative position of the shooting point relative to the other two planes; Delete the points where the first relative position is inconsistent with the second relative position.

4. The improved three-sided structural weld finding method according to claim 3, It is characterized in that The step of determining the second relative position of the shooting point relative to the other two planes comprises: Calculate the centroid point based on the points on each of the planes; Calculating the direction vector between the shooting point and each of the centroid points; The dot product between the direction vector and the plane is calculated, and a second relative position of the shooting point with respect to the other two planes is obtained according to the dot product.

5. The improved three-sided structural weld finding method according to claim 1, It is characterized in that The step of detecting whether the two planes are concave structures according to the normal vectors of the two planes and the direction vector between the auxiliary points comprises: Determine a starting auxiliary point and an ending auxiliary point of a direction vector between the two auxiliary points; Obtaining a first angle between a normal vector of a plane to which the starting auxiliary point belongs and the direction vector, and a second angle between a normal vector of a plane to which the terminating auxiliary point belongs and the direction vector; If the first angle is an obtuse angle and the second angle is an acute angle, it is determined that the two planes are concave structures.

6. The improved three-sided structural weld finding method according to claim 1, It is characterized in that The step of matching the intersection lines in the three-surface structure with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information includes: Obtaining a weld direction vector from the pre-stored weld information of the target object, and obtaining an intersection direction vector of each intersection line in the three-surface structure; Calculating the angle between each of the intersection direction vectors and the weld direction vector; If there is at least one of the angles or 180 degrees minus the angle that is smaller than the preset angle, the intersection line corresponding to the minimum equivalent angle is obtained as the intersection line corresponding to the weld information, wherein the equivalent angle is the angle or 180 degrees minus the angle.

7. An improved three-sided structural weld finding device, It is characterized in that The device is used to implement the improved three-sided structure weld finding method according to any one of claims 1 to 6, comprising: An acquisition module, used for acquiring a point cloud of a target object, and acquiring a plurality of planes based on the point cloud detection, wherein the plurality of planes constitute a plurality of three-plane structures, and each of the three-plane structures includes three planes; A screening module, used for screening out three-sided structures with intersection lines between any two planes, and obtaining the intersection lines between any two planes in the screened three-sided structures; A cutting module, used for determining the quadrant where the shooting point is located in space, and cutting off the parts of the screened three-surface structure located in quadrants other than the quadrant, so as to retain the three-surface structure located in the quadrant; A detection module is used to detect the three-surface structures that are concave structures among the three-surface structures that are retained, and to detect whether the intersection lines of the three-surface structures that are concave structures meet preset requirements; The search module is used to match the intersection lines in the three-sided structure that meets the preset requirements with the pre-stored weld information of the target object to find the intersection lines corresponding to the weld information.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.