Weld joint construction method and electronic equipment

Abstract

The invention provides a welding seam construction method and electronic equipment, a plurality of local images are obtained by controlling camera shooting equipment to shoot a target workpiece for multiple times under different poses, and local point clouds in all the local images are obtained. And for each local point cloud, performing plane detection and cylindrical surface detection on the local point cloud to judge whether a plane structure and a cylindrical surface structure exist in the local point cloud. And obtaining a plane structure under the condition that the plane is detected, storing the structure information of the plane structure, obtaining a cylindrical surface structure under the condition that the cylindrical surface is detected, and storing the structure information of the cylindrical surface structure. And combining all the stored structure information of the plane structure and the cylindrical surface structure, and constructing welding seam information of the target workpiece according to the combined result information. According to the scheme, a welding seam database which does not need to be marked is shot for multiple times, and the plane structure and the cylindrical surface structure which are obtained through multiple times of shooting are combined, so that all welding seams of the target workpiece are constructed, and the application range of the scheme is expanded.

Description

Technical Field

[0001] The present invention relates to the technical field of automated welding, and in particular to a welding seam construction method and electronic equipment. Background Art

[0002] In the field of workpiece welding, it is necessary to search for workpiece welds, which involves the search for straight welds and arc welds. In the existing search method, it is necessary to use the workpiece model point cloud and the weld database. The workpiece model point cloud is a saved point cloud that can be regarded as a standard, and the weld database is the weld-related information for the workpiece model point cloud. In the existing method, generally after the workpiece point cloud is collected by a 3D camera, the workpiece model point cloud is registered with the collected workpiece point cloud to obtain the registration matrix from the workpiece model point cloud to the collected workpiece point cloud. The registration matrix is ​​used to transfer the information in the weld database to the base coordinate system. Then, plane detection algorithms, cylindrical detection algorithms, etc. are used to perform plane and cylindrical detection, and then plane and cylindrical structures are searched and screened, etc., to finally determine the straight welds and arc welds.

[0003] The existing search method requires the use of workpiece model point cloud and an annotated weld database. In some scenarios, if there is a lack of workpiece model point cloud or no annotated weld database information, the existing search method will be limited and welds cannot be identified. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a weld construction method and electronic equipment, which are used to construct all welds of a target workpiece and expand the scope of application of the solution.

[0005] In a first aspect, the present invention provides a method for constructing a weld, the method comprising: Controlling the camera device to shoot the target workpiece multiple times at different positions and postures to obtain multiple local images of the target workpiece, and obtaining a local point cloud in each of the local images; For each of the local point clouds, performing plane detection and cylinder detection on the local point cloud to determine whether there is a plane structure and a cylinder structure in the local point cloud; When a plane is detected, a plane structure is obtained and structural information of the plane structure is saved; when a cylinder is detected, a cylinder structure is obtained and structural information of the cylinder structure is saved; The saved structural information of all plane structures and cylindrical structures are combined and processed, and the weld information of the target workpiece is constructed based on the combined structural information.

[0006] In an optional embodiment, the method further comprises: Determine the plane structure and the cylinder structure having an associated relationship according to the saved structure information of all the plane structures and the structure information of the cylinder structure; The association information of the plane structure and the cylindrical structure having an association relationship is obtained, and the association information is saved.

[0007] In an optional embodiment, the planar structure includes a three-sided planar structure, and the cylindrical structure includes a three-sided cylindrical structure. The three-sided planar structure is a structure having three planes intersecting with each other, and the three-sided cylindrical structure is a structure having one cylindrical surface and two planes, wherein the cylindrical surface intersects with each plane and the two planes intersect with each other. The step of determining the associated plane structures and cylindrical structures according to the stored structural information of all plane structures and structural information of the cylindrical structures comprises: Detecting whether there is a three-sided plane structure and a three-sided cylindrical structure having the same two planes according to the saved structure information of the three-sided plane structure and the structure information of the three-sided cylindrical structure; If so, it is then detected whether the concavity and convexity of the two planes are consistent. If they are consistent, it is determined that the three-sided planar structure with the two planes and the three-sided cylindrical structure have a correlation relationship.

[0008] In an optional embodiment, the step of detecting whether the concavity and convexity of the two planes are consistent includes: Obtaining the concavity and convexity of the two planes in the three-plane structure based on the positional relationship between the center point of the camera device used to photograph the three-plane structure and the two planes in the three-plane structure; Obtaining the concavity and convexity of the two planes in the three-sided cylindrical structure based on the positional relationship between the center point of the camera device that photographs and obtains the three-sided cylindrical structure and the two planes in the three-sided cylindrical structure; It is detected whether the concavity and convexity of the two planes in the three-plane structure are consistent with the concavity and convexity in the three-plane cylindrical structure.

[0009] In an optional implementation manner, the step of merging the saved structural information of all planar structures and structural information of cylindrical structures includes: Determine the associated intersection lines in the structural information of the three-sided planar structure and the structural information of the three-sided cylindrical structure having an associated relationship; Update the voting information of the associated intersection.

[0010] In an optional embodiment, the plane structure includes a three-plane structure, and the cylindrical structure includes a two-cylindrical structure, wherein the two-cylindrical structure is a structure having a cylindrical surface and a plane that intersect each other; The step of determining the associated plane structures and cylindrical structures according to the stored structural information of all plane structures and structural information of the cylindrical structures comprises: Detecting whether there is a 3-sided planar structure and a 2-sided cylindrical structure having a same plane according to the saved structural information of the 3-sided planar structure and the structural information of the 2-sided cylindrical structure; If so, determine the endpoint in the three-sided planar structure that is closest to the cylinder in the two-sided cylindrical structure, and detect whether the distance between the endpoint and the cylinder is less than or equal to a preset distance; If the distance between the endpoint and the cylinder is less than or equal to the preset distance, it is determined that the three-surface planar structure and the two-surface cylindrical structure have an associated relationship.

[0011] In an optional implementation manner, the step of merging the saved structural information of all planar structures and structural information of cylindrical structures includes: Determine the plane structures that can be merged based on the saved structural information of all the plane structures, determine the common intersections of the plane structures that can be merged, and update the voting information of the common intersections; Based on the saved structural information of all cylinder structures, the cylinder structures that can be merged are determined, and the common intersection lines of the cylinder structures that can be merged are determined, and the voting information of the common intersection lines is updated.

[0012] In an optional embodiment, the step of constructing the weld information of the target workpiece according to the merged structural information includes: Obtaining each intersection line and voting information of each intersection line in the merged structural information; An intersection line having a number of votes exceeding a preset number in the voting information is determined to be a weld of the target workpiece, and weld information of the weld is obtained according to the intersection line information of the intersection line.

[0013] In an optional implementation, the step of saving the structural information of the plane structure includes: In the case where the local point cloud is a local point cloud obtained by first shooting, the obtained structural information of the plane structure is saved, and the local point cloud is saved; In the case that the local point cloud is not the local point cloud obtained by the first shooting, the local point cloud is aligned with the point cloud obtained by splicing the local point clouds obtained by the previous shooting to obtain a alignment matrix, the structural information of the obtained planar structure is converted using the alignment matrix, and the converted structural information is saved.

[0014] In a second aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method described in any of the aforementioned implementation modes are implemented.

[0015] The present invention provides a weld construction method and electronic device, which obtains multiple local images by controlling a camera device to shoot a target workpiece multiple times in different postures, and obtains a local point cloud in each local image. For each local point cloud, plane detection and cylinder detection are performed on the local point cloud to determine whether there is a plane structure and a cylindrical structure in the local point cloud. When a plane is detected, a plane structure is obtained, and the structural information of the plane structure is saved. When a cylinder is detected, a cylindrical structure is obtained, and the structural information of the cylindrical structure is saved. The structural information of all the saved plane structures and cylindrical structures is merged, and the weld information of the target workpiece is constructed based on the result information after the merging process. In this solution, a weld database without annotation is not required. Through multiple shooting, the plane structures and cylindrical structures obtained under multiple shootings are merged, so as to construct all welds of the target workpiece, thereby expanding the scope of application of the solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A flow chart of a weld construction method provided by an embodiment of the present invention; Figure 2 for Figure 1 A flowchart of the sub-steps included in S15; Figure 3 for Figure 1 A flowchart of the sub-steps included in S16; Figure 4 A flowchart of a method for determining an association relationship in a weld construction method provided in an embodiment of the present invention; Figure 5 for Figure 4 A flowchart of the sub-steps included in S21; Figure 6 for Figure 5 A flowchart of the sub-steps included in S212A; Figure 7 Schematic diagram of a three-sided planar structure and a three-sided cylindrical structure in an embodiment of the present invention; Figure 8 for Figure 4 Another flow chart of the sub-steps included in S21; Fig. 9 Schematic diagram of a three-surface planar structure and a two-surface cylindrical structure in an embodiment of the present invention; Fig.10 A functional module block diagram of a weld construction device provided by an embodiment of the present invention; Fig.11 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0019] See also Figure 1 , is a flow chart of a weld construction method provided by an embodiment of the present invention, the weld construction method can be executed by a weld construction device, the weld construction device can be implemented by software and / or hardware, and can be configured in an electronic device, the electronic device can be a computer device, or a controller, processor, etc. in a robot. The detailed steps of the weld construction method are described as follows.

[0020] S11, controlling the camera device to shoot the target workpiece multiple times at different positions and postures, obtaining multiple local images of the target workpiece, and obtaining a local point cloud in each of the local images.

[0021] S12, for each of the local point clouds, performing plane detection and cylinder detection on the local point cloud to determine whether there is a plane structure and a cylinder structure in the local point cloud.

[0022] S13, when a plane is detected, obtaining a plane structure and saving structural information of the plane structure.

[0023] S14, obtaining a cylinder structure when a cylinder is detected, and saving structural information of the cylinder structure.

[0024] S15, merging the saved structural information of all plane structures and cylindrical structures.

[0025] S16, constructing and obtaining the weld information of the target workpiece according to the combined structural information.

[0026] In this embodiment, the target workpiece can be any workpiece that needs to be welded for subsequent welding. The target workpiece can include only planes, or planes and cylinders. Therefore, the welds in the target workpiece can include only straight welds, or straight welds and arc welds.

[0027] The camera device may be a 3D camera, and the image of the target workpiece is captured by the camera device. In this embodiment, the camera device may be controlled to capture the target workpiece multiple times in different positions and postures, and each capture may only obtain a partial image of a part of the target workpiece, as long as the collection of all captured partial images can cover all parts of the target workpiece.

[0028] By analyzing and processing each local image, a local point cloud therein, that is, a local point cloud of the target workpiece, can be obtained.

[0029] On this basis, each local point cloud is processed separately. For workpieces with linear welds and arc welds, the local point cloud obtained by local shooting may only capture the planes therein. Therefore, the local point cloud can be firstly subjected to plane detection and cylinder detection to first determine whether the local point cloud only includes planes. If only planes are included, only plane detection and plane structure search are required for the local point cloud. If cylinders are also included, plane detection, cylinder detection, plane structure search, and cylinder structure search are required for the local point cloud.

[0030] Therefore, for the local point cloud, if a plane is detected, the plane structure is searched, and if a cylinder is also detected, the cylinder structure is also searched. The structural information of the found plane structure and cylindrical structure is saved.

[0031] The detection methods of plane detection and cylindrical detection are not limited in this embodiment, and currently commonly used detection methods can be used. On this basis, the method of searching and determining the plane structure and cylindrical structure is also not limited.

[0032] The obtained structural information of the plane structure may include the type of the plane structure, such as a 2-plane structure and a 3-plane structure. The 2-plane structure is a plane structure composed of two intersecting planes, and the 3-plane structure is a plane structure composed of three planes intersecting with each other. In addition, the structural information of the plane structure also includes the information of the intersection lines in the plane structure, such as the voting information of the intersection lines, the starting point and the end point of the intersection lines, etc.

[0033] The structural information of the cylindrical structure includes the type of the cylindrical structure, the information of the intersection lines in the cylindrical structure, etc. The types of cylindrical structures include 2-sided cylindrical structures and 3-sided cylindrical structures, wherein the 2-sided cylindrical structure is a structure including a cylinder and a plane, and the two intersect, but the intersection structure does not exist in any 3-sided cylindrical structure. The 3-sided cylindrical structure is a structure including a cylinder and two planes, and the cylinder intersects with each plane, and the two planes intersect. The intersection lines in the cylindrical structure include straight line intersection lines and circular arc intersection lines.

[0034] Since there may be overlaps in the shooting fields in multiple shots, repeated plane structures or cylindrical structures may exist in different local point clouds. Therefore, in this embodiment, the structural information of the plane structure and the structural information of the cylindrical structure detected for each local point cloud are combined to further determine the weld information of the weld in the target workpiece.

[0035] In this embodiment, the so-called merging is to merge the same plane structures detected from different local point clouds, and merge the same cylindrical structures. When merging, the information of the intersection of the two merged plane structures will be updated, for example, the number of votes in their voting information is increased by 1. Similarly, when two merged cylindrical structures are merged, the voting information of the intersection is updated. In addition, the plane structure and the cylindrical structure may also have overlapping intersections, so the intersections in such plane structures and cylindrical structures can also be merged.

[0036] In this embodiment, by performing a merging process on all the plane structures and cylindrical structures, firstly, repeated counting of the same intersection line can be avoided, and secondly, the weld in the target workpiece can be determined through the specific information of each intersection line after merging.

[0037] Specifically, based on the voting information of the merged intersection lines, it is possible to determine which intersection lines actually exist in the target workpiece, and the intersection lines actually existing in the target workpiece are determined to be welds, thereby obtaining weld information, including information such as the start and end points of the welds.

[0038] In this embodiment, for each local point cloud, when a plane structure is detected and the structural information of the plane structure is saved, it can be achieved in the following manner: In the case where the local point cloud is a local point cloud obtained by first shooting, the obtained structural information of the plane structure is saved, and the local point cloud is saved.

[0039] When the local point cloud is not the local point cloud obtained by the first shooting, the local point cloud is aligned with the point cloud spliced ​​from the local point clouds obtained by the previous shooting to obtain a alignment matrix, and the structural information of the obtained planar structure is converted using the alignment matrix, and the converted structural information is saved.

[0040] In this embodiment, each local point cloud can be numbered in the order of shooting. The local point cloud obtained by the first shooting can be marked as i=0. When i is 0, the local point cloud can be saved as a saved point cloud, and the plane structure detected in the local point cloud is stored in the initialized structure list.

[0041] In order to ensure that the information of each local point cloud is unified when it is subsequently merged, when the local point cloud is not the local point cloud obtained by the first shooting, that is, when i is not 0, the local point cloud needs to be converted. Specifically, the current local point cloud is registered with the point cloud spliced ​​by all the local point clouds shot previously, and a registration matrix from the current local point cloud to the spliced ​​point cloud is obtained, and the registration matrix includes a rotation matrix and a translation matrix. In this way, the structural information of the planar structure detected in the current local point cloud can be converted based on the registration matrix, so that the converted structural information remains in the coordinate system under the same shooting posture as the structural information in the point cloud spliced ​​by the previous shooting.

[0042] In addition, the processing method of the structural information of the cylindrical structure detected in each local point cloud is the same as the processing method of the structural information of the planar structure described above, that is, it is converted to the same coordinate system as the structural information of the cylindrical structure in the local point cloud obtained by the first shooting. Therefore, this embodiment is not described in detail here.

[0043] In this embodiment, for each local point cloud that is not obtained by the first shooting, it is aligned with the point cloud obtained by the previous shooting and stitching. In this way, all subsequent local point clouds can be aligned with the local point cloud obtained by the first shooting, ensuring that all structural information is in the coordinate system with the same posture, and determining the accuracy of subsequent merging and other processing.

[0044] In this embodiment, based on the above unified processing, the obtained structural information of all plane structures and the structural information of the cylindrical structure are combined and processed. For details, please refer to Figure 2 , which can be achieved by: S151, determining the plane structures that can be merged based on the saved structural information of all the plane structures, determining the common intersections of the plane structures that can be merged, and updating the voting information of the common intersections.

[0045] S152, based on the saved structural information of all cylindrical structures, determine the cylindrical structures that can be merged, determine the common intersection lines of the cylindrical structures that can be merged, and update the voting information of the common intersection lines.

[0046] In this embodiment, the planar structure is merged first, and then the cylindrical structure is merged.

[0047] For each local point cloud, the point cloud information of each plane structure detected in the local point cloud is saved. The point cloud information of each plane structure includes the point cloud on each plane in the plane structure. Therefore, by comparison, it can be determined which plane structures include overlapping point clouds.

[0048] For example, if two 2-sided plane structures have two planes respectively included in the two 2-sided plane structures and have the same point clouds (which may be partially or completely the same), it can be determined that the two 2-sided plane structures can be merged. In addition, for two 3-sided plane structures, if the three planes respectively included in the two 3-sided plane structures have the same point clouds (which may be partially or completely the same), it can be determined that the two 3-sided plane structures can be merged. In addition, for a 2-sided plane structure and a 3-sided plane structure, if two planes in the 2-sided plane structure and two planes in the 3-sided plane structure have the same point clouds, it can be determined that the 2-sided plane structure and the 3-sided plane structure can be merged.

[0049] After the plane structures that can be merged are determined, for the intersection lines in the plane structures, the number of votes for the intersection lines may be increased by 1 each time a merge is performed.

[0050] In addition, when performing the merging of cylindrical structures, the merging method is the same as the merging method of the above-mentioned planar structures. The merging of two 3-sided cylindrical structures, the merging of two 2-sided cylinders, and the merging of a 3-sided cylindrical structure and a 2-sided cylindrical structure can be performed respectively. The specific implementation method can refer to the merging method of the above-mentioned planar results, which will not be repeated here.

[0051] On this basis, see Figure 3 , the step of constructing the weld information of the target workpiece according to the combined structural information can be implemented in the following ways: S161, obtaining each intersection line in the structure information after the merge processing and the voting information of each intersection line.

[0052] S162, determining that the intersection line having a number of votes exceeding a preset number in the voting information is a weld of the target workpiece, and obtaining weld information of the weld according to the intersection information of the intersection line.

[0053] In this embodiment, the above-mentioned process of merging each plane structure and cylindrical structure is also a process of updating the voting information of the intersection lines contained therein. Each time a merge is performed, the number of votes of the intersection line is increased by 1. Finally, it can be determined that the intersection line with a number of votes exceeding the preset number is the actual weld in the target workpiece. In this way, the starting point, end point, starting point structure flag, end point structure flag, etc. of the weld can be obtained. Among them, the starting point structure flag refers to the structure identifier of the structure to which the weld starting point belongs, and the end point structure flag refers to the structure identifier of the structure to which the weld end point belongs.

[0054] In this embodiment, in addition to directly merging the plane structures and cylindrical structures that can be merged, there may be some associated plane structures and cylindrical structures among the detected plane structures and cylindrical structures. These associated plane structures and cylindrical structures may be structures with the same intersection line, or the relative positions between the two may be special, and this special relationship may affect the subsequent operation of the welding tool. Therefore, it is necessary to record the plane structures and cylindrical structures with special relationships so that the subsequent welding operation can proceed smoothly.

[0055] Based on the above considerations, please refer to Figure 4 The weld construction method provided in this embodiment further includes the following steps: S21, determining the associated plane structures and cylinder structures according to the stored structure information of all plane structures and the structure information of all cylinder structures.

[0056] S22, obtaining association information of the plane structure and the cylindrical structure having an association relationship, and saving the association information.

[0057] In this embodiment, the determination of the association relationship is mainly the determination of the association relationship between the plane structure and the cylindrical structure, and whether the plane structures and the cylindrical structures can be merged is mainly considered.

[0058] The association relationship between the plane structure and the cylindrical structure mainly considers two situations, the association relationship between the three-sided plane structure and the three-sided cylindrical structure, and the association relationship between the three-sided plane structure and the two-sided cylindrical structure.

[0059] After determining the association information of the associated plane structure and cylindrical structure, the association information is saved, for example, in an association list. Each plane structure and cylindrical structure detected has a structure identifier, and each intersection line and plane also has a corresponding identifier. Therefore, the association information may include the structure identifiers of the associated plane structure and cylindrical structure, as well as the identifiers of the common intersection lines and planes of the two.

[0060] Therefore, please refer to Figure 5 As a possible implementation method, when analyzing the association relationship between the three-sided plane structure and the three-sided cylindrical structure, when determining the plane structure and the cylindrical structure with the association relationship according to the saved structural information of the plane structure and the cylindrical structure, it can be implemented in the following way: S211A, based on the saved structural information of the 3-sided plane structure and the structural information of the 3-sided cylindrical structure, detect whether there is a 3-sided plane structure and a 3-sided cylindrical structure with the same 2 planes. If so, execute the following S212A; if not, execute the following S214A.

[0061] S212A, detect whether the concavity and convexity of the two planes are consistent. If they are consistent, execute the following S213A. If they are inconsistent, execute the following S214A.

[0062] S213A, determining that the three-plane structure having the two planes and the three-cylindrical structure have an associated relationship.

[0063] S214A, there are no three-sided planar structure and three-sided cylindrical structure with an associated relationship.

[0064] In this embodiment, based on determining that the three-sided planar structure and the three-sided cylindrical structure have the same two planes, if the concavity and convexity of the two planes are consistent, it can be determined that the three-sided planar structure and the three-sided cylindrical structure have an associated relationship.

[0065] Among them, see Figure 6 , when detecting whether two identical planes have the same concavity and convexity, it can be achieved in the following ways: S2121A, obtaining the concavity and convexity of the two planes in the three-plane structure based on the positional relationship between the center point of the camera device used to photograph the three-plane structure and the two planes in the three-plane structure.

[0066] S2122A, obtaining the concavity and convexity of the two planes in the three-sided cylindrical structure based on the positional relationship between the center point of the camera device used to photograph the three-sided cylindrical structure and the two planes in the three-sided cylindrical structure.

[0067] S2123A, detecting whether the convexity and concavity of the two planes in the three-plane structure are consistent with the convexity and concavity of the three-plane cylindrical structure.

[0068] Combination Figure 7 The 3-sided planar structure and 3-sided cylindrical structure shown in Figure 7 The 3-sided plane structure includes two 3-sided plane structures, namely the 3-sided plane structure on the left and the 3-sided plane structure on the right. The 3-sided plane structure on the left is composed of two vertical planes on the left and a horizontal plane at the bottom, and the 3-sided plane structure on the right is composed of two vertical planes on the right and a horizontal plane at the bottom. Figure 7 There are two three-sided cylindrical structures in it, which are composed of a cylindrical surface, a horizontal surface above, and a vertical plane on the right, and a cylindrical surface, a horizontal surface below, and a vertical plane on the right.

[0069] In this embodiment, the center point of the shooting device is the tool center point (TCP). The determined identical two planes are placed in the three-plane structure and the three-cylindrical structure for analysis. If the two planes are both concave structures or both convex structures in the two structures, it can be determined that the concavity and convexity of the two planes in the three-plane structure and the three-cylindrical structure are consistent. It can then be determined that the three-plane structure and the three-cylindrical structure have an associated relationship.

[0070] When the 3-sided plane structure and the 3-sided cylindrical structure have an associated relationship, it means that the two have the same intersection line. Therefore, in the above-mentioned step of merging the structural information of the plane structure and the structural information of the cylindrical structure, the associated intersection lines in the structural information of the 3-sided plane structure and the 3-sided structure with an associated relationship can also be determined, and the voting information of the associated intersection lines can be updated, that is, the number of votes for the common intersection lines of the two can be increased by 1. In this way, the merged update of the point clouds of the 2 common planes of the 3-sided plane structure and the 3-sided cylindrical structure, the merged update of the common intersection lines, and the update of the number of votes for the common intersection lines can be achieved.

[0071] Also, see Figure 8 As another possible implementation, when analyzing the association relationship between the three-sided planar structure and the two-sided cylindrical structure, the above-mentioned determination of the planar structure and the cylindrical structure with the association relationship based on the saved structural information of the planar structure and the cylindrical structure can be implemented in the following manner: S211B, based on the saved structural information of the 3-sided planar structure and the structural information of the 2-sided cylindrical structure, detect whether there is a 3-sided planar structure and a 2-sided cylindrical structure with a same plane. If so, execute the following S212B; if not, execute the following S214B.

[0072] S212B, determine the endpoint in the three-sided planar structure that is closest to the cylinder in the two-sided cylindrical structure, and detect whether the distance between the endpoint and the cylinder is less than or equal to a preset distance. If the distance between the endpoint and the cylinder is less than or equal to the preset distance, execute the following S213B; if it is greater, execute the following S214B.

[0073] S213B, determining that the three-surface planar structure and the two-surface cylindrical structure are associated with each other.

[0074] S214B, there are no associated 3-sided planar structures and 2-sided cylindrical structures.

[0075] Please refer to Fig. 9 The 3-sided planar structure and the 2-sided cylindrical structure shown in FIG. 3-sided planar structure refers to Fig. 9 The left side of the middle includes a plane structure consisting of two vertical planes and one horizontal plane. The 2-plane cylindrical structure is Fig. 9 The right side of the center includes a cylindrical structure consisting of a cylinder and horizontal planes located above and below the cylinder.

[0076] for Fig. 9 The 3-sided plane structure and the 2-sided cylindrical structure have the same plane, which is the horizontal plane below the cylindrical surface. Therefore, on this basis, the endpoints in the 3-sided plane structure that are closest to the cylindrical surface in the 2-sided cylindrical structure are determined, where the endpoints refer to the endpoints of each intersection line in the 3-sided plane structure, where the 3 intersection lines have a common endpoint, which is the intersection point of the three, and each of them has an endpoint far away from the intersection point, so the 3-sided plane structure has 4 endpoints.

[0077] The endpoint closest to the cylinder among the four endpoints is determined, wherein the distance between the endpoint and the cylinder is determined by using the point-to-plane distance determination method. On this basis, it is detected whether the obtained distance is less than or equal to the preset distance. If it is less than or equal to the preset distance, it can be determined that the three-sided planar structure and the two-sided cylindrical structure have an associated relationship.

[0078] When the 3-sided planar structure and the 2-sided cylindrical structure have an associated relationship, it means that there is an intersection line in the 3-sided planar structure located near the arc intersection line of the 2-sided cylindrical structure, that is, the distance between the two is very close. In this case, the plane close to the arc intersection line in the 3-sided planar structure is similar to a "baffle", and the welding tool is not easy to operate when welding is performed subsequently using the welding tool. Therefore, in this embodiment, the 3-sided planar structure and the 2-sided cylindrical structure with an associated relationship are determined in the above manner, and the associated information of the two is saved, so that when the 3-sided planar structure and the 2-sided cylindrical structure with an associated relationship are subsequently welded, the posture of the welding tool can be adjusted accordingly, thereby smoothly achieving welding.

[0079] When the three-surface planar structure and the two-surface cylindrical structure are associated with each other, the two structures have a common plane. Therefore, the previously saved point clouds of the two structures can be updated to filter out duplicate point cloud information.

[0080] The weld construction method provided in this embodiment can construct all straight welds and arc welds in the point cloud of the target workpiece. Thus, a complete workpiece point cloud and weld database can be constructed, wherein the starting point, end point, starting point structure mark, and end point structure mark of each weld are identified in the weld database. Based on the constructed workpiece point cloud and weld database, welding of the target workpiece can be performed, and welding of the same type of workpieces in other positions can also be performed. In this way, workpiece welding can be successfully completed without a workpiece model.

[0081] Based on the same inventive concept, please refer to Fig.10, an embodiment of the present invention further provides a functional module schematic diagram of a weld construction device, and this embodiment can divide the functional modules of the weld construction device according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0082] For example, when each functional module is divided into corresponding functional modules, Fig.10 The weld construction device shown is only a schematic diagram of the device. The weld construction device may include a control module, a detection module, a first acquisition module, a second acquisition module, a processing module and a construction module. The functions of each functional module of the weld construction device are described in detail below.

[0083] A control module, used to control the camera to take multiple shots of the target workpiece at different positions and postures, obtain multiple local images of the target workpiece, and obtain a local point cloud in each of the local images; A detection module, for performing plane detection and cylinder detection on each of the local point clouds to determine whether there is a plane structure and a cylinder structure in the local point cloud; A first acquisition module is used to acquire a plane structure when a plane is detected, and save structural information of the plane structure; A second acquisition module is used to acquire a cylinder structure when a cylinder is detected, and save structural information of the cylinder structure; A processing module, used for merging and processing the saved structural information of all plane structures and the structural information of the cylindrical structures; The construction module is used to construct the weld information of the target workpiece according to the combined structural information.

[0084] It can be understood that the above-mentioned control module, detection module, first acquisition module, second acquisition module, processing module and construction module can be used to execute the above-mentioned S11 to S16. The detailed implementation method of the control module, detection module, first acquisition module, second acquisition module, processing module and construction module can refer to the relevant contents of the above-mentioned S11 to S16.

[0085] In a possible implementation, the detection module may also be used to: Determine the plane structure and the cylinder structure having an associated relationship according to the saved structure information of all the plane structures and the structure information of the cylinder structure; The association information of the plane structure and the cylindrical structure having an association relationship is obtained, and the association information is saved.

[0086] In a possible implementation, the plane structure includes a three-plane structure, the cylindrical structure includes a three-cylindrical structure, the three-plane structure is a structure having three planes intersecting with each other, and the three-cylindrical structure is a structure having one cylinder and two planes, wherein the cylinder intersects with each plane and the two planes intersect; the above detection module can be specifically used for: Detecting whether there is a three-sided plane structure and a three-sided cylindrical structure having the same two planes according to the saved structure information of the three-sided plane structure and the structure information of the three-sided cylindrical structure; If so, it is then detected whether the concavity and convexity of the two planes are consistent. If they are consistent, it is determined that the three-sided planar structure with the two planes and the three-sided cylindrical structure have a correlation relationship.

[0087] In a possible implementation, the detection module may be specifically used for: Obtaining the concavity and convexity of the two planes in the three-plane structure based on the positional relationship between the center point of the camera device used to photograph the three-plane structure and the two planes in the three-plane structure; Obtaining the concavity and convexity of the two planes in the three-sided cylindrical structure based on the positional relationship between the center point of the camera device that photographs and obtains the three-sided cylindrical structure and the two planes in the three-sided cylindrical structure; It is detected whether the concavity and convexity of the two planes in the three-plane structure are consistent with the concavity and convexity in the three-plane cylindrical structure.

[0088] In a possible implementation, the processing module may be used to: Determine the associated intersection lines in the structural information of the three-sided planar structure and the structural information of the three-sided cylindrical structure having an associated relationship; Update the voting information of the associated intersection.

[0089] In a possible implementation, the plane structure includes a three-plane structure, the cylindrical structure includes a two-cylindrical structure, and the two-cylindrical structure is a structure having a cylindrical surface and a plane that intersect each other; the above-mentioned detection module can be used for: Detecting whether there is a 3-sided planar structure and a 2-sided cylindrical structure having a same plane according to the saved structural information of the 3-sided planar structure and the structural information of the 2-sided cylindrical structure; If so, determine the endpoint in the three-sided planar structure that is closest to the cylinder in the two-sided cylindrical structure, and detect whether the distance between the endpoint and the cylinder is less than or equal to a preset distance; If the distance between the endpoint and the cylinder is less than or equal to the preset distance, it is determined that the three-surface planar structure and the two-surface cylindrical structure have an associated relationship.

[0090] In a possible implementation, the processing module may be used to: Determine the plane structures that can be merged based on the saved structural information of all the plane structures, determine the common intersections of the plane structures that can be merged, and update the voting information of the common intersections; Based on the saved structural information of all cylinder structures, the cylinder structures that can be merged are determined, and the common intersection lines of the cylinder structures that can be merged are determined, and the voting information of the common intersection lines is updated.

[0091] In a possible implementation, the above building blocks may be used to: Obtaining each intersection line and voting information of each intersection line in the merged structural information; An intersection line having a number of votes exceeding a preset number in the voting information is determined to be a weld of the target workpiece, and weld information of the weld is obtained according to the intersection line information of the intersection line.

[0092] In a possible implementation, the first obtaining module may be used to: In the case where the local point cloud is a local point cloud obtained by first shooting, the obtained structural information of the plane structure is saved, and the local point cloud is saved; In the case that the local point cloud is not the local point cloud obtained by the first shooting, the local point cloud is aligned with the point cloud spliced ​​from the local point clouds shot previously to obtain a alignment matrix, the structural information of the obtained planar structure is converted using the alignment matrix, and the converted structural information is saved.

[0093] See also Fig.11 , is a block diagram of an electronic device provided in an embodiment of the present invention, and the electronic device may be a computer device communicating with a robot, or a controller, a processor, etc. in a robot. The electronic device includes a memory, a processor, and a communication module. The memory, the processor, and the communication module are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components may be electrically connected to each other via one or more communication buses or signal lines.

[0094] The memory is used to store computer programs or data. The memory can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0095] The processor is used to read / write data or programs stored in the memory and execute the weld construction method provided by any embodiment of the present invention.

[0096] The communication module is used to establish a communication connection between the electronic device and other communication terminals through the network, and is used to send and receive data through the network.

[0097] It should be understood that Fig.11 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Fig.11 More or fewer components as shown, or with Fig.11 Different configurations are shown.

[0098] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed, the weld construction method provided in the above embodiment is implemented.

[0099] Specifically, the computer-readable storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the computer-readable storage medium is executed, the above-mentioned weld construction method can be executed. Regarding the process involved when the computer-readable storage medium and its executable instructions are executed, reference can be made to the relevant description in the above-mentioned method embodiment, which will not be described in detail here.

[0100] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0101] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0102] Furthermore, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0103] It should be noted that if the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0104] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0105] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A weld construction method, characterized in that: The method comprises: Controlling the camera device to shoot the target workpiece multiple times at different positions and postures to obtain multiple local images of the target workpiece, and obtaining a local point cloud in each of the local images; For each of the local point clouds, performing plane detection and cylinder detection on the local point cloud to determine whether there is a plane structure and a cylinder structure in the local point cloud; When a plane is detected, a plane structure is obtained and structural information of the plane structure is saved; when a cylinder is detected, a cylinder structure is obtained and structural information of the cylinder structure is saved; The saved structural information of all plane structures and cylindrical structures are combined and processed, and the weld information of the target workpiece is constructed based on the combined structural information.

2. The weld construction method according to claim 1, characterized in that: The method further comprises: Determine the plane structure and the cylinder structure having an associated relationship according to the saved structure information of all the plane structures and the structure information of the cylinder structure; The association information of the plane structure and the cylindrical structure having an association relationship is obtained, and the association information is saved.

3. The weld construction method according to claim 2, characterized in that: The plane structure includes a three-plane structure, and the cylindrical structure includes a three-cylindrical structure. The three-plane structure is a structure having three planes intersecting with each other, and the three-cylindrical structure is a structure having one cylinder and two planes, wherein the cylinder intersects with each plane and the two planes intersect with each other. The step of determining the associated plane structures and cylindrical structures according to the stored structural information of all plane structures and structural information of the cylindrical structures comprises: Detecting whether there is a three-sided plane structure and a three-sided cylindrical structure having the same two planes according to the saved structure information of the three-sided plane structure and the structure information of the three-sided cylindrical structure; If so, it is then detected whether the concavity and convexity of the two planes are consistent. If they are consistent, it is determined that the three-sided planar structure with the two planes and the three-sided cylindrical structure have a correlation relationship.

4. The weld construction method according to claim 3, characterized in that: The step of detecting whether the concavity and convexity of the two planes are consistent comprises: Obtaining the concavity and convexity of the two planes in the three-plane structure based on the positional relationship between the center point of the camera device used to photograph the three-plane structure and the two planes in the three-plane structure; Obtaining the concavity and convexity of the two planes in the three-sided cylindrical structure based on the positional relationship between the center point of the camera device that photographs and obtains the three-sided cylindrical structure and the two planes in the three-sided cylindrical structure; It is detected whether the concavity and convexity of the two planes in the three-plane structure are consistent with the concavity and convexity in the three-plane cylindrical structure.

5. The weld construction method according to claim 3, characterized in that: The step of merging the saved structural information of all plane structures and the structural information of the cylindrical structures comprises: Determine the associated intersection lines in the structural information of the three-sided planar structure and the structural information of the three-sided cylindrical structure having an associated relationship; Update the voting information of the associated intersection.

6. The weld construction method according to claim 2, characterized in that: The plane structure includes a three-plane structure, and the cylindrical structure includes a two-cylindrical structure, wherein the two-cylindrical structure is a structure having a cylindrical surface and a plane that intersect each other; The step of determining the associated plane structures and cylindrical structures according to the stored structural information of all plane structures and structural information of the cylindrical structures comprises: Detecting whether there is a 3-sided planar structure and a 2-sided cylindrical structure having a same plane according to the saved structural information of the 3-sided planar structure and the structural information of the 2-sided cylindrical structure; If so, determine the endpoint in the three-sided planar structure that is closest to the cylinder in the two-sided cylindrical structure, and detect whether the distance between the endpoint and the cylinder is less than or equal to a preset distance; If the distance between the endpoint and the cylinder is less than or equal to the preset distance, it is determined that the three-surface planar structure and the two-surface cylindrical structure have an associated relationship.

7. The weld construction method according to claim 1, characterized in that: The step of merging the saved structural information of all plane structures and the structural information of the cylindrical structures comprises: Determine the plane structures that can be merged based on the saved structural information of all the plane structures, determine the common intersections of the plane structures that can be merged, and update the voting information of the common intersections; Based on the saved structural information of all cylinder structures, the cylinder structures that can be merged are determined, and the common intersection lines of the cylinder structures that can be merged are determined, and the voting information of the common intersection lines is updated.

8. The weld construction method according to claim 7, characterized in that: The step of constructing the weld information of the target workpiece according to the combined structural information includes: Obtaining each intersection line and voting information of each intersection line in the merged structural information; An intersection line having a number of votes exceeding a preset number in the voting information is determined to be a weld of the target workpiece, and weld information of the weld is obtained according to the intersection line information of the intersection line.

9. The weld construction method according to claim 1, characterized in that: The step of storing the structural information of the plane structure includes: In the case where the local point cloud is a local point cloud obtained by first shooting, the obtained structural information of the plane structure is saved, and the local point cloud is saved; In the case that the local point cloud is not the local point cloud obtained by the first shooting, the local point cloud is aligned with the point cloud obtained by splicing the local point clouds obtained by the previous shooting to obtain a alignment matrix, the structural information of the obtained planar structure is converted using the alignment matrix, and the converted structural information is saved.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

Images