Welded part point cloud normal vector direction correction and weld joint identification method

Abstract

The invention discloses a welding part point cloud normal vector direction correction and welding seam identification method. Comprising the following steps: calibrating a 3D camera and a turntable to obtain a conversion matrix; the rotary table is driven to scan the workpiece at multiple angles and splice point clouds; aiming at each point in the point cloud, calculating a finger-eye vector pointing to the scanning camera according to the calibration parameter; and by comparing the direction relationship between the initial normal vector and the finger eye vector, the normal vector direction is uniformly corrected, so that the normal vector direction is consistent to the outside of the workpiece. The method comprises the following steps: segmenting the point cloud with the consistent normal direction to obtain a workpiece surface area; and a geometrical relationship is calculated by utilizing consistent outward normal vectors among the regions, and effective welding seams are accurately filtered and extracted. According to the method, the problem of ambiguity of the normal vector direction of the point cloud in multi-view scanning is thoroughly solved, and a reliable data basis is provided for automatic welding seam recognition.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional machine vision and automated welding technology, specifically a method for correcting the direction of the point cloud normal vector of a welded part and identifying the weld seam. Background Technology

[0002] In machine vision-based intelligent welding systems, acquiring point cloud data of the workpiece through 3D scanning and automatically planning the welding path based on this data is a key technology for achieving welding automation and intelligence. In this process, accurate estimation of the point cloud normal vectors is fundamental for subsequent processing such as point cloud registration, surface reconstruction, feature extraction, and weld seam recognition.

[0003] Currently, commonly used point cloud processing libraries (such as Open3D, PCL, and CGAL) generally employ Principal Component Analysis (PCA) to estimate the normal vectors of points. While PCA can calculate the direction of the normal vector based on the local neighborhood distribution of a point, it has an inherent limitation: it can only determine the direction of the line containing the normal vector (i.e., both positive and negative directions), but cannot uniquely determine whether the normal vector should point to the outside or inside of the surface. This uncertainty in direction is often referred to as the "ambiguity" or "inconsistency" of the normal vector direction.

[0004] In welding applications, this inconsistency in direction can lead to a series of problems. For example, when identifying fillet welds and T-welds, it is usually necessary to determine the type of the angle between the two surfaces and extract their intersection line as the weld based on the geometric relationship between the normal vectors of adjacent surfaces (such as cross product and dot product operations). If the directions of the normal vectors are inconsistent (some point to the outside of the workpiece, and some point to the inside), it will directly cause the geometric judgment logic based on the normal vectors to fail, making it impossible to accurately distinguish between valid welds and invalid plane intersection lines, thus seriously affecting the accuracy and reliability of automatic weld extraction.

[0005] To overcome this deficiency in PCA methods, existing techniques typically require the introduction of additional global or local information to unify the normal vector direction, such as using the viewpoint consistency principle for orientation. However, in typical scenarios of weldment scanning, the workpiece is usually fixed on a turntable for multi-view scanning to obtain a complete point cloud, and the "viewpoint" position under each viewpoint is dynamically changing. When processing such a holistic point cloud after multi-view point cloud stitching, traditional methods struggle to find a stable and unified global viewpoint for effective normal vector orientation, leading to unreliable or incomplete orientation results.

[0006] Therefore, existing technologies lack a solution that can effectively adapt to multi-view scanning scenarios of welded parts and ensure that the normal vector direction of the overall point cloud after splicing is consistent and points outward from the workpiece. This technological bottleneck restricts the accuracy and practicality of point cloud-based automated weld seam recognition technology. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method for correcting the consistency of the normal vector direction of the point cloud of welded parts and for weld seam recognition. This method aims to solve the problem of inconsistent normal vector direction of the point cloud of welded parts. By calibrating the camera and turntable parameters, calculating the eye-pointing vector, and correcting the normal vector direction, the method ensures that the normal vector consistently points to the outside of the workpiece, thereby improving the accuracy and reliability of weld seam recognition.

[0008] To achieve the above objectives, the present invention provides the following technical solution: I. A method for correcting the consistency of the normal vector direction of a welded component's point cloud, comprising the following steps: Step S1: Calibrate the 3D camera used to acquire the point cloud of the welded part and the turntable that carries the welded part, and obtain the camera extrinsic matrix and the turntable transformation matrix respectively. Step S2: Control the turntable to rotate the welded part at multiple angles, and use the 3D camera to capture a frame of point cloud at each angle, which can be stored in the computer; subsequently, process the point cloud data in the computer using the camera extrinsic matrix and the turntable transformation matrix, including transforming and stitching the point clouds of each frame to the same reference coordinate system based on the turntable transformation matrix, to obtain the transformed single point cloud and the stitched point cloud. Step S3: In the computer, for each point of each single point cloud in the stitched point cloud, the vector pointing to the 3D camera coordinates at the corresponding acquisition time is calculated based on the camera extrinsic matrix and the turntable transformation matrix as the eye-finding vector. Then, the eye-finding vector is used to perform consistency correction processing on the initial normal vector of each point in the stitched point cloud.

[0009] The reference coordinate system in step S2 is the coordinate system where the turntable is located, which is the coordinate system of frame 0, that is, the three-dimensional coordinate system established with the turntable as a reference, which serves as the world coordinate system.

[0010] In step S1, the camera extrinsic parameter matrix T cam Represented as: ; Where: R cam Let R be the rotation matrix. cam ∈R 3×3 ;t cam It is a translation vector; The turntable transformation matrix T rot Represented as: ; Where: t rot R represents the coordinates of the turntable center point; rot This is the rotation matrix about the axis of rotation, and its value is determined by the axis of rotation and the rotation angle of the turntable.

[0011] In step S2, the point cloud stitching method is as follows: stitch the i-th frame of point cloud... i p is rotated to the reference coordinate system, i.e., the coordinate system of frame 0, to obtain the single point cloud of frame i. i 0p, and then integrate and stitch together the point cloud: i 0p= i p T rot -1 (θ rot ) Where: θ rot T represents the rotation angle of the point cloud in frame i relative to the point cloud in frame 0; rot This is the turntable transformation matrix.

[0012] Specifically, step S3 involves calculating the finger-eye vector using the following methods: S31. Calculate the camera coordinates corresponding to each frame of point cloud using the following formula. i 0p cam : i 0p cam= i p cam T rot (θ rot ) in: i p cam Let be the coordinates of the camera in the point cloud of the i-th frame. i 0p cam These are coordinates relative to the reference coordinate system of frame 0. S32. The i-th frame of a single point cloud in the stitched point cloud i Subtracting the corresponding camera coordinates from the coordinates of each point on 0p yields the finger-eye vector for each point: i v= i 0p- i 0p cam in: i 0p represents the set of coordinates of all points in the single point cloud of the i-th frame. i 0p cam This represents the camera coordinates corresponding to a single point cloud in the i-th frame. i v represents the set of finger-eye vectors for each point in the single point cloud of the i-th frame.

[0013] In step S3, the consistency correction process includes: 31) First, calculate the initial normal vector of each point in the stitched point cloud; 32) Then, for each point, calculate the inner product of its initial normal vector and its corresponding finger-eye vector; If the inner product is greater than 0, then the direction of the initial normal vector of that point is flipped; If the inner product is not greater than 0, no processing is performed; This ensures that the initial normal vector direction of all points satisfies the predetermined consistency relationship with the direction of their respective finger-eye vector, thereby achieving consistency correction.

[0014] In step 31), the initial normal vector is calculated using principal component analysis (PCA) based on the coordinate data of each point in the stitched point cloud.

[0015] In step 32), the predetermined consistency relationship is: the direction of the normal vector of each point is consistent with the direction of its corresponding eye-pointing vector, that is, the inner product is less than 0.

[0016] The welded parts described in this invention are metal welded parts, which can typically be building components, aerospace parts, mechanical parts, etc.

[0017] II. A method for identifying weld seams in welded components, comprising the following steps: Step 1: Obtain point cloud data of the welded part, wherein the normal vector direction of each point in the point cloud data has been corrected by the welded part point cloud normal vector direction consistency correction method as described in any one of claims 1-7, to ensure that all normal vectors point to the outside of the welded part entity; Step 2: Segment the point cloud data to obtain multiple regions representing different surfaces of the welded parts; Step 3: Based on the normal vector information of each region after consistency correction, identify the valid welds on the welded parts.

[0018] In step two, a continuous smooth surface of the welded part is divided into regions, each region representing a surface, which can be a plane or a curved surface; adjacent different regions are connected by intersecting straight lines or curves.

[0019] In step three, the method for identifying valid welds is as follows: For any two intersecting surface regions, based on their respective normal vectors n1 and n2, the determination operator σ is calculated according to the expression, and then the determination is made. σ=( n1×n2)×e1 Where: e1 is a preset reference unit vector, the direction of which is the direction of the intersection line between two intersecting surface regions, and the specific direction is determined by the normal vector n1 according to the right-hand screw rule; When σ < 0, the angle between the two planes is determined to be less than 180°; when σ > 0, the angle between the two planes is greater than 180°.

[0020] When the operator σ < 0, the intersection of the two surface regions is determined to be a valid weld.

[0021] When the operator σ≥0, the intersection of the two surface regions is not considered a valid weld.

[0022] The innovation of this invention lies in the fact that after splicing the point cloud of the welded parts, the finger-eye vector is calculated, and then the normal vector of each point in the point cloud is corrected by using the finger-eye vector, and a decision operator processing relationship is established to detect and determine the weld of the welded parts.

[0023] The beneficial effects of this invention are as follows: The weld seam recognition method of this invention fundamentally solves the industry problem of inconsistent normal vector directions of point clouds in welded parts under multi-view scanning scenarios, providing a reliable foundation for subsequent advanced geometric processing, and achieving the following technical effects: (1) Achieved highly reliable unified orientation of normal vectors: By accurately calibrating the camera and turntable, and innovatively calculating the dynamic "pointing-eye vector" for each point as the direction reference, the normal vectors of all points can be robustly corrected to point to the outside of the workpiece entity, thus completely eliminating the inherent directional ambiguity of the PCA method. (2) Directly improves the accuracy and success rate of automatic weld identification: Based on the normal vector with consistent direction and clear physical meaning, when performing plane / surface segmentation and intersection geometry judgment, the algorithm can accurately distinguish between valid welds (such as fillet welds) and invalid surface intersections, significantly reducing false detections and missed detections, making the weld extraction results more reliable; (3) Enhanced the practicality and stability of the overall system: The method of this invention closely follows the standard industrial process of "rotary table multi-view scanning", without changing the existing hardware configuration or introducing complex assumptions. Through rigorous mathematical transformation, dynamic viewpoint information is transformed into stable global direction constraints. The solution has strong engineering practicality and lays a solid data foundation for the automatic path planning of intelligent welding. Attached Figure Description

[0024] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration: Figure 1 This is a flowchart of the method for correcting the consistency of the normal vector direction of the welded part in this invention; Figure 2 This is a schematic diagram of a 3D camera and a turntable. Figure 3 To obtain the point cloud of the welded part by stitching together; where (a) represents the actual welded part, (b) represents the multi-frame point cloud obtained by 3D camera scanning, and (c) represents the stitched point cloud; Figure 4 The image shows a comparison of the point cloud normal vectors before and after correction. In the image, blue represents the eye vector and red represents the normal vector; (a) is before correction; (b) is after correction. Figure 5 This is a flowchart of the weld seam identification method for welded parts according to the present invention; Figure 6 This is a schematic diagram of planar segmentation using RANSAC; (a) shows the actual welded part, and (b) shows the point cloud planar segmentation result, with each planar segmentation result marked with a different color; Figure 7 The relationship between a valid weld and a correct normal vector is given; where (a) represents the case where the decision operator σ < 0, i.e., a valid weld, and (b) represents the case where σ > 0, i.e., a non-valid weld. Figure 8 To identify valid welds by filtering them using the correct normal vector; Figure 9 These are examples of welded components extracted based on the premise of ensuring the consistency of the normal vector direction. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0026] like Figure 1 As shown in the figure, the method for correcting the consistency of the normal vector direction of the welded part point cloud in this embodiment includes the following steps.

[0027] Step S1: Calibrate the 3D camera used to acquire the point cloud of the welded part and the turntable that carries the welded part, and obtain the camera extrinsic parameter matrix and the turntable transformation matrix respectively.

[0028] like Figure 2 As shown, the welded parts are placed on a turntable, and a fixed 3D camera is used to capture multiple frames of point clouds.

[0029] Hand-eye calibration of a 3D camera, camera extrinsic parameter matrix T cam ∈R 4×4 Represented as: ; Where: R cam Let R be the rotation matrix. cam ∈R 3×3 R represents the real number space; t cam It is a translation vector.

[0030] The camera's own coordinates are: p cam =R cam t cam Where: p cam These are the camera coordinates.

[0031] The turntable is calibrated using a 3D camera, and the turntable transformation matrix T is used. rot Represented as: ; Where: t rot R represents the coordinates of the turntable center point; rot This is the rotation matrix about the axis of rotation, and its value is determined by the axis of rotation and the rotation angle of the turntable.

[0032] Step S2: Control the turntable to rotate the welded part at multiple angles, and use the 3D camera to acquire a frame of point cloud at each angle; based on the turntable transformation matrix, stitch all the frame point clouds together in the same coordinate system to obtain the stitched single point cloud of the welded part.

[0033] Specifically, the welded component is placed on a turntable, and a 3D camera scans the component at each rotation angle to obtain multi-frame point clouds. In this embodiment, the point cloud stitching method is as follows: the i-th frame of point cloud... i p is rotated to the reference coordinate system, i.e., the coordinate system of frame 0, to obtain the single point cloud of frame i. i 0p, and then integrate and stitch together the point cloud: i 0p= i pT rot -1 (θ rot ) Where: θ rot T represents the rotation angle of the point cloud in frame i relative to the point cloud in frame 0; rot This is the turntable transformation matrix.

[0034] Point cloud stitching effect as Figure 3 As shown. Figure 3 (a) is a photograph of the actual welded part. Figure 3 (b) The effect of directly stitching together each frame of point cloud acquired by the 3D camera (without rotation processing). Figure 3 (c) shows the point cloud stitching effect after using the above rotation operation.

[0035] Step S3: For each point in the stitched single point cloud, calculate the vector pointing from the point to the 3D camera coordinates at the corresponding acquisition time based on the camera extrinsic matrix and the turntable transformation matrix, denoted as the eye-pointing vector.

[0036] For ease of description and understanding, the turntable is considered stationary, while the camera rotates around it. In this embodiment, the method for calculating the finger-eye vector includes: calculating the camera coordinates corresponding to each frame of the point cloud: i 0p cam= i p cam T rot (θ rot ) in:i p cam Let be the coordinates of the camera in the point cloud of the i-th frame. i 0p cam These are coordinates relative to the reference coordinate system of frame 0. Obtain the finger-eye vector for each point: i v= i 0p- i 0p cam in: i v represents the set of finger-eye vectors for each point in the single point cloud of the i-th frame.

[0037] It is not difficult to conclude that this vector always points outward from the weldment.

[0038] Step S4: Calculate the initial normal vector of each point in the single point cloud; for each point, calculate the inner product of its initial normal vector and its finger-eye vector; if the inner product is greater than 0, flip the direction of the normal vector of the point so that the direction of the normal vector of all points satisfies the predetermined consistency relationship with the direction of the corresponding finger-eye vector.

[0039] In this embodiment, the initial normal vector of the point cloud is calculated using existing conventional methods. i n, calculate its inner product with the corresponding finger-eye vector: i ω= i n× i v if i ω>0 indicates that the normal vector i n and the finger-eye vector i The direction of the normal vector is opposite, requiring correction of its corresponding direction. In this embodiment, the predetermined consistency relationship is: the direction of the normal vector at each point is the same as the direction of its eye-pointing vector, i.e., the inner product is less than 0. In this embodiment, the initial normal vector is calculated using Principal Component Analysis (PCA), and the effect of normal vector correction is as follows: Figure 4 As shown. Figure 4 The blue line represents the eye vector, and the red line represents the normal vector. Figure 4 The normal vector in (a) has not been corrected for its normal direction. Figure 4 The normal vector in (b) is the result after correction using the normal vector direction.

[0040] The following describes the specific implementation of the weld seam identification method of the present invention, in conjunction with the above-described method for correcting the consistency of the normal vector direction of the welded component point cloud in this embodiment.

[0041] like Figure 5 As shown, the weld seam identification method for welded parts in this embodiment includes the following steps.

[0042] Step 1: Obtain the point cloud data of the welded part. The normal vector direction of each point in the point cloud data has been corrected by the welded part point cloud normal vector direction consistency correction method described in this embodiment to ensure that all normal vectors point to the outside of the welded part entity.

[0043] Step 2: Segment the point cloud data to obtain multiple regions representing different surfaces of the welded parts.

[0044] In this embodiment, conventional methods such as RANSAC and region growing are used to segment the point cloud, obtaining a series of planes or curved surfaces representing the surface features of the welded part. This embodiment takes plane segmentation using RANSAC as an example. Figure 6 As shown, (a) represents the actual welded part, and (b) is the point cloud plane segmentation result. Each plane segmentation result is marked with a different color.

[0045] Step 3: Based on the normal vector information of each region, identify the valid welds on the weldment.

[0046] Based on the point cloud segmentation results, a series of intersection lines that could potentially become welds can be obtained by calculating the intersection lines between planes / surfaces. There are usually many combinations of these planes intersecting each other, but the combinations that can actually form effective welds are much fewer. Therefore, the method proposed in this embodiment to ensure the consistency of normal vector direction is needed to filter out effective welds in the curves.

[0047] Specifically, in this embodiment, the method for identifying valid welds is as follows: for any two intersecting surface regions, whose normal vectors are n1 and n2 respectively, the expression is calculated as follows: σ=( n1×n2)×e1 Where: e1 is a preset reference unit vector, the direction of which is the direction of the intersection line between two intersecting surface regions, and the specific direction is determined by the normal vector n1 according to the right-hand screw rule; When σ < 0, the angle between the two planes is determined to be less than 180°; when σ > 0, the angle between the two planes is greater than 180°, such as... Figure 7 As shown in the diagram, (a) represents the case where the decision operator σ < 0, indicating a valid weld, and (b) represents the case where σ > 0, indicating a non-valid weld. Therefore, when the decision operator σ < 0, the intersection of the two surface regions is determined to be a valid weld. However, when the decision operator σ ≥ 0, the intersection of the two surface regions is not determined to be a valid weld.

[0048] Thus, the combination of intersecting planes can be filtered using normal vectors. For common fillet welds and T-welds, σ<0 can be used as the filtering condition. Note that the prerequisite for this judgment rule to hold is that the two normal vectors n1 and n2 are correctly pointing to the outside of the weldment, which is also the key significance of the method proposed in this embodiment.

[0049] like Figure 8 As shown, when two workpieces are in contact, by ensuring the consistency of the normal vector direction, normal vectors n1 and n2 pointing outwards from the workpiece are obtained. Based on the above judgment rule, p1p2 can be determined as a valid weld.

[0050] like Figure 9 The figure shows an example of automatic weld seam extraction based on the method proposed in this embodiment. As can be seen from the figure, by correcting the direction of the normal vector and applying the decision operator, all valid weld seams are successfully extracted, while invalid edges are filtered out.

[0051] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for correcting the consistency of the normal vector direction of a welded component's point cloud, characterized in that: Includes the following steps: Step S1: Calibrate the 3D camera used to acquire the point cloud of the welded part and the turntable that carries the welded part, and obtain the camera extrinsic matrix and the turntable transformation matrix respectively. Step S2: Control the turntable to rotate the welded part by multiple angles, and use the 3D camera to acquire a frame of point cloud at each angle; based on the turntable transformation matrix, stitch the point clouds of each frame to the same reference coordinate system to obtain the transformed single point cloud and the stitched point cloud. Step S3: For each point in the stitched point cloud, calculate the vector pointing to the 3D camera coordinates at the corresponding acquisition time based on the camera extrinsic matrix and the turntable transformation matrix. Then, use the eye-point vector to perform consistency correction processing on the initial normal vector of each point in the stitched point cloud.

2. The method for correcting the consistency of the normal vector direction of the welded component point cloud according to claim 1, characterized in that: In step S1, the camera extrinsic parameter matrix T cam Represented as: ； Where: R cam t is the rotation matrix; cam It is a translation vector; The turntable transformation matrix T rot Represented as: ； Where: t rot R represents the coordinates of the turntable center point; rot Let be the rotation matrix about the axis of rotation.

3. The method for correcting the consistency of the normal vector direction of the point cloud of welded parts according to claim 1, characterized in that: In step S2, the point cloud stitching method is as follows: stitch the i-th frame of point cloud... i Rotate p to the reference coordinate system to obtain the single point cloud of the i-th frame. i 0p, and then integrate and stitch together the point cloud: i 0p= i p T rot -1 (i rot ) Where: θ rot T represents the rotation angle of the point cloud in frame i relative to the point cloud in frame 0; rot This is the turntable transformation matrix.

4. The method for correcting the consistency of the normal vector direction of the point cloud of welded parts according to claim 3, characterized in that: Specifically, step S3 involves calculating the finger-eye vector using the following methods: S31. Calculate the camera coordinates corresponding to each frame of point cloud using the following formula. i 0p cam : i 0p cam= i p cam T rot (i rot ) in: i p cam Let be the coordinates of the camera in the point cloud of the i-th frame. i 0p cam These are coordinates relative to the reference coordinate system. S32. The i-th frame of a single point cloud in the stitched point cloud i Subtracting the corresponding camera coordinates from the coordinates of each point on 0p yields the finger-eye vector for each point: i in= i 0p- i 0p cam in: i 0p represents the set of coordinates of all points in the single point cloud of the i-th frame. i 0p cam This represents the camera coordinates corresponding to a single point cloud in the i-th frame. i v represents the set of finger-eye vectors for each point in the single point cloud of the i-th frame.

5. The method for correcting the consistency of the normal vector direction of the point cloud of welded parts according to claim 1, characterized in that: In step S3, the consistency correction process includes: 31) First, calculate the initial normal vector of each point in the stitched point cloud; 32) Then, for each point, calculate the inner product of its initial normal vector and its corresponding finger-eye vector; If the inner product is greater than 0, then the direction of the initial normal vector of that point is flipped; If the inner product is not greater than 0, no processing is performed; This ensures that the initial normal vector direction of all points satisfies the predetermined consistency relationship with the direction of their respective finger-eye vector, thereby achieving consistency correction.

6. The method for correcting the consistency of the normal vector direction of the point cloud of a welded component according to claim 5, characterized in that: In step 31), the initial normal vector is calculated based on the spliced ​​point cloud using the principal component analysis (PCA) method.

7. The method for correcting the consistency of the normal vector direction of the point cloud of welded parts according to claim 5, characterized in that: In step 32), the predetermined consistency relationship is: the direction of the normal vector of each point is consistent with the direction of its eye-pointing vector, that is, the inner product is less than 0.

8. A method for identifying weld seams in welded components, characterized in that: The process includes the following steps: Step 1: Obtain point cloud data of the welded part, wherein the normal vector direction of each point in the point cloud data is corrected by the welded part point cloud normal vector direction consistency correction method as described in any one of claims 1-7, to ensure that all normal vectors point to the outside of the welded part entity; Step 2: Segment the point cloud data to obtain multiple regions representing different surfaces of the welded parts; Step 3: Based on the normal vector information of each region after consistency correction, identify the valid welds on the welded parts.

9. The method for identifying weld seams in welded components according to claim 8, characterized in that: In step three, the method for identifying valid welds is as follows: For any two intersecting surface regions, based on their respective normal vectors n1 and n2, the determination operator σ is calculated according to the expression, and then the determination is made. σ=(n1×n2)×e1 Where: e1 is a preset reference unit vector, the direction of which is the direction of the intersection line between two intersecting surface regions, and is determined by the normal vector n1 according to the right-hand screw rule; When the operator σ < 0, the intersection of the two surface regions is determined to be a valid weld. When the operator σ≥0, the intersection of the two surface regions is not considered a valid weld.

Images