A method and system for three-dimensional visual identification of welds of small group structure parts

Abstract

This application belongs to the field of intelligent manufacturing and welding automation technology, specifically disclosing a three-dimensional visual recognition method and system for weld seams in small-scale structural components. This application extracts a reference plane from the base plate by fitting it, aligns the base plate normal to the global coordinate system Z-axis, and then combines the mean and standard deviation of the base plate point cloud's height to perform layered segmentation of the ground point cloud, base plate point cloud, and stiffener point cloud. Combined with region growing based on normal vector and curvature constraints, and reconstruction based on the pairwise relationship of plane normals and centroid collinearity, it achieves automatic separation of single stiffener point cloud clusters. Robust recognition of the base plate and stiffeners can be completed without relying on the three-dimensional model of the structural component, and the three-dimensional information of the weld seam can be obtained based on this.

Description

Technical Field

[0001] This application belongs to the field of intelligent manufacturing and welding automation technology, and more specifically, relates to a three-dimensional visual recognition method and system for weld seams of small-scale structural components. Background Technology

[0002] Welding is a crucial step in shipbuilding, accounting for a significant portion of the workload and cost. Sub-assembly structural components, typically composed of flat plates and stiffening plates, are the basic units of the hull sections. They are characterized by large work areas, numerous and densely distributed welds, making them suitable for robotic welding on production lines. However, in some shipyards, welding of sub-assembly structural components still relies on manual labor, which not only limits efficiency and makes it difficult to guarantee consistent quality but also faces practical problems such as harsh working environments and a shortage of highly skilled welders. Therefore, promoting the automation and intelligent upgrading of sub-assembly structural component welding has significant engineering implications and application value.

[0003] Currently, industrial welding robots can generally be categorized into teach-and-playback type, offline programming type, and intelligent welding type based on their operation organization methods. Teach-and-playback robots generate welding trajectories through manual point-to-point teaching. Their applicability relies on the workpiece's stable position and orientation within the workstation, consistent with the teaching program. However, in ship assembly structural components, assembly errors, misalignment, and weld morphology fluctuations are common, making it difficult to maintain consistent workpiece posture. This often necessitates repeated adjustments to the workpiece's position before welding to match the existing program, requiring multiple teaching sessions and manual corrections, resulting in a large workload, low efficiency, and insufficient flexibility. Offline programming robots rely on the workpiece's 3D model and process information for weld recognition, trajectory planning, and program generation. While reducing on-site teaching, they are highly dependent on model accuracy and assembly consistency, and typically involve model acquisition and automated welding program writing, making the overall process complex. In actual manufacturing, dimensional deviations and component deformation can easily cause inconsistencies between the offline trajectory and the actual weld, requiring on-site alignment and correction. In contrast, intelligent welding robots improve environmental adaptability through 3D vision and multi-sensor fusion, but the key to achieving stable and high-quality welding lies in reliable 3D perception of the welding scene and accurate acquisition of the spatial geometric information of the weld seam to support automatic positioning, trajectory generation and online correction.

[0004] Welding systems based on 3D sensing typically acquire 3D point cloud data by scanning the work area, and then perform scene reconstruction, weld location, and trajectory generation based on this data. For ship assembly components, the point cloud data often contains noise interference and occlusion due to factors such as large structural scale spans, numerous and adjacent welds, and complex surface morphology. Furthermore, assembly deviations can cause uncertainty in weld geometry, making weld extraction more challenging. Existing 3D sensing technology still faces the following prominent problems in the application of weld extraction for ship assembly components: 1. Large-sized workpieces often exceed the coverage of a single field of view, requiring multi-view measurement and splicing within a large workspace. In large-scale, densely repetitive structural scenarios like ship assembly components, registration and splicing are prone to getting trapped in local optima, leading to registration and splicing failure; 2. Ribs and other structures are prone to self-occlusion, and the strong reflectivity, oxide scale, and unstable grinding texture of steel plate surfaces make feature extraction based on a single sensor susceptible to interference, resulting in missed detections, false detections, or excessively large fluctuations in measurement errors. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this application is to provide a three-dimensional visual recognition method and system for weld seams in small assembled structural components. This system aims to solve the problems of unstable acquisition of weld seam three-dimensional information and large workload for pre-welding preparation and on-site correction in scenarios involving large-size, multi-weld, and complex small assembled structural components, which are affected by factors such as insufficient field of view coverage, missing occlusions, assembly deviations, and cumulative errors from multi-view splicing.

[0006] To achieve the above objectives, in a first aspect, this application provides a three-dimensional visual recognition method for weld seams in small-scale structural components, comprising: Using the reference camera coordinate system as the world coordinate system, the point cloud acquired by each camera in the line laser scanning camera group at the same time is uniformly transformed to the world coordinate system through the pose transformation matrix between cameras, so as to achieve geometric alignment of the point cloud from multiple perspectives. The line laser scanning camera group consists of multiple line laser cameras arranged side by side along the width direction of the small assembled structure, which are used to synchronously scan and acquire small assembled structures moving along the guide rail. The point clouds of geometrically aligned multi-view structural components are stitched together to obtain a complete point cloud; Extract the base plate reference plane using the overall point cloud, and obtain the base plate point cloud belonging to the base plate reference plane. Align the normal of the base plate reference plane with the Z-axis of the global coordinate system. Based on the mean and standard deviation of the height under the global coordinate system Z-axis of the base plate point cloud, the height layering threshold is adaptively determined to divide the complete point cloud into ground point cloud, base plate point cloud and rib plate point cloud; Under the dual constraints of normal continuity and curvature upper limit, the stiffener point cloud is clustered to obtain multiple approximately coplanar stiffener plane clusters. Merging rib planar clusters with paired normals and collinear centroids yields multiple independent single rib point clouds. For each individual stiffener point cloud, a local two-dimensional uv coordinate system aligned with the stiffener body is constructed based on the individual stiffener point cloud. After projecting the individual stiffener point cloud onto the local two-dimensional uv coordinate system, the weld seam division and endpoint estimation are adaptively completed based on the coordinate distribution statistics of the point cloud in the u and v directions. The distance between the start and end points of the weld is taken as the weld length, and the vector from the start point to the end point of the weld is taken as the direction vector of the weld in the local two-dimensional uv coordinate system. By transforming the local two-dimensional UV coordinate system with the world coordinate system, the starting coordinates and direction information of the weld in the base plate plane are obtained.

[0007] Preferably, the step of extracting the base plate reference plane using the overall point cloud specifically involves: in each iteration, three points are randomly selected from the overall point cloud for planar model fitting, the vertical distance from other points to the planar model is calculated, and points with a distance less than a preset threshold are recorded as interior points. The iteration is repeated until the maximum number of iterations is reached or the number of interior points converges, and the planar model with the most interior points is taken as the base plate reference plane.

[0008] Preferably, aligning the normal of the base plate reference plane with the global coordinate system Z-axis specifically involves: re-estimating the plane parameters using the least squares method for the set of interior points belonging to the base plate reference plane to obtain a more accurate base plate fitting plane; determining the unit normal vector of the base plate fitting plane; calculating the three-dimensional rotation matrix that rotates the former to the latter based on the angle and cross product between the unit normal vector and the positive unit vector of the global coordinate system Z-axis; and applying this rotation transformation to the overall point cloud through the three-dimensional rotation matrix to make the normal of the base plate plane coincide with the positive direction of the global coordinate system Z-axis.

[0009] Preferably, the least squares method is used to re-estimate the plane parameters for the set of interior points belonging to the base plate reference plane to obtain a more accurate base plate fitting plane, specifically as follows: The initial fitting plane is obtained using the least squares method. For the first one belonging to the base plate reference plane Inner points Calculate the signed residuals to the initial fitting plane. ,in, Let x, y, and z be the components of the normal vector of the initial fitted plane in the x, y, and z directions. This is a constant term for the initial fitted plane. Let be the three-dimensional coordinates of any point belonging to the base plate reference plane in the world coordinate system; Let the set of interior points belonging to the reference plane of the base plate be a total of For the nth interior point, the i-th The residuals of each interior point are denoted as . The median absolute deviation of all interior point residuals Robust metric for estimating residuals ,in, For the mean value operator, This indicates taking the median of the residuals over all interior points. , This indicates taking the median of all absolute deviations. ; Define weight function Among them, standardized residuals Weight function , The cutoff parameter for the weighting function, and the standardized residual. ; Based on weight function Construct the objective function Through normalization constraints This yields the corrected unit normal vector and plane intercept; The above process of "coarse fitting – residual evaluation – weight update – weighted least squares" is iterated several times until the change in the plane parameters obtained in two adjacent iterations is less than the preset convergence threshold.

[0010] Preferably, the clustering of the rib point cloud under the dual constraints of normal continuity and upper limit of curvature to obtain multiple approximately coplanar rib plane clusters is specifically as follows: Using the rib point cloud as input, construct a spatial neighborhood index based on a kd-tree; For each point in the rib plate point cloud Given a number of neighborhood points, calculate its covariance matrix and obtain its eigenvalues. and the corresponding feature vectors; Minimum eigenvalue The corresponding unit eigenvector is used as a point normal vector ,by As a point Local curvature; Select a normally stable point from the ribbed point cloud as the seed point, and initiate region growing: for the current seed point... Find a point among its k nearest neighbors that simultaneously satisfies the following three conditions. And add them to the current region, and at the same time use these points as new candidate seeds to continue growing until no new points satisfy the conditions, to obtain an approximately coplanar ribbed plane cluster: (1) normal vector The included angle (2) (3) ; Repeat the above process for all unmarked points to eventually obtain multiple clusters of approximately coplanar stiffener planes.

[0011] Preferably, the step of merging rib planar clusters whose normals are paired and whose centroids are collinear to obtain multiple independent single rib point clouds specifically involves: For each stiffener plane cluster The least squares method is used to fit the plane equation. The unit normal vector is obtained. and intercept Simultaneously, the centroid of the point cloud cluster is calculated; Based on the included normal angle, pairwise merging of stiffener plane clusters is performed: if two clusters exist... Satisfying the normal vector The included angle And the nearest point distance between the two clusters is less than a preset distance threshold. Then, the two planar clusters are merged, and the planar parameters of the merged stiffener planar cluster are re-estimated. The normal angle threshold for the ribbed plane cluster is used to limit the approximate opposite normals of the two plane clusters; Project the centroid of each merged stiffener plane cluster onto the base plate reference plane to obtain two-dimensional coordinates; By performing linear fitting on these centroid points in the plane of the base plate, the principal direction vector of the length of the candidate stiffener and the corresponding support line are obtained; For any planar cluster, if its planar normal is approximately orthogonal to the base plate normal, the distance from the centroid to the support line is less than the collinear distance threshold, and the centroid projection does not fall within the bounding box boundary, then it is classified into the same single rib plate; otherwise, planar clusters whose centroids are close to the scene edge or do not satisfy the collinearity constraint are regarded as edge interference clusters and are eliminated.

[0012] Preferably, the step of constructing a local two-dimensional u-v coordinate system aligned with the stiffener body based on the point cloud of a single stiffener plate, projecting the point cloud of the single stiffener plate onto the local two-dimensional u-v coordinate system, and adaptively completing the weld seam division based on the coordinate distribution statistics of the point cloud in the u and v directions, specifically involves: Project the point cloud of a single stiffener plate onto the reference plane of the base plate to obtain the two-dimensional coordinate components in that plane. Calculate the two-dimensional covariance matrix of the point cloud of the single stiffener plate and perform principal component analysis to obtain the unit eigenvector corresponding to the largest eigenvalue. ; Taking the projection of the centroid of a single stiffening plate point cloud onto the plane of the base plate as the origin, and... The direction is the u-axis, and with Unit vectors orthogonal in the plane With the direction as the v-axis, construct a local two-dimensional uv coordinate system for the stiffener, where the u-axis represents the length direction of the stiffener and the v-axis represents the width direction of the stiffener. Project all points in the point cloud of this single stiffener plate onto a local two-dimensional UV coordinate system, and statistically analyze the geometric boundary of the stiffener plate point cloud along the length direction within this coordinate system. and geometric boundaries in the width direction ; Sort all points in the projected point cloud set in ascending order of their v-axis coordinates; Determine the average spacing in the width direction of the stiffener. , To determine the number of projected point clouds participating in the width-direction statistics, the width-direction interval threshold is further determined. ,in, These are constant coefficients set based on experience. This is the lower limit of the width-direction interval threshold, corresponding to the spatial resolution of the point cloud data; When the difference in the v-axis coordinates of adjacent points after sorting is greater than If the current accumulated points are not less than the preset minimum point threshold, the current accumulated point set is divided into a relatively concentrated V-band in the width direction, and the accumulation of the next V-band is restarted; if a V-band that meets the point requirement is not formed in the end, all points are treated as a single V-band. For each point within the v-band that meets the minimum number of points threshold, sort them in ascending order by the u-axis coordinate; Determine the average interval along the length direction ,in, The number of points within the V-band, with a length-direction spacing threshold. Determined adaptively by the following formula: ， in, These are coefficients tuned empirically. This is the lower limit of the length direction interval threshold, used to avoid generating too many short segments when the point cloud is sparse or has missing segments; When the coordinate difference between adjacent points along the u-axis is greater than When the current accumulated point set is divided into a weld candidate sub-segment that is basically continuous along the length direction, the accumulation of the next sub-segment begins; for each sub-segment, the number of points is required to be no less than the minimum number of points threshold in order to filter out pseudo-segments caused by isolated noise or local defects.

[0013] Preferably, the step of constructing a local two-dimensional u-v coordinate system aligned with the stiffener body based on the point cloud of a single stiffener, projecting the point cloud of the single stiffener onto the local two-dimensional u-v coordinate system, and then adaptively estimating the endpoints based on the coordinate distribution statistics of the point cloud in the u and v directions, specifically involves: For each candidate weld segment, determine the window size. ,in, The minimum number of points within the window. This is the window point ratio coefficient. This indicates rounding. The number of points contained in the candidate weld segment; For each candidate weld segment, sort the points within that segment by their u-axis coordinates from smallest to largest, and then select the top points from the sorted segments. These points constitute the candidate set of starting points. , sort the last These points constitute the candidate endpoint point set. ; Calculate separately Average values ​​of u and v coordinates of each point in the middle , Average values ​​of u and v coordinates of each point in the middle ; Will As the coordinates of the weld start point in the local two-dimensional UV coordinate system, The coordinates of the weld endpoint in the local two-dimensional uv coordinate system.

[0014] Preferably, the step of obtaining the starting coordinates and direction information of the weld in the base plate plane through the transformation between the local two-dimensional uv coordinate system and the world coordinate system specifically involves: combining the centroid position of the stiffener and the local basis vector to restore the local endpoint to the two-dimensional coordinates in the base plate reference plane; then combining the average height of the base plate plane to obtain the three-dimensional coordinates of the weld endpoint in the base plate reference plane coordinate system; and finally, through the transformation matrix between the base plate reference plane and the world coordinate system, mapping the start and end points of the weld back to the world coordinate system to obtain the three-dimensional coordinates of the start and end points of the weld in the world coordinate system, as well as the direction vector from the start point to the end point.

[0015] To achieve the above objectives, in a second aspect, this application provides a three-dimensional visual recognition system for weld seams of small-scale structural components, including a memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions; The one or more processors invoke the computer instructions to cause the system to perform the weld seam three-dimensional visual recognition method as described in the first aspect.

[0016] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application proposes a three-dimensional visual recognition method and system for weld seams of small-scale structural components. In terms of point cloud preprocessing and structural component segmentation, a method combining voxel grid downsampling based on scene scale adaptive voxel size and statistical filtering is adopted to effectively compress the number of points and remove outlier noise. On this basis, the pose transformation matrix between line laser camera groups is used to perform coarse stitching of multi-view point clouds, and fine stitching is completed by combining the point-to-surface ICP algorithm with weighted function, which improves the accuracy and stability of large-scale, multi-view point cloud registration. Furthermore, the reference plane of the base plate is extracted by RANSAC plane fitting and weighted least squares fine fitting based on residual statistics and Huber weights. The base plate normal is aligned with the global coordinate system Z-axis. Then, the ground point cloud, base plate point cloud and stiffener point cloud are segmented by combining the height mean and standard deviation of the base plate point cloud. Combined with region growing based on normal vector and curvature constraints and reconstruction of plane normal pairing relationship and centroid collinearity, the automatic separation of single stiffener point cloud clusters can be achieved. Robust identification of base plate and stiffener can be completed without relying on the three-dimensional model of structural components.

[0017] (2) This application proposes a three-dimensional visual recognition method and system for welds of small-scale structural components. In terms of automatic weld extraction, considering that the upper surface point cloud of stiffeners in small-scale structural components can only be obtained from a top-view perspective and the welds at the ends of the stiffeners cannot be directly observed, an adaptive V-band division, U-direction continuous segment division, and endpoint window average estimation combined with the geometric characteristics of the stiffeners are designed to extract welds. By adaptively determining the width and length direction interval thresholds according to the point cloud distribution statistics in the local uv coordinate system of the stiffeners, and using the mean coordinates of the point cloud windows at both ends to estimate the start and end points of the welds, and with the regularization of repeated welds based on the directional angle, lateral offset, and projection overlap, the spatial geometric information of the welds can be robustly located by relying only on the point cloud on the upper surface of the stiffeners, effectively suppressing the influence of noise, occlusion, and point cloud inhomogeneity.

[0018] (3) This application proposes a three-dimensional visual recognition method and system for welds in small-scale structural components. Regarding weld extraction, considering the limitation that only the top surface point clouds of the base plate and stiffeners can be obtained under line laser top-view scanning conditions, an automatic weld extraction method based on the principal direction analysis of stiffener point clouds and the construction of a local coordinate system is proposed. Principal component analysis (PCA) is performed on each stiffener point cloud cluster in the plane of the base plate to construct a local two-dimensional coordinate system with the stiffener length direction as the u-axis and the stiffener width direction as the v-axis. The coordinate range of the stiffener point cloud in the u and v directions is statistically analyzed under the uv coordinate system. The extreme values ​​of the u-direction coordinates are used to determine the end regions of the stiffener. The local coordinates of the weld start and end points are calculated by combining region growth and coordinate mean values, thereby obtaining the weld start, end, length, and direction. This method automatically locates the spatial geometric information of the weld by relying solely on the top surface point clouds of the stiffeners, without requiring complete observation of the stiffener side or manual teaching or pre-marking of the weld path. It is highly adaptable and has a simple calculation process.

[0019] (4) This application proposes a three-dimensional visual recognition method and system for weld seams of small modular structural components, which can be integrated into the automatic scanning assembly line and intelligent welding robot system of small modular structural components. Scanning, point cloud splicing, base plate and stiffener plate segmentation and automatic weld seam extraction can be completed in real time or near real time during the workpiece conveying or welding preparation stage. There is no need to separately establish a three-dimensional model of the structural component and compile an offline welding program before welding, which significantly reduces the workload of pre-welding preparation and on-site correction, and improves the automation and intelligence level of welding production of small modular structural components.

[0020] (5) This application proposes a three-dimensional visual recognition method and system for weld seams of small modular components. It utilizes a line laser scanning camera group composed of several line laser cameras arranged side by side along the width direction of the small modular components. The workpiece moves in one direction and the camera group is synchronously triggered to scan on the lead screw guide and slider guide of the simple small modular component automatic scanning assembly line. Compared with the single-camera static measurement method, it realizes continuous coverage scanning of large-size small modular components under the premise of simple structure and small modification. It can acquire multi-view, fully overlapping three-dimensional point cloud data in a large working space, providing a reliable data foundation for subsequent automatic weld seam extraction. Attached Figure Description

[0021] Figure 1 This is a flowchart of a three-dimensional visual recognition method for weld seams of a small-scale structural component provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of a simplified group assembly structure automatic scanning assembly line provided in the embodiments of this application.

[0023] Figure 3 This is a scanned schematic diagram of a small assembled structural component provided in an embodiment of this application.

[0024] Figure 4 This is a diagram showing the fitting result of the point cloud on the base plate plane, provided in an embodiment of this application.

[0025] Figure 5 This is a schematic diagram of the rib point cloud and its projection provided in the embodiments of this application.

[0026] Figure 6 This is a schematic diagram of the local UV two-dimensional coordinate system of the stiffener plate provided in the embodiment of this application.

[0027] Figure 7 This is a weld point cloud extraction result image provided in the embodiments of this application; The annotations in the attached figures are explained as follows: 1-Screw guide rail, 2-Slider guide rail, 3-Support base, 4-Aluminum profile bracket, 5-Line laser scanning camera group, 6-Rib plate point cloud, 7-Rib plate projection point cloud. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0029] In this application, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A existing alone, A and B existing simultaneously, and B existing alone. In this application, the symbol " / " indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0030] In this application, the terms "first" and "second," etc., are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0031] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0032] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0033] The embodiments of this application are described below with reference to the accompanying drawings.

[0034] like Figure 1 As shown, this application provides a three-dimensional visual recognition method for weld seams in small-scale structural components, including the following steps: Step 1: Acquisition of point cloud data for the structural components of the group.

[0035] The line laser scanning camera group consists of several line laser cameras arranged side by side along the width direction of the small assembled structure. Each line laser camera is connected to the displacement encoding device through a synchronous trigger module to realize synchronous scanning and acquisition as it moves with the guide rail, thereby obtaining segmented point cloud data of the small assembled structure.

[0036] Furthermore, in step one above, when scanning begins, the workpiece is placed on the support base and moves unidirectionally along the lead screw guide to complete the full scanning of the set area.

[0037] Furthermore, in step one above, the line laser camera group consists of... indivual( Linear laser cameras are arranged in parallel at fixed intervals on an aluminum profile support, wherein the overlap scan length between adjacent cameras is [value missing]. The total scan length is The camera sampling frequency is Camera triggering and stitching are controlled by an encoder via pulses, with an encoder pulse interval of [value missing]. The encoder trigger interval is Then the speed of the lead screw and guide rail is: .

[0038] Step 2: System calibration and establishment of a global coordinate system.

[0039] In step two above, the direction of the guide rail's movement is calibrated by fitting the straight line direction of the center of the standard ball as it moves on the guide rail.

[0040] In step two above, with Using the camera coordinate system as the world coordinate system, the poses between cameras are calibrated using a standard vision calibration board to obtain the pose transformation matrix between cameras.

[0041] Step 3: Perform splicing processing on the point clouds of multi-view structural components.

[0042] Furthermore, in step three above, the scanned point cloud is subjected to statistical filtering to remove outliers that show significant anomalies in neighborhood distance statistics, in order to remove isolated noise and retain valid point clouds with a consistent overall distribution. The statistically filtered point cloud is then subjected to voxel grid downsampling based on scene-scale adaptive voxel size, such as according to the bounding box side length of the point cloud. Set the voxel side length , The value range is 1 / 500 to 1 / 200, in order to compress the number of points and reduce the amount of computation in subsequent processing while preserving geometric details as much as possible.

[0043] Furthermore, in step three above, the point clouds obtained from different camera viewpoints are coarsely stitched together based on the pose transformation matrix between cameras obtained in step two, reducing the initial pose difference of the overlapping parts of the point clouds obtained from different viewpoints. Subsequently, the overlapping parts of the coarsely stitched point clouds are finely stitched together using a point-to-surface ICP algorithm combined with a weighting function.

[0044] Step 4: Multi-level point cloud segmentation based on adaptive base plate reference plane.

[0045] Without relying on the CAD model of the structural components, the system automatically constructs a reference plane for the base plate using the overall point cloud. Under a unified height coordinate system aligned with this reference plane, the system performs three-layer segmentation of the ground / support device, base plate, and stiffener plate. At the same time, it combines normal and curvature constraints to achieve automatic clustering and extraction of the point cloud of a single stiffener plate.

[0046] Step four includes the following sequential processing: ① Using a two-stage plane estimation method of "RANSAC coarse fitting + least squares fine correction", the base plate reference plane is robustly extracted from the overall point cloud to obtain the base plate candidate point cloud; ② Based on the unit normal of the base plate reference plane, a rigid body rotation is constructed to rotate the overall point cloud to a height-aligned coordinate system where the base plate normal coincides with the positive Z-axis of the global coordinate system; ③ In this coordinate system, multi-level height thresholds are adaptively determined based on the mean and standard deviation of the Z-coordinates of the base plate candidate point cloud, and the ground / support point cloud, base plate point cloud, and stiffener point cloud are layered and separated; ④ A kd-tree neighborhood index is constructed for the layered stiffener point cloud, and region growing clustering is performed under the dual constraints of normal continuity and curvature upper limit. The pairwise relationship of the plane cluster normal and the collinearity of the centroids are combined to reconstruct and number the individual stiffener point clouds.

[0047] Further, in step four above, ①, the overall point cloud obtained by stitching in step three is fitted with a planar model using the Random Sample Consensus (RANSAC) algorithm. Each time, three points are randomly selected from the overall point cloud to calculate the planar parameters. The vertical distance from other points to the plane is calculated. Points with a distance less than the preset inlier threshold ε1 are recorded as inliers. The iteration is repeated and the planar model with the largest number of inliers is selected as the base reference plane. Then, the plane parameters are re-estimated using the least squares method for the set of inliers belonging to the reference plane to obtain a more accurate base fitting plane, and the set of inliers is marked as the base candidate point cloud.

[0048] Further, in step four above, regarding ②, let the unit normal vector of the base plate fitting plane obtained in step ① be... According to the normal vector Calculate the angle and cross product between the point cloud and the positive unit vector of the global coordinate system Z-axis, calculate the three-dimensional rotation matrix that rotates the former to the latter, and apply this rotation transformation to the overall point cloud so that the normal of the base plane coincides with the (0,0,1) direction.

[0049] Furthermore, in step four above, step ③ involves calculating the average Z-coordinate of the candidate base points obtained in step ② in the rotated coordinate system. and standard deviation And set the first threshold coefficient. Second threshold coefficient Third threshold coefficient ,in, A positive number set for experience. The Z-coordinate is less than... The point cloud is identified as ground or support device point cloud and is removed. The Z coordinate is located at The point cloud within the interval is marked as the base point cloud, and the Z coordinate is greater than 1. The point cloud is labeled as rib point cloud.

[0050] Further, in step four above, step ④ involves constructing a kd-tree-based spatial neighborhood index for the subset of point clouds marked as stiffeners in step ③. Based on this, region growth clustering based on normal vector and curvature constraints is performed on the stiffener point clouds: first, a normal estimation algorithm is used to calculate the normal vector and curvature of each point under a given number of neighborhood points; then, region growth is performed using the normal angle being less than a preset smoothness threshold and the curvature being less than a preset curvature threshold as similarity criteria to obtain several approximately coplanar stiffener plane clusters.

[0051] Furthermore, based on the spatial position and normal relationship of each stiffener plane cluster, geometrically paired stiffener plane clusters with approximately opposite normals are merged, and stiffener plane clusters whose centroids are close to the scene boundary and belong to the edge interference region are removed. Finally, the retained inner stiffener plane clusters are merged according to the collinearity relationship in the stiffener length direction, and the point set that satisfies the collinearity constraint is divided into a stiffener point cloud cluster. Each stiffener point cloud cluster is assigned a unique number, thereby obtaining multiple independent single stiffener point clouds, providing basic data for subsequent weld extraction based on stiffener point clouds.

[0052] Step 5: Extract weld seams based on stiffener point cloud.

[0053] Step five includes the following processing steps performed in sequence: ① Based on Principal Component Analysis (PCA), perform principal direction analysis on the point cloud of each stiffener in the plane of the base plate, and construct a local two-dimensional coordinate system with the stiffener length direction as the u-axis and the stiffener width direction as the v-axis; ② Under the u-v coordinate system, project the stiffener point cloud as... The coordinate points are sorted according to their v coordinates, and the width direction interval threshold is adaptively determined based on the overall v range and the average interval. When the difference in the v coordinates of adjacent points is greater than If the current number of aggregate points is not less than the preset minimum point threshold, the rib point cloud is divided into several relatively concentrated V-bands in the width direction; within each V-band, the points are then sorted according to the u-coordinate, and the length direction interval threshold is adaptively determined based on the overall u-range and the average interval. When the difference in coordinates of adjacent points u is greater than Then, the V-band is divided into several weld candidate segments that are basically continuous along the length direction; ③ For each weld candidate segment that meets the point number threshold, end windows containing a preset number of points are selected from both ends of the segment as the starting point candidate point set. and the set of candidate endpoints Calculate its average coordinates in the u and v directions respectively. and ,Will and The coordinates of the start and end points of the weld in the local uv coordinate system are used as the coordinates. Further, the transformation relationship between the local coordinate system and the world coordinate system of the stiffener is used to map the local endpoints to the world coordinate system to obtain the start coordinates, end coordinates and direction vector of the weld. ④ Perform geometric connectivity analysis and clustering merging on all candidate weld segments: Based on the angle between the direction vectors of the segments, the lateral offset in the direction of the weld normal and the projection overlap in the length direction, the geometric connectivity of the weld segments is judged. Multiple candidate segments that are determined to belong to the same physical weld are aggregated into a representative weld segment to obtain the final weld set.

[0054] Further, in step five above, ① refers to each stiffener point cloud cluster obtained from the stiffener clustering process in step four. First, the point cloud cluster is projected onto the base plate reference plane to obtain its two-dimensional coordinate components within the base plate reference plane. The two-dimensional covariance matrix of the point cloud cluster is calculated and principal component analysis (PCA) is performed to obtain the unit eigenvector corresponding to the largest eigenvalue, which is used as the main length direction of the stiffener in the plane of the base plate. The local two-dimensional coordinate system uv of the stiffener is constructed with the projection of the centroid of the stiffener point cloud cluster in the plane of the base plate as the origin, the main length direction as the u-axis, and the direction orthogonal to the u-axis in the plane of the base plate as the v-axis.

[0055] Furthermore, in step five above, step ② involves projecting all point clouds in the rib point cloud cluster onto the local coordinate system uv, and calculating the minimum value of the coordinates of all points in the u direction. and maximum value and the minimum value of the coordinate in the v direction. and maximum value With the u coordinate range Characterizing the geometric boundary of the stiffener along its length, within the range of the v-coordinate. Characterizes the geometric boundary of the stiffener in the width direction.

[0056] Further, in step five above, step ③ involves adaptively dividing the weld candidate region using the coordinate distribution of the stiffener point cloud in the uv coordinate system, and calculating the local coordinates of the weld start and end points within the candidate region. Specifically, all points in the stiffener point cloud cluster are sorted from smallest to largest according to their v coordinates, and their overall v range and average interval are statistically analyzed. A width-direction interval threshold is then determined based on the average interval and a preset lower limit. When the difference in v-coordinates between adjacent points is greater than At that time, the rib plate point cloud cluster was divided into several V-bands that were relatively concentrated in the width direction.

[0057] For each candidate weld segment that meets the point count threshold, select several points from the partial point clouds located at both ends of the segment to form a candidate starting point set. and the set of candidate endpoints Calculate the candidate point set of the starting point respectively. Average value of the coordinates of each point in the u direction and the average value of the coordinates in the v direction and the set of candidate endpoints Average value of the coordinates of each point in the u direction and the average value of the coordinates in the v direction ,by As the coordinates of the weld start point in the local coordinate system, The coordinates of the weld endpoint in the local coordinate system are used as the Euclidean distance between the two points as the weld length, and the vector from the weld start point to the weld endpoint is used as the direction vector of the weld in the local coordinate system. Then, the starting coordinates and direction information of the weld in the base plate plane are obtained through the transformation relationship between the local coordinate system and the world coordinate system.

[0058] Example Corresponding to step one above, the system described in this embodiment is built on a simple, modular, automated scanning and assembly line for structural components. For example... Figure 2As shown, the assembly line includes a conveyor line for carrying and transporting sub-assembly structural components, a lead screw guide and a slider guide arranged along the conveyor line, and an aluminum profile bracket mounted above the conveyor line. Two line laser cameras are arranged side-by-side along the width of the sub-assembly structural components via mounting brackets to form a line laser scanning camera group. The camera group is electrically connected to a displacement encoder mounted on the lead screw guide via a synchronization trigger module, and is also connected to an industrial computer to achieve synchronous acquisition and data processing by multiple cameras.

[0059] In this embodiment, the lead screw guide is arranged at the center of the conveyor line, and the slider guide is symmetrically arranged on both sides of the lead screw guide. The slider and the workpiece are connected through the base device on the guide, which is used to drive the small assembly structure to move unidirectionally at a constant speed along the length of the assembly line. The line laser camera group is fixed on the aluminum profile bracket.

[0060] like Figure 3 As shown, the small, upright structural components are fixed to the support base and move unidirectionally at a constant speed along the guide rail, receiving scanning from the line laser camera group within the scanning area. The displacement encoder operates at a set pulse interval. A trigger signal is output to the camera group, and the line laser camera group operates at a sampling frequency. The structured light stripe profile of the synchronous acquisition line is used to obtain frame-by-frame cross-sectional point clouds through triangulation. Let the encoder pulse interval be... The speed of the lead screw guide rail Satisfying the aforementioned relation This ensures that the point cloud sampling spacing along the guide rail meets the system resolution requirements.

[0061] In practical applications, the selection is based on the maximum width of the assembled structural components under test and the effective field of view of a single line laser camera. tower( Linear laser cameras are installed at fixed baseline intervals, so that the scanning areas of adjacent cameras overlap by a certain length. This allows the entire width of the small, assembled structure to be covered by a parallel field of view from multiple cameras. Time synchronization between the cameras is achieved by a synchronization trigger module, and spatial pose relationships are obtained through a calibration process.

[0062] Corresponding to the guide rail motion direction calibration in step two above, this embodiment uses a standard sphere as the calibration target. The standard sphere is mounted on a mounting base that moves with the slider, causing it to reciprocate along the guide rail and perform multiple scans within the field of view of each line laser camera. The point cloud of the standard sphere is sliced ​​along the guide rail direction. In each slice, circle fitting is used to obtain the corresponding center coordinates. Then, least-squares linear fitting is performed on all sphere center coordinates to obtain the guide rail motion direction vector. and in the world coordinate system with camera 1 as the reference. This direction is defined as the Y-axis direction, providing a basis for the subsequent establishment of a global coordinate system.

[0063] Corresponding to the system calibration and global coordinate system establishment in step two above, assuming there are a total of Tabletop laser cameras, numbered Linear laser camera coordinate system As a world coordinate system A standard vision calibration plate is placed on the guide rail, and the first image is obtained through multi-view calibration. Individual laser camera coordinate system Relative to the world coordinate system The extrinsic parameters (rotation matrix and translation vector) are used to obtain the camera's extrinsic parameters. Pose transformation matrix from coordinate system to world coordinate system:

[0064] in, In subsequent processing, the transformation matrix is ​​used to uniformly transform the point clouds acquired by each camera to the world coordinate system, thereby achieving geometric alignment of point clouds from multiple perspectives.

[0065] Corresponding to step three above, as the workpiece moves through the scanning area on the assembly line, the line laser camera group continuously acquires the line laser cross-sectional profile of the structural component surface and reconstructs the segmented point cloud of the small assembled structural component in an industrial computer. For each segment of the point cloud, preprocessing and coarse stitching are first performed. Specifically, statistical filtering is applied to each segment of the original point cloud, and the average distance and standard deviation of each point to other points in its neighborhood are calculated. Points whose neighborhood distances deviate significantly from the statistical characteristics are identified as outliers and removed to remove isolated noise and retain effective point clouds with a consistent overall distribution. Then, for the statistically filtered point cloud, a voxel grid downsampling method based on scene scale adaptive voxel size is adopted. The voxel side length is automatically adjusted according to the spatial scale of the current scanning scene to compress the number of points while preserving the geometric details of the weld and stiffener as much as possible.

[0066] After single-frame preprocessing, using the pose transformation matrices between cameras obtained from the aforementioned calibration steps, point cloud fragments acquired by different line laser cameras at the same time are first transformed to a unified world coordinate system, completing coarse stitching based on the initial pose and reducing the initial pose differences between overlapping areas of multi-view point clouds. Subsequently, based on the coarse stitching results, overlapping areas of adjacent viewpoint point clouds are selected to construct point-to-plane correspondences. A weighted iterative nearest point (ICP) algorithm based on point-to-plane distance is then used to jointly optimize the rigid body poses between point clouds from different viewpoints. Let the... In each corresponding point pair, the source point is... A point on the plane containing the target point is The unit normal vector of the plane is Then the weighted point-to-surface loss function is defined as:

[0067] in, and Let be the rotation matrix and translation vector to be determined. For the first The point-to-surface distance residuals for each corresponding point pair It is the angle between the local normal of the source point and the normal of the target plane.

[0068] In this example, the weight function is defined. for:

[0069] in, For distance scale parameters, This is the normal angle scale parameter, used to control the suppression strength for points corresponding to large residuals and inconsistencies in normal direction. In this example, The preferred setting is 1.0mm to 1.5mm. The preferred setting is the radian value corresponding to 5° to 10°.

[0070] Corresponding to step four above, proceed to the step of dividing the base plate and stiffening plate based on the adaptive base plate reference plane.

[0071] This application first fits the base plate through consistency, and then segments the stiffener point cloud based on the coordinates of the base plate. Since the base plate reference plane is robustly fitted first, and the layering threshold is adaptively determined based on the height statistics of the base plate point cloud, the criteria can be automatically adjusted under different plate thicknesses and assembly states, which significantly improves the accuracy and stability of the base plate and stiffener segmentation in multi-specification group assembly structural component scenarios.

[0072] First, the Random Sample Consensus (RANSAC) algorithm is applied to fit a plane model to the overall point cloud: in each iteration, three points are randomly selected to calculate the plane parameters, and the vertical distance from all points to the plane is estimated. Points with a distance less than a preset inlier threshold are selected. Points with the highest number of interior points are considered interior points. The iterations are repeated until the maximum number of iterations is reached or the number of interior points converges. The fit with the highest number of interior points is taken as the base reference plane (the base plane fitting result is shown in the figure). Figure 4 (As shown).

[0073] Obtain the set of interior points for this iteration. Subsequently, this embodiment further constructs a weighted least squares refinement process based on residual statistics, instead of directly using the initial plane parameters. Let the initial fitted plane be:

[0074] For each interior point Calculate the signed residuals to the initial plane:

[0075] Estimating the noise scale using the median absolute deviation (MAD) of all interior point residuals:

[0076]

[0077] in, This is a robust scaling estimate for the residuals.

[0078] make In this embodiment, a Huber-type weight function with an upper bound is selected as the weight function. Specifically defined as:

[0079]

[0080]

[0081] And through normalization constraints This yields the corrected unit normal vector and plane intercept. The above process of "coarse fitting – residual evaluation – weight update – weighted least squares" can be iterated several times as needed until the change in plane parameters obtained from two adjacent iterations is less than the preset convergence threshold.

[0082] Let the unit normal vector of the fitted plane of the base plate be... To facilitate subsequent height-based layering, this embodiment uses the normal vector. The angle and cross product between the vector and the positive unit vector of the global coordinate system Z-axis are used to calculate the relationship between the vector and the cross product. 3D rotation matrix rotated to the Z-axis The rotation transformation is applied to the entire point cloud so that the base plate fitting plane approximately coincides with the xoy plane. The rotated point cloud remains in a unified world coordinate system, only the coordinate axis directions have been rotated, thus normalizing the complex spatial pose to a unified height coordinate system without changing the relative geometric relationships of the point cloud.

[0083] In the rotated coordinate system, the average Z-coordinate of the candidate base plate points obtained in step 4 (①) is calculated. and standard deviation And set the first threshold coefficient. Second threshold coefficient Third threshold coefficient ,in, A positive number set for experience. The Z-coordinate is less than... The point cloud is identified as ground or support device point cloud and is removed. The Z coordinate is located at The point cloud within the interval is marked as the base point cloud, and the Z coordinate is greater than 1. The point cloud is labeled as the stiffener point cloud. Based on the above height-based layering criteria, the overall point cloud can be divided into three categories: ground / support point cloud, base plate point cloud, and stiffener point cloud without the need to import structural component CAD models.

[0084] For the point cloud subset labeled as stiffeners, this embodiment employs a region growing method based on normal vectors and curvature constraints to extract the point cloud of a single stiffener. Specifically, a kd-tree-based spatial neighborhood index is constructed using the stiffener point cloud as input, and for each point... Given a number of neighborhood points Calculate its covariance matrix and obtain the eigenvalues. and the corresponding eigenvectors. The smallest eigenvalue... The corresponding unit eigenvector is denoted as the normal vector of that point. and with As a point The local curvature index. Then, a normal smoothness threshold is set. Curvature threshold and neighborhood distance threshold Prioritize normal stability ( Select a point from the smaller (or less) points as the seed point and initiate region growth: for the current seed point Find the satisfying among its k nearest neighbors point The points are then added to the current region and used as new candidate seeds to continue growing until no new points satisfy the conditions, resulting in a cluster of stiffening planes that are approximately coplanar and have low curvature. This process is repeated for all unlabeled points to obtain several initial stiffening plane point cloud clusters. Compared to simple clustering methods that rely solely on Euclidean distance, this "normal-curvature constrained region growth" method maintains the overall connectivity and geometric consistency of the stiffening planes even in the presence of interference such as weld reinforcement, plate warping, and local notches, significantly suppressing the misclassification of weld reinforcement or local protrusions into independent clusters.

[0085] Furthermore, after obtaining the initial planar point cloud clusters of the stiffeners, in order to restore the complete structure of a single stiffener, this embodiment further performs geometric reconstruction of the planar clusters based on the pairwise relationship of planar normals and the collinearity constraint of the centroids. Specifically, for each planar cluster... The plane equation is fitted using the least squares method: The unit normal vector is obtained. and intercept Simultaneously calculate the centroid of the point cloud cluster. First, the planar clusters are paired and merged according to the included normal angle: if two clusters exist... , satisfy And the distance between the nearest points of the two clusters is less than a preset distance threshold. If the two are considered to be opposite side plates of the same stiffener, their point cloud sets are merged, and the planar parameters are re-estimated to obtain the merged result of the two side plates of the stiffener. Then, the centroid of each merged planar cluster is... Projecting onto the base reference plane yields two-dimensional coordinates. Perform RANSAC-based linear fitting on these centroids in the plane of the base plate to obtain the principal direction vector of the length of the candidate stiffener. and the corresponding supporting straight line. For any planar cluster, if its planar normal... Approximately orthogonal to the normal of the base plate, and the distance from the centroid to this line is less than the collinearity distance threshold. Furthermore, the centroid projection does not fall on the bounding box boundary of the scan. If the plane clusters are within the same rib, they are classified as edge interference clusters; otherwise, plane clusters whose centroids are close to the scene edge or do not satisfy the collinearity constraint are regarded as edge interference clusters and are removed.

[0086] Corresponding to step five above, after completing the segmentation of the base plate and stiffening plates, the next step is to extract the weld seam based on the stiffening plate point cloud. For example... Figure 5 As shown in the figure, 6 represents the original point cloud of the stiffener, and 7 represents the projection of this stiffener point cloud onto the base plate reference plane. First, for a given stiffener point cloud cluster, it is projected onto the base plate reference plane to obtain the two-dimensional coordinate components within that plane. Calculate the two-dimensional covariance matrix of the point cloud cluster and perform principal component analysis (PCA) to obtain the unit eigenvector corresponding to the largest eigenvalue. ,Will The direction in the plane is taken as the principal direction of the length of the stiffener in the plane of the base plate; the projection of the centroid of the stiffener point cloud cluster in the plane of the base plate is taken as the origin, and... The direction is the u-axis, and with Unit vectors orthogonal in the plane Construct a local two-dimensional coordinate system uv for the stiffener with the direction of the v-axis, such as... Figure 6 As shown, in this coordinate system, the u-axis represents the length direction of the stiffener, and the v-axis represents the width direction.

[0087] Then, all points in the stiffener point cloud cluster are projected onto the local coordinate system uv, that is, the coordinates of each point in the u-axis and v-axis directions are calculated. Within this coordinate system, the coordinate range of the rib point cloud in the u and v directions is statistically analyzed, and the minimum value of the u coordinate is obtained. and maximum value and the minimum value of the v coordinate. and maximum value , in interval Characterizing the geometric boundary of the stiffener along its length, in intervals. Characterizes the geometric boundary of the stiffener in the width direction. Let v be the range. Sort all points in ascending order of their v-coordinates. If the number of points is... Then the average interval in the width direction can be defined:

[0088] To achieve adaptive width segmentation under different stiffener widths and point cloud densities, this embodiment preferably adopts... The form determines the width direction interval threshold ,in This coefficient is set based on experience, and a typical value is 10. This is the lower limit of the width-direction spacing threshold, corresponding to the spatial resolution of the point cloud data, typically set to 1.0 mm. When the difference in v-coordinates between adjacent points is greater than... And the current accumulated points are not less than the preset minimum points threshold. When the current accumulated point set is divided into a relatively concentrated V-band in the width direction, the accumulation of the next V-band will start again; if a V-band that meets the point requirement is not formed in the end, all points will be treated as a single V-band.

[0089] For each point within the v-band that meets the minimum number of points threshold, sort them by their u-coordinates from smallest to largest, and calculate the overall range of the v-band in the u-direction. And define the average interval in the length direction. ,in, This represents the number of points within the V-band. Similarly, the length-direction spacing threshold... The preferred method is to adaptively determine the following formula:

[0090] in, The coefficients are empirically tuned, with a typical value of 10. This is the lower limit of the length-direction spacing threshold, used to avoid generating too many short segments when the point cloud is sparse or contains missing data. When the coordinate difference between adjacent points along the u-direction is greater than... When the current accumulated point set is divided into a candidate weld segment that is basically continuous along the length direction, the accumulation of the next segment begins; for each segment, the number of points is required to be no less than the minimum point threshold. This is to filter out pseudo-segments caused by isolated noise or local defects.

[0091] To robustly estimate the start and end points of the corresponding weld seam while only acquiring the point cloud on the upper surface of the stiffener, this embodiment introduces an end-window averaging estimation method in each weld seam candidate segment that meets the conditions. Suppose a segment contains... If there are several points, then the window size should be selected as follows:

[0092] in, The minimum number of points within the window, preferably 5. The window point ratio coefficient is preferably set to 0.1. Indicates rounding. The first point in the set sorted in ascending order of its u-coordinate. These points constitute the candidate set of starting points. ,back These points constitute the candidate endpoint point set. Calculate separately Average values ​​of u and v coordinates of each point in the middle and Average values ​​of u and v coordinates of each point in the middle ,Will As the coordinates of the weld start point in the local coordinate system, The coordinates of the weld endpoint in the local coordinate system. This is combined with the position of the stiffener centroid and the local basis vectors. The local endpoints can be restored to two-dimensional coordinates within the base plate reference plane:

[0093]

[0094] in, This represents the projection of the centroid of the ribbed cloud cluster onto the plane of the base plate. This is then combined with the average height of the base plate plane. The three-dimensional coordinates of the weld endpoint in the base plate reference plane coordinate system can be obtained. By using the transformation matrix between the base plate reference plane and the world coordinate system, the above endpoints are mapped back to the world coordinate system, thereby obtaining the three-dimensional coordinates of the start and end points of the weld in the world coordinate system, as well as the direction vector from the start point to the end point.

[0095] After obtaining the start and end points of all candidate weld segments, this embodiment further performs clustering and merging processing based on geometric connectivity on the weld segment set. Specifically, each weld segment is represented by its endpoint coordinates and unit direction vector, and for any two segments, the following is calculated: 1) the angle between the direction vectors. ;2) The projected distance d⊥ of the line connecting the midpoints of the two line segments in the direction perpendicular to the weld;3) The overlap length of the projected lengths of the two line segments in the direction of the weld. and its overlap ratio relative to the shorter line segment length .like Figure 7 The image shows the extracted weld seam segment.

[0096] Cluster the entire set of line segments according to the above geometric connectivity criteria, when both conditions are met... Less than the preset angle threshold (5° in this example), d⊥ is less than the preset lateral offset threshold. (3mm in this example), and the overlap ratio Not less than the preset lower limit When the value is 0.8 in this example, the two weld segments are determined to be the same physical weld segment that is geometrically connected.

[0097] In practical implementation, based on the obtained start and end points of the weld, weld segments are sampled at equal intervals along the weld direction according to a preset spatial step size to generate a weld path point sequence for welding robot trajectory planning. Furthermore, all weld segments obtained from different stiffeners can be repeatedly detected and merged, treating geometrically approximately overlapping weld segments as the same weld to avoid multiple counting. By repeating the above process for all stiffener point cloud clusters, the spatial geometric information and path information of all welds in the entire sub-assembly structure can be automatically extracted.

[0098] In practical engineering applications, the threshold parameters in this embodiment can be adjusted based on the measurement accuracy of the line laser camera, workpiece dimensions, and assembly tolerances. Parameters directly related to the key steps of this application mainly include: a robust scalar threshold c used to control the attenuation of residual weights during adaptive base plate plane estimation; and a width-direction interval threshold in the local UV coordinate system of the stiffener. and length direction interval threshold The adaptive value coefficients; and the discrimination thresholds for directional angle, lateral offset, and projection overlap in weld geometric connectivity clustering. These parameters can be determined through experimental calibration or adaptive rules within a given sensing accuracy range, thereby ensuring the robustness and stability of the weld geometric information extraction process under conditions of noise, occlusion, and assembly errors.

[0099] Meanwhile, without changing the above steps and processing ideas, the number of line laser cameras, baseline spacing, sampling frequency, and guide rail movement speed can be adjusted to adapt to the size and production line cycle requirements of different specifications of small modular components; alternatively, the line laser cameras can be replaced with structured light or other three-dimensional sensors, as long as they can acquire three-dimensional point cloud data that meets the processing requirements of this application.

[0100] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0101] Based on the methods in the above embodiments, this application provides an electronic device that may include a processor, a communications interface, a memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor may invoke logical instructions stored in the memory to execute the methods in the above embodiments.

[0102] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0103] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0104] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0105] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0106] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0107] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0108] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0109] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A three-dimensional visual recognition method for weld seams of small-scale structural components, characterized in that, include: Using the reference camera coordinate system as the world coordinate system, the point cloud acquired by each camera in the line laser scanning camera group at the same time is uniformly transformed to the world coordinate system through the pose transformation matrix between cameras, so as to achieve geometric alignment of the point cloud from multiple perspectives. The line laser scanning camera group consists of multiple line laser cameras arranged side by side along the width direction of the small assembled structure, which are used to synchronously scan and acquire small assembled structures moving along the guide rail. The point clouds of geometrically aligned multi-view structural components are stitched together to obtain a complete point cloud; Extract the base plate reference plane using the overall point cloud, and obtain the base plate point cloud belonging to the base plate reference plane. Align the normal of the base plate reference plane with the Z-axis of the global coordinate system. Based on the mean and standard deviation of the height under the global coordinate system Z-axis of the base plate point cloud, the height layering threshold is adaptively determined to divide the complete point cloud into ground point cloud, base plate point cloud and rib plate point cloud; Under the dual constraints of normal continuity and curvature upper limit, the stiffener point cloud is clustered to obtain multiple approximately coplanar stiffener plane clusters. Merging rib planar clusters with paired normals and collinear centroids yields multiple independent single rib point clouds. For each individual stiffener point cloud, a local two-dimensional uv coordinate system aligned with the stiffener body is constructed based on the individual stiffener point cloud. After projecting the individual stiffener point cloud onto the local two-dimensional uv coordinate system, the weld seam division and endpoint estimation are adaptively completed based on the coordinate distribution statistics of the point cloud in the u and v directions. The distance between the start and end points of the weld is taken as the weld length, and the vector from the start point to the end point of the weld is taken as the direction vector of the weld in the local two-dimensional uv coordinate system. By transforming the local two-dimensional UV coordinate system with the world coordinate system, the starting coordinates and direction information of the weld in the base plate plane are obtained.

2. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, The extraction of the base plate reference plane using the overall point cloud is specifically as follows: in each iteration, three points are randomly selected from the overall point cloud for plane model fitting, the vertical distance from other points to the plane model is calculated, and points with a vertical distance less than a preset threshold are recorded as interior points. The iteration is repeated until the maximum number of iterations is reached or the number of interior points converges. The plane model with the most interior points is taken as the base plate reference plane.

3. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, The step of aligning the normal of the base plate reference plane with the global coordinate system Z-axis specifically involves: re-estimating the plane parameters using the least squares method for the set of interior points belonging to the base plate reference plane to obtain a more accurate base plate fitting plane. Determine the unit normal vector of the base plate fitting plane. Based on the angle and cross product between the unit normal vector and the positive unit vector of the global coordinate system Z-axis, calculate the three-dimensional rotation matrix that rotates the former to the latter. Apply this rotation transformation to the overall point cloud through the three-dimensional rotation matrix so that the normal vector of the base plate plane coincides with the positive Z-axis of the global coordinate system.

4. The three-dimensional visual recognition method for welds as described in claim 3, characterized in that, For the set of interior points belonging to the base plate reference plane, the plane parameters are re-estimated using the least squares method to obtain a more accurate base plate fitting plane, specifically: The initial fitting plane is obtained using the least squares method. For the first one belonging to the base plate reference plane Inner points Calculate the signed residuals to the initial fitting plane. ,in, Let x, y, and z be the components of the normal vector of the initial fitted plane in the x, y, and z directions. This is a constant term for the initial fitted plane. Let be the three-dimensional coordinates of any point belonging to the base plate reference plane in the world coordinate system; The median absolute deviation of all interior point residuals Robust metric for estimating residuals ,in, For the mean value operator, This indicates taking the median of the residuals over all interior points. This indicates taking the median of the absolute deviations over all interior points. For the first The residuals of each interior point , For the first The residuals of each interior point , This represents the number of interior points belonging to the set of interior points on the base plate reference plane. Define weight function Among them, standardized residuals Weight function , The cutoff parameter for the weighting function, and the standardized residual. ; Based on weight function Construct the objective function Through normalization constraints This yields the corrected unit normal vector and plane intercept; The above process of "coarse fitting – residual evaluation – weight update – weighted least squares" is iterated several times until the change in the plane parameters obtained in two adjacent iterations is less than the preset convergence threshold.

5. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, Under the dual constraints of normal continuity and upper limit of curvature, the stiffener point cloud is clustered to obtain multiple approximately coplanar stiffener plane clusters, specifically: Using the rib point cloud as input, construct a spatial neighborhood index based on a kd-tree; For each point in the rib plate point cloud Given a number of neighborhood points, calculate its covariance matrix and obtain its eigenvalues. and the corresponding feature vectors; Minimum eigenvalue The corresponding unit eigenvector is used as a point normal vector ,by As a point Local curvature; Select a normally stable point from the ribbed point cloud as the seed point, and initiate region growing: for the current seed point... Find a point among its k nearest neighbors that simultaneously satisfies the following three conditions. And add them to the current region, and at the same time use these points as new candidate seeds to continue growing until no new points satisfy the conditions, to obtain an approximately coplanar ribbed plane cluster: (1) normal vector The included angle (2) (3) ; Repeat the above process for all unmarked points to eventually obtain multiple clusters of approximately coplanar stiffener planes.

6. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, The process of merging rib planar clusters whose normals are paired and whose centroids are collinear to obtain multiple independent single rib point clouds is as follows: For each stiffener plane cluster The least squares method is used to fit the plane equation. The unit normal vector is obtained. and intercept Simultaneously, the centroid of the point cloud cluster is calculated; Based on the included normal angle, pairwise merging of stiffener plane clusters is performed: if two clusters exist... Satisfying the normal vector The included angle And the nearest point distance between the two clusters is less than a preset distance threshold. Then, the two planar clusters are merged, and the planar parameters of the merged stiffener planar cluster are re-estimated. The normal angle threshold for the ribbed plane cluster is used to limit the approximate opposite normals of the two plane clusters; Project the centroid of each merged stiffener plane cluster onto the base plate reference plane to obtain two-dimensional coordinates; By performing linear fitting on these centroid points in the plane of the base plate, the principal direction vector of the length of the candidate stiffener and the corresponding support line are obtained; For any planar cluster, if its planar normal is approximately orthogonal to the base plate normal, the distance from the centroid to the support line is less than the collinear distance threshold, and the centroid projection does not fall within the bounding box boundary, then it is classified into the same single rib plate; otherwise, planar clusters whose centroids are close to the scene edge or do not satisfy the collinearity constraint are regarded as edge interference clusters and are eliminated.

7. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, The process involves constructing a local two-dimensional u / v coordinate system aligned with the stiffener body based on the point cloud of a single stiffener plate, projecting the point cloud of the single stiffener plate onto the local two-dimensional u / v coordinate system, and then adaptively dividing the weld seam based on the coordinate distribution statistics of the point cloud in the u and v directions. Specifically: Project the point cloud of a single stiffener plate onto the reference plane of the base plate to obtain the two-dimensional coordinate components in that plane. Calculate the two-dimensional covariance matrix of the point cloud of the single stiffener plate and perform principal component analysis to obtain the unit eigenvector corresponding to the largest eigenvalue. ; Taking the projection of the centroid of a single stiffening plate point cloud onto the plane of the base plate as the origin, and... The direction is the u-axis, and with Unit vectors orthogonal in the plane With the direction as the v-axis, construct a local two-dimensional uv coordinate system for the stiffener, where the u-axis represents the length direction of the stiffener and the v-axis represents the width direction of the stiffener. Project all points in the point cloud of this single stiffener plate onto a local two-dimensional UV coordinate system, and statistically analyze the geometric boundary of the stiffener plate point cloud along the length direction within this coordinate system. and geometric boundaries in the width direction ; Sort all points in the projected point cloud set in ascending order of their v-axis coordinates; Determine the average spacing in the width direction of the stiffener. , To determine the number of projected point clouds participating in the width-direction statistics, the width-direction interval threshold is further determined. ,in, These are constant coefficients set based on experience. This is the lower limit of the width-direction interval threshold, corresponding to the spatial resolution of the point cloud data; When the difference in the v-axis coordinates of adjacent points after sorting is greater than If the current accumulated points are not less than the preset minimum point threshold, the current accumulated point set is divided into a relatively concentrated V-band in the width direction, and the accumulation of the next V-band is restarted; if a V-band that meets the point requirement is not formed in the end, all points are treated as a single V-band. For each point within the v-band that meets the minimum number of points threshold, sort them in ascending order by the u-axis coordinate; Determine the average interval along the length direction ,in, The number of points within the V-band, with a length-direction spacing threshold. Determined adaptively by the following formula: ， in, These are coefficients tuned empirically. This is the lower limit of the length direction interval threshold, used to avoid generating too many short segments when the point cloud is sparse or has missing segments; When the coordinate difference between adjacent points along the u-axis is greater than When the current accumulated point set is divided into a weld candidate sub-segment that is basically continuous along the length direction, the accumulation of the next sub-segment begins; for each sub-segment, the number of points is required to be no less than the minimum number of points threshold in order to filter out pseudo-segments caused by isolated noise or local defects.

8. The three-dimensional visual recognition method for welds as described in claim 7, characterized in that, The process involves constructing a local two-dimensional u / v coordinate system aligned with the stiffener body based on the point cloud of a single stiffener plate. After projecting the point cloud of the single stiffener plate onto the local two-dimensional u / v coordinate system, endpoint estimation is adaptively performed based on the coordinate distribution statistics of the point cloud in the u and v directions. Specifically: For each candidate weld segment, determine the window size. ,in, The minimum number of points within the window. This is the window point ratio coefficient. This indicates rounding. The number of points contained in the candidate weld segment; For each candidate weld segment, sort the points within that segment by their u-axis coordinates from smallest to largest, and then select the top points from the sorted segments. These points constitute the candidate set of starting points. , sort the last These points constitute the candidate endpoint point set. ; Calculate separately Average values ​​of u and v coordinates of each point in the middle , Average values ​​of u and v coordinates of each point in the middle ; Will As the coordinates of the weld start point in the local two-dimensional UV coordinate system, The coordinates of the weld endpoint in the local two-dimensional uv coordinate system.

9. The three-dimensional visual recognition method for welds as described in claim 1, characterized in that, The process of obtaining the starting coordinates and orientation information of the weld in the base plate plane by transforming between the local two-dimensional UV coordinate system and the world coordinate system is as follows: combining the centroid position of the stiffener plate and the local basis vector, the local endpoint is restored to the two-dimensional coordinates in the base plate reference plane; then, combining the average height of the base plate plane, the three-dimensional coordinates of the weld endpoint in the base plate reference plane coordinate system are obtained. By using the transformation matrix between the base plate reference plane and the world coordinate system, the start and end points of the weld are mapped back to the world coordinate system, thus obtaining the three-dimensional coordinates of the start and end points of the weld in the world coordinate system, as well as the direction vector from the start point to the end point.

10. A three-dimensional visual recognition system for weld seams of small-scale structural components, characterized in that, Includes memory and one or more processors; The memory is coupled to the one or more processors, and the memory is used to store computer program code, the computer program code including computer instructions; The one or more processors invoke the computer instructions to cause the system to perform the weld seam three-dimensional visual recognition method as described in any one of claims 1 to 9.

Images