Welding seam recognition and trajectory optimization method and system based on 3D vision

Abstract

The invention provides a welding seam identification and trajectory optimization method and system based on three-dimensional vision, and the method specifically comprises the following steps: firstly, collecting the three-dimensional point cloud data of a welding seam, extracting the local geometric features of the welding seam, and carrying out the feature enhancement of the local geometric features, so as to screen out representative effective edge points; and then, collecting multi-source point cloud data of the welding seam area, constructing a multi-scale welding seam feature recognition network, inputting effective edge points obtained by screening into the network, realizing accurate recognition of the welding seam area, and further extracting a linear welding seam area. According to the recognition result, a fitting algorithm is adopted for conducting curve fitting on the weld joint area, and a preliminary welding track is generated; and for the condition that multiple sections of welding seams are intersected, an optimal fitting point searching mechanism is further introduced, the preliminary track is optimized, and the continuity and integrity of a welding path are ensured. And finally, based on the center line of the welding seam, the posture of the welding gun is precisely planned in combination with the dihedral structure model, and therefore high-precision track control and posture adjustment are achieved.

Description

Technical Field

[0001] This invention belongs to the field of weld seam recognition technology, and specifically relates to a method and system for weld seam recognition and trajectory optimization based on 3D vision. Background Technology

[0002] With the rapid development of my country's economy and technological strength, the demand for large transportation vehicles in heavy industries such as railway transportation and shipbuilding is increasing. Multi-structure medium-thick plate components, due to their excellent mechanical properties and structural strength, are widely used in key parts of these vehicles. The welds on these components are typically multi-planar composite welds, composed of various types of welds such as butt welds, fillet welds, and lap welds. These welds are distributed on multiple intersecting planes, and the precision and consistency of the welding process directly affect the overall strength and stability of the structure. Therefore, automated welding technology for complex structural components has become a research hotspot in the industrial manufacturing field.

[0003] Currently, traditional 2D vision technology is widely used in welding robot systems, mainly relying on standard image processing algorithms (such as edge detection, threshold segmentation, and morphological manipulation) to identify and track weld seams. However, 2D vision sensors are limited by their imaging principles and cannot provide depth information. They are also limited in complex environments and dynamically changing industrial scenarios, and struggle to effectively handle image quality fluctuations caused by workpiece geometric errors and changes in ambient lighting, resulting in insufficient recognition accuracy and stability.

[0004] In recent years, although deep learning methods that integrate RGB-D vision sensors have emerged (such as the weld seam recognition and preliminary trajectory generation method based on Mask R-CNN developed by Tran et al.; and the weld seam recognition and classification method based on the improved YOLOv5 algorithm proposed by Chen et al.), the inherent limitations of 2D vision methods still exist: sensor accuracy is unstable, and images are easily affected by the surface reflection characteristics of objects, complex ambient lighting conditions, and single-view shooting angles, which limits the practical application and promotion of 2D vision technology in complex welding environments.

[0005] To overcome the shortcomings of 2D vision, welding robot systems based on 3D vision sensing technology have gradually emerged. 3D vision sensors can acquire complete three-dimensional geometric information of objects, including spatial position, shape and depth data, which significantly improves the accuracy and real-time performance of weld inspection in complex environments.

[0006] Existing 3D vision technologies mainly include structured light, laser scanning, and 3D camera imaging. Among these, three-line laser vision sensors, laser scanners, and laser 3D cameras have been successfully applied in welding scenarios. Meng et al. used a three-line laser sensor to capture the coordinates of weld feature points and employed a genetic algorithm to optimize welding parameters; Warner et al. constructed a weld feature extraction system that integrates a laser scanner and a vision camera; Rao et al. developed a bevel scanning and point cloud reconstruction method based on a laser 3D camera, effectively achieving adaptive planning and optimization of the welding path. These solutions fully utilize the advantages of 3D vision technology, such as high precision and robustness, to achieve real-time perception and dynamic adjustment of the welding path.

[0007] As welding scenarios become increasingly complex, traditional algorithms based on geometric structure analysis are struggling to effectively handle large-scale, diverse welding tasks, exhibiting insufficient accuracy and robustness in recognizing complex weld features. Therefore, deep learning methods, with their stronger generalization and learning capabilities, are gradually replacing traditional methods and have become a significant research trend in weld recognition and trajectory optimization. Currently, while deep learning models based on 2D image data have achieved some success in simple scenarios, they are limited by image quality and viewpoint conditions, making it difficult to accurately capture the 3D geometric features of complex workpieces, thus affecting practical application results. In contrast, deep learning methods based on point cloud data, although still in the early stages of research, directly use unstructured 3D data as input, demonstrating unique advantages and enormous potential in feature extraction from complex geometric structures.

[0008] To better handle unstructured point cloud data, Qi et al. proposed the groundbreaking PointNet model, which significantly improved the learning ability of unstructured data by automatically extracting global features directly from the raw point cloud data. However, PointNet lacks local feature extraction capabilities, limiting its effective recognition of detailed features. To address this issue, Qi et al. further proposed the PointNet++ model, which significantly improved the model's detection accuracy and generalization performance for complex structures and small-scale features through a multi-scale local region feature aggregation method. Building upon PointNet++, PointNext further optimized the network structure, employing residual connections and a split multilayer perceptron (MLP) to enhance the network's expressive power, further improving the model's generalization ability and processing efficiency for point cloud data.

[0009] Furthermore, to further improve the performance of point cloud data processing, researchers have explored various new network structures and methods. Wang et al. proposed Dynamic Graph CNN (DGCNN), which, by constructing a dynamically updated graph structure, more effectively captures the local geometric features of point clouds, improving the model's recognition ability in complex structures. Li et al.'s PointCNN model utilizes the X-Conv operator to adaptively learn the local arrangement relationships of points in the point cloud, significantly improving the robustness of point cloud feature extraction. Zhao et al. proposed the Point Transformer model, which, by introducing position encoding and self-attention mechanisms, designed a feature extraction module capable of simultaneously capturing global and local multi-scale information, demonstrating leading performance in multiple 3D point cloud recognition tasks and pioneering a new paradigm that does not rely on traditional feature processing methods. Zhou et al. proposed the first end-to-end 3D object detection network, VoxelNet, which, by converting point cloud data into dense voxel tensors and combining voxel feature extraction networks and region proposal networks, effectively achieves efficient 3D feature extraction and bounding box regression, significantly improving detection performance in complex scenes (such as the KITTI dataset).

[0010] Therefore, deep learning technology based on 3D point cloud data, combined with various advanced network structures, can not only overcome the limitations of traditional 2D methods, but also effectively improve the accuracy, efficiency and robustness of weld seam recognition and trajectory planning, thus promoting the development of welding automation technology to a higher level and having broad research value and application prospects.

[0011] Current welding automation technologies mainly include methods based on geometric structure analysis and data-driven deep learning methods. Among these, methods based on geometric structure analysis utilize traditional mathematical models and algorithms to analyze point cloud data and extract features, thereby achieving weld seam positioning and path planning, effectively improving welding accuracy. However, because this method relies on fixed mathematical models and predefined rules, it exhibits significant limitations when handling complex variations in workpiece size, shape, and position in actual industrial production, making it difficult to flexibly adapt to the complex structures and dynamic environmental requirements of welding scenarios.

[0012] In addition, although data-driven deep learning methods have shown a high level of automation in weld seam recognition and path planning, there are still many technical defects: (1) Single-view randomness problem: Most current deep learning models are mainly trained based on data collected under a single viewpoint, lacking an effective strategy for multi-view fusion. As a result, when the viewpoint changes in actual welding scenarios, the recognition effect of weld seam features will be significantly reduced, affecting the stability and reliability of the welding system under random viewpoints; (2) Insufficient model generalization ability: Existing deep learning methods are highly dependent on training data of specific weld types, and have limited generalization ability. In particular, for unknown composite welds that have not appeared in the training data, the accuracy of weld seam recognition and path planning is significantly reduced, making it difficult to meet the diverse and complex welding needs in actual industrial production; (3) Welding gun posture planning problem: Faced with the spatial orientation of multi-plane composite weld seams, existing methods lack the ability to uniformly model and analyze the geometric features and trajectory direction of weld seams, making it difficult to achieve accurate planning and continuous guidance of welding gun posture. Summary of the Invention

[0013] The purpose of this invention is to provide a method and system for weld seam recognition and trajectory optimization based on 3D vision, so as to solve the above-mentioned problems.

[0014] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a weld seam recognition and trajectory optimization method based on 3D vision, comprising: Point cloud data of the weld is collected to obtain the local geometric features of the weld. The local geometric features of the weld are enhanced and effective edge points are selected. Collect weld seam data, construct a multi-scale weld seam feature recognition network, input the selected effective edge points into the multi-scale weld seam feature recognition network to identify the weld seam region, and obtain the straight weld seam region; The straight weld region identified by the multi-scale weld feature recognition network is fitted to obtain the fitted welding trajectory. The fitted welding trajectory is used to describe and plan the welding torch posture in detail using a dihedral structure.

[0015] Furthermore, the point cloud data of the weld is collected to obtain the local geometric features of the weld, and the local geometric features of the weld are enhanced to filter out effective edge points, including: The K-nearest neighbor algorithm is used to find the k nearest neighbors of each point, and the Euclidean distance between each point and its neighbors is calculated. The calculation of local geometric features quantitatively describes the geometric characteristics of point cloud data of straight weld seams through spatial distribution characteristics, normal vector consistency, and corner change degree. It respectively characterizes the spatial distribution characteristics of local areas of point cloud, the smoothness of surface areas, and local turning characteristics, which work together to achieve a quantitative description of the geometric characteristics of point cloud data. After establishing the spatial relationship between each point and its neighbors using the K-Nearest Neighbors algorithm, a two-level Multilayer Perceptron (MLP) is used to further process the local geometric features of these points. The main function of the first-level MLP is to transform the local geometric features obtained in the previous step, such as spatial distribution characteristics, normal vector consistency, and corner change degree, into a higher-dimensional feature space. Then, the second-level MLP continues to perform deeper abstraction processing on these mapped features, enhancing the model's ability to recognize and process complex geometric information. To select effective edge points, each point is first considered as a node in the graph, and the connections between a point and its neighbors constitute the edges of the graph. By applying a Gaussian kernel function, the weights of the edges are calculated and defined, thereby constructing an overall weight matrix. Subsequently, weighting factors are introduced to adjust and control the relative contributions of different geometric parameters in local feature calculation, achieving an effective balance among the parameters. By weighting and combining each geometric feature parameter according to preset weights, the comprehensive geometric feature value of each point is calculated. Finally, based on these comprehensive geometric feature values, M representative points are selected as key edge points.

[0016] Furthermore, the process of collecting weld data, constructing a multi-scale weld feature recognition network, and inputting the selected effective edge points into the multi-scale weld feature recognition network for weld region recognition yields straight weld regions, including: We collected and labeled the weld seam dataset. First, we obtained weld seam point cloud data from a real welding environment. We divided the weld seams into short weld seams, standard weld seams, and extended weld seams according to different length standards and labeled them with label1, label2, and label3 respectively. The background point cloud was labeled with label0. At the same time, the data was divided into training set and validation set according to a certain ratio for subsequent model training and evaluation. The design incorporates a weld seam recognition network that integrates multi-scale point cloud feature fusion to extract point clouds of effective weld seam regions. The coarsely extracted point cloud data is then input into a target recognition network model for weld seam region identification. This network addresses the multi-scale and irregular distribution characteristics of weld seam point cloud features by fusing local and global features. For the input weld seam point cloud data that has already been grouped, multi-scale geometric feature extraction and aggregation are performed. First, a hierarchical SetAbstraction module extracts local features at different scales, fully exploring the geometric information of the weld surface at different scales. Then, a Multilayer Perceptron (MLP) is used to perform deep feature mapping on the coordinates, normal vectors, etc., of each local region, and max pooling is used to generate a representation of the region's geometry. In the multi-scale feature aggregation stage, a multi-scale grouping MSG strategy and feature fusion mechanism further integrate local and global information to form a comprehensive representation of the weld seam as a whole and its details.

[0017] Furthermore, the process of fitting the straight weld region identified by the multi-scale weld feature recognition network to obtain the fitted welding trajectory includes: Calculate the centroid of the point cloud. The average centroid of the point cloud data is calculated, and its coordinates are:

[0018] Subtract the centroid coordinates from the coordinates of each point so that the dataset is distributed with the centroid as the origin:

[0019] The covariance matrix is ​​used to describe the variance of the data on each coordinate axis and the covariance between different coordinate axes; the centered point set Organized into an N*3 matrix X:

[0020] Eigenvalue decomposition: The covariance matrix C is decomposed into eigenvalues ​​λ1, λ2, λ3 and their corresponding eigenvectors v1, v2, v3, satisfying the following conditions:

[0021] The eigenvalues ​​are arranged in descending order; the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the data; in line fitting, the eigenvector v1 corresponding to the largest eigenvalue λ1 is selected as the direction vector of the fitted line.

[0022] Furthermore, for the three-way intersecting trajectory, based on the above-mentioned straight line fitting strategy, the best fitting point search method is adopted. First, the start and end coordinates of each line segment are determined; second, an optimization model is defined to minimize the sum of squares of the Euclidean distances from an assumed intersection point to each line segment; numerical optimization methods are used to iteratively adjust the intersection point position until the point that minimizes the total distance is found, and this point is selected as the best fitting intersection point.

[0023] Furthermore, by extracting the start and end feature points of the weld, the weld centerline is accurately fitted. Based on this, a dihedral structure is used to describe and plan the welding torch posture in detail, ensuring precise alignment between the welding torch posture and the weld centerline. The steps for planning the welding torch posture and extracting the weld centerline are as follows: First, establish the weld coordinate system, and then establish the local coordinate system of the weld. : The axis is aligned with the weld axis. The axis is aligned with the centerline of the weld. The axis is determined by the right-hand rule. and Axis determined; The direction vector of the axis is calculated using the derivative at a point on the weld path, and is expressed as:

[0024] In the formula, Is the welding path in Rate of change in direction They are respectively The unit vector of direction; Fit the weld centerline directly through the characteristic points at the beginning and end of the weld. Perform linear fitting; Next, the normal vector of the welding torch posture is calculated. It can be obtained by cross-multiplying the weld centerline direction with the reference axis:

[0025] in, It is the unit vector of the reference axis; Determine the coordinate system relationship and use the secondary transformation matrix. Describe the rotation and translation relationship between the welding torch coordinate system and the weld coordinate system:

[0026] In the formula, These are elements of the rotation matrix, describing the relative rotation between the welding torch coordinate system and the weld seam coordinate system. It represents the displacement of the welding torch relative to the weld coordinate system; define the welding posture matrix, which passes through the weld centerline and has a normal vector. and homogeneous transformation matrix To define the homogeneous transformation matrix of the welding posture The formula is:

[0027] in, It is any point on the center line of the weld; Using a homogeneous transformation matrix at the current welding point The Euler angles of the welding posture are calculated to precisely control the spatial posture of the welding torch, ensuring that the welding torch always contacts the workpiece at the optimal angle along the welding path.

[0028] Secondly, the present invention provides a weld seam recognition and trajectory optimization system based on 3D vision, comprising: The data acquisition module is used to collect point cloud data of the weld, obtain the local geometric features of the weld, enhance the local geometric features of the weld, and filter out effective edge points; The identification network construction module is used to collect weld data, construct a multi-scale weld feature recognition network, input the selected effective edge points into the multi-scale weld feature recognition network to identify the weld region, and obtain the straight weld region. The welding trajectory fitting module is used to fit the straight weld area identified by the multi-scale weld feature recognition network to obtain the fitted welding trajectory. The trajectory optimization module employs a best-fit point search method to find the optimal common intersection point for common three-way intersecting trajectories, ensuring trajectory continuity.

[0029] Thirdly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a 3D vision-based weld seam recognition and trajectory optimization method.

[0030] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a 3D vision-based weld seam recognition and trajectory optimization method.

[0031] Compared with the prior art, the present invention has the following technical effects: The adaptive feature fusion method based on dynamic weight adjustment proposed in this invention has significant technical advantages. In the coarse extraction stage of weld edges, a weld plate geometric edge extraction algorithm based on point cloud local geometric feature metrics effectively replaces the traditional downsampling strategy. This method utilizes the K-nearest neighbor algorithm to accurately construct neighborhood relationships and comprehensively considers spatial distribution characteristics, normal vector consistency, and corner variation to quantitatively describe the geometric characteristics of the weld point cloud data. In this way, the invention not only avoids edge information loss caused by downsampling but also significantly improves the detection accuracy of weld edges, greatly enhancing the model's ability to express and predict weld characteristics, making it suitable for improving the automation and accuracy of industrial welding quality control.

[0032] In addition, this invention proposes a multi-scale weld feature recognition network that also demonstrates outstanding performance in weld recognition of complex welded structures. Traditional methods suffer from poor accuracy and stability in workpieces with irregular shapes, varied postures, and severe occlusion. This invention, combining AI deep learning technology, effectively solves these problems through multi-scale feature fusion, improving the accuracy and robustness of weld quality detection. This invention adaptively selects the neighborhood scale to ensure that weld point cloud data can uniformly and effectively retain structural information, thus addressing the problem of diverse weld features in complex welding environments. Secondly, it further employs a multilayer perceptron (MLP) and max pooling operation to extract robust local features, and introduces residual connections to eliminate the gradient vanishing problem caused by the deep network structure, while retaining high-frequency edge features. Regarding feature aggregation and fusion, this invention integrates local and global features through a Set Abstraction (SA) module and a multi-scale grouping (MSG) strategy, improving the model's adaptability and accuracy. Finally, the trained weld detection model can accurately identify weld regions and potential defects and map them to the actual workpiece coordinate system, providing support for automated processing. This technical solution improves the accuracy and efficiency of welding quality monitoring and ensures the stable identification of welding defects.

[0033] In welding trajectory optimization, for the corners of common three-way intersecting trajectories, the best-fit point search method is adopted. First, the start and end coordinates of each line segment are determined. Second, an optimization model is defined to minimize the sum of squares of the Euclidean distances from an assumed intersection point to each line segment. Numerical optimization methods are used to iteratively adjust the intersection point position until the point that minimizes the total distance is found. This point is then selected as the best-fit intersection point.

[0034] Furthermore, this invention also precisely extracts the characteristic points at the beginning and end of the weld and performs linear fitting of the weld centerline, combined with a dihedral structure to plan the welding torch's posture in detail. This innovative method significantly improves the posture control accuracy during the welding process, enabling the welding torch to always be precisely aligned with the weld centerline. By establishing a local coordinate system for the weld and using the homogeneous transformation of the welding torch posture matrix, the spatial relationship between the welding torch and the weld centerline is clarified, solving the technical challenge of aligning the welding torch posture with the weld trajectory. Attached Figure Description

[0035] Figure 1 This is a diagram illustrating the algorithm framework of the technical solution of this invention.

[0036] Figure 2 This is a framework diagram of the algorithm for extracting the geometric edges of the soldering plate.

[0037] Figure 3 To create the overall architecture diagram. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings: Figure 1 This is a diagram of the algorithm framework of this invention. (See diagram below.) Figure 1 As shown, it includes: For coarse extraction of the effective area of ​​the weld, this invention proposes a weld plate geometric edge extraction algorithm that replaces the traditional downsampling strategy. By constructing a local geometric feature metric for the point cloud, it effectively highlights the geometric features of the edge region and avoids the loss of edge information caused by downsampling in traditional methods. The specific steps are as follows: Step 1: Construct the domain relationships. Find the k nearest neighbors of each point using the K-Nearest Neighbor (KNN) algorithm and calculate the Euclidean distance between each point and its neighbors.

[0039] Step 2: Calculate local geometric features. Quantitatively describe the geometric characteristics of the point cloud data of the straight weld seam through spatial distribution characteristics, normal vector consistency, and corner variation. This step characterizes the spatial distribution characteristics of the local area of ​​the point cloud, the smoothness of the surface area, and the local turning characteristics, which work together to achieve a quantitative description of the geometric characteristics of the point cloud data.

[0040] Step 3: Enhancing Point Cloud Data Features. After establishing the spatial relationships between each point and its neighbors using the K-Nearest Neighbors algorithm, this invention employs a two-level Multilayer Perceptron (MLP) to further process the local geometric features of these points. The primary function of the first-level MLP is to transform the local geometric features obtained in the previous step, such as spatial distribution characteristics, normal vector consistency, and the degree of corner variation, into a higher-dimensional feature space, thereby enriching the representational dimensions of the data. Next, the second-level MLP continues to perform deeper abstraction processing on these mapped features, enhancing the model's ability to recognize and process complex geometric information. This series of nonlinear transformations and feature aggregations is completed through the two-level MLP, effectively providing high-quality feature input for the subsequent critical edge point screening stage, ensuring the accuracy and efficiency of the weld detection algorithm. Step 4: Filtering Effective Edge Points. To quantify the correlation between adjacent points in the point cloud data, each point is first considered as a node in a graph, and the connections between a point and its neighbors constitute the edges of the graph. By applying a Gaussian kernel function, the weights of the edges are calculated and defined, thus constructing an overall weight matrix. Subsequently, weighting factors are introduced to adjust and control the relative contributions of different geometric parameters in local feature calculation, achieving an effective balance among the parameters. By weighting and combining each geometric feature parameter according to preset weights, the comprehensive geometric feature value of each point is calculated. Finally, based on these comprehensive geometric feature values, M representative points are selected as key edge points, thereby effectively highlighting and extracting key feature information in solder plate edge detection.

[0041] 2. To address the challenges of traditional methods in identifying and extracting weld features from complex structural components, especially for workpieces with irregular shapes, varied postures, and severe occlusion, which leads to difficulties in ensuring recognition accuracy and stability, this invention combines AI deep learning technology to construct a multi-scale weld feature recognition network. Through a hierarchical structure and multi-scale feature aggregation design, it effectively solves the complex challenges of welding tasks in multiple scenarios and with multiple workpiece structures. This network can more accurately identify point cloud data of welding plates through local feature aggregation, especially when processing the geometric features of different types of weldments, adapting to welds in planar structures and complex interlaced structures. Specifically, this invention improves the recognition accuracy of welding point cloud data under various geometric conditions by introducing a multi-scale feature extraction mechanism combined with fine aggregation of local information, and successfully addresses the challenges of geometric deformation and structural complexity in different welding tasks. Specific technical solutions include: (1) Collect and label the weld seam dataset. First, obtain weld seam point cloud data in the real welding environment (e.g., by using an industrial camera). Divide the weld seam into short weld seam, standard weld seam, and extended weld seam according to different length standards, and label them with label1, label2, and label3 respectively. The background point cloud is labeled with label0. At the same time, divide the data into training set and validation set according to a certain ratio for subsequent model training and evaluation.

[0042] (2) Design a weld recognition network that combines multi-scale point cloud feature fusion to extract point clouds of effective weld regions. Input the coarsely extracted point cloud data of the weld region into the target recognition network model for weld region recognition. This network performs local and global feature fusion based on the characteristics of multi-scale and irregular distribution of weld point cloud features. For the weld point cloud input that has already been grouped, multi-scale geometric feature extraction and aggregation are directly performed. First, local features are extracted at different scales through the hierarchical Set Abstraction (SA) module to fully explore the geometric information of the weld surface at different scales. Then, the coordinates, normal vectors, etc. of each local region are mapped using a multilayer perceptron (MLP), and a geometric shape that can represent the region is generated through max pooling. At the same time, residual connections are introduced to eliminate the gradient vanishing problem caused by the deep network structure, while retaining high-frequency edge features. In the multi-scale feature aggregation stage, local and global information are further integrated through the multi-scale grouping (MSG) strategy and feature fusion mechanism to form a comprehensive representation of the weld as a whole and its details. This effectively addresses the diversity and complexity of the weld surface structure and improves the accuracy and robustness of weld identification and defect detection.

[0043] 3. Welding trajectory fitting: This invention uses principal component analysis (PCA) to fit the straight weld seam region identified by the network. The specific steps are as follows: Step 1: Calculate the centroid of the point cloud. Calculate the average position (centroid) of the point cloud data, and its coordinates are:

[0044] Subtract the centroid coordinates from the coordinates of each point so that the dataset is distributed with the centroid as the origin:

[0045] Step 2: Calculate the covariance matrix to describe the variance of the data on each coordinate axis and the covariance between different coordinate axes. Then, calculate the centered point set... Organized into an N*3 matrix X:

[0046] Step 3: Eigenvalue decomposition. Perform eigenvalue decomposition on the covariance matrix C to find its eigenvalues ​​λ1, λ2, λ3 and their corresponding eigenvectors v1, v2, v3, satisfying:

[0047] The eigenvalues ​​are arranged in descending order. The eigenvector corresponding to the largest eigenvalue is selected as the main direction of the data. In line fitting, the eigenvector v1 corresponding to the largest eigenvalue λ1 is selected as the direction vector of the fitted line.

[0048] Specifically, for three-way intersecting trajectories, this invention employs an optimal fitting point search method based on the aforementioned straight-line fitting strategy. First, the start and end coordinates of each line segment are determined. Second, an optimization model is defined to minimize the sum of squares of the Euclidean distances from an assumed intersection point to each line segment. Numerical optimization methods are used to iteratively adjust the intersection point position until a point that minimizes the total distance is found; this point is then selected as the optimal fitting intersection point. This method combines geometric constraint handling with optimization algorithms, ensuring both accuracy and efficiency in the solution. 4. Welding posture planning: This invention extracts the first and last feature points of the weld to accurately fit the weld centerline. Based on this, a dihedral structure is used to describe and plan the welding torch posture in detail, ensuring precise alignment between the welding torch posture and the weld centerline. The steps for planning the welding torch posture and extracting the weld centerline are as follows: Step 1: First, establish the weld coordinate system, starting with the local weld coordinate system. : The axis is aligned with the weld axis. The axis is aligned with the centerline of the weld. The axis is determined by the right-hand rule. and Axis determined.

[0049] The direction vector of the axis is calculated from the derivative at a point on the weld path and can be expressed as:

[0050] In the formula, Is the welding path in Rate of change in direction They are respectively A unit vector in direction.

[0051] Step 2: Fit the weld centerline directly through the characteristic points at the beginning and end of the weld. Perform linear fitting.

[0052] Next, the normal vector of the welding torch posture is calculated. The direction of the weld centerline can be aligned with the reference axis (e.g.) The cross product of the axes yields:

[0053] in, It is the unit vector of the reference axis.

[0054] Determine the coordinate system relationship and use the secondary transformation matrix. Describe the rotation and translation relationship between the welding torch coordinate system and the weld coordinate system:

[0055] In the formula, These are elements of the rotation matrix, describing the relative rotation between the welding torch coordinate system and the weld seam coordinate system. This represents the displacement of the welding torch relative to the weld coordinate system. Define the welding posture matrix, which passes through the weld centerline and has a normal vector. and homogeneous transformation matrix To define the homogeneous transformation matrix of the welding posture The formula is:

[0056] in, It is any point on the center line of the weld.

[0057] Step 3: Apply a homogeneous transformation matrix at the current welding point. The Euler angles of the welding posture are calculated to precisely control the spatial posture of the welding torch, ensuring that the welding torch always contacts the workpiece at the optimal angle along the welding path.

[0058] This invention proposes an adaptive feature fusion method based on dynamic weight adjustment. This method comprehensively constructs multiple geometric features in the neighborhood of point cloud, including spatial distribution characteristics, normal vector consistency, and corner point changes. It also introduces weight factors to flexibly adjust the influence of each parameter in the calculation of local geometric features, thereby effectively replacing the traditional downsampling strategy and highlighting the edge characteristics of the solder plate.

[0059] To address diverse and complex welding tasks, this invention proposes a welding point cloud learning method based on a multi-scale weld feature recognition network. This network employs a hierarchical structure and a multi-scale feature aggregation mechanism, enhancing the model's ability to perceive the local geometry of the weld region and improving weld detection capabilities in unknown welding scenarios. Compared to traditional algorithms, this method offers significant advantages in accuracy and robustness, better adapting to the changing environments of complex welding scenarios and meeting the demands of demanding industrial applications.

[0060] In another embodiment of the present invention, a 3D vision-based weld seam recognition and trajectory optimization system is provided, which can be used to implement the above-mentioned 3D vision-based weld seam recognition and trajectory optimization method. Specifically, the system includes: The data acquisition module is used to collect point cloud data of the weld, obtain the local geometric features of the weld, enhance the local geometric features of the weld, and filter out effective edge points; The identification network construction module is used to collect weld data, construct a multi-scale weld feature recognition network, input the selected effective edge points into the multi-scale weld feature recognition network to identify the weld region, and obtain the straight weld region. The welding trajectory fitting module is used to fit the straight weld area identified by the multi-scale weld feature recognition network to obtain the fitted welding trajectory. The trajectory optimization module employs a best-fit point search method to find the optimal common intersection point for common three-way intersecting trajectories, ensuring trajectory continuity.

[0061] The module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the invention can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0062] In another embodiment of the present invention, a computer device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to achieve a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a 3D vision-based weld seam recognition and trajectory optimization method.

[0063] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the 3D vision-based weld seam recognition and trajectory optimization method in the above embodiments.

[0064] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0065] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0066] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0067] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A 3D vision-based weld seam identification and trajectory optimization method, characterized in that, The method comprises the following steps: Collecting point cloud data of the weld to obtain local geometric features of the weld, enhancing the local geometric features of the weld, and screening effective edge points; Collecting weld data, constructing a multi-scale weld feature recognition network, inputting the screened effective edge points into the multi-scale weld feature recognition network for weld region recognition, and obtaining a straight-line weld region; Fitting the straight-line weld region recognized by the multi-scale weld feature recognition network to obtain a fitted welding trajectory; Using a dihedral structure to describe and plan the posture of the welding gun in detail based on the fitted welding trajectory.

2. The 3D vision-based weld seam identification and trajectory optimization method of claim 1, wherein, The method of collecting point cloud data of the weld to obtain local geometric features of the weld, enhancing the local geometric features of the weld, and screening effective edge points comprises the following steps: Find the k nearest neighbors of each point by K-neighbor algorithm, and calculate the Euclidean distance between each point and its neighbor; Calculate the local geometric features, quantitatively describe the geometric characteristics of the point cloud data of the straight-line weld through the spatial distribution characteristics, normal vector consistency and corner change degree, respectively depict the spatial distribution characteristics of the local region of the point cloud, the smoothness of the surface region and the local turning characteristics, and jointly act to quantitatively describe the geometric characteristics of the point cloud data; After establishing the spatial relationship of each point and its neighbor by K-neighbor algorithm, a two-level multi-layer perception (MLP) is used to further process the local geometric features of the points; the main function of the first-level MLP is to convert the local geometric features obtained from the previous step, such as spatial distribution characteristics, normal vector consistency and corner change degree, to a higher-dimensional feature space; then, the second-level MLP continues to abstract the mapped features in a deeper level, enhancing the model's ability to recognize and process complex geometric information; Screening effective edge points, first considering each point as a node of a graph, and the connection between the point and its neighbor points constitutes the edge of the graph; by applying a Gaussian kernel function, the weight of the edge is calculated and defined, and then the overall weight matrix is constructed; subsequently, a weight factor is introduced to adjust and control the relative contribution of different geometric parameters in local feature calculation, achieving effective balance between parameters; by weighting and combining each geometric feature parameter according to the preset weight, the comprehensive geometric feature value of each point is calculated; finally, based on these comprehensive geometric feature values, M representative points are selected as key edge points.

3. The 3D vision based weld seam identification and trajectory optimization method of claim 1, wherein, The method of collecting weld data, constructing a multi-scale weld feature recognition network, inputting the screened effective edge points into the multi-scale weld feature recognition network for weld region recognition, and obtaining a straight-line weld region comprises the following steps: Collecting weld data set and labeling, first obtaining weld point cloud data in a real welding environment, dividing the weld into short weld, standard weld and extended weld according to different length standards, marking label1, label2 and label3 respectively, and marking background point cloud as label0; at the same time, the data is divided into training set and validation set in a certain proportion for subsequent model training and evaluation; A welding seam recognition network combining multi-scale point cloud feature fusion is designed to extract the point cloud of the effective welding seam area; the point cloud data after rough extraction of the welding seam area is input into a target recognition network model for welding seam area recognition. The network is characterized by multi-scale and irregular distribution of welding seam point cloud features, and performs local and global feature fusion; multi-scale geometric feature extraction and aggregation are performed for the welding seam point cloud after data grouping; first, the local features are extracted at different scales through the hierarchical SetAbstraction module, fully mining the geometric information of the welding seam surface at different scales; then, the coordinates, normal vectors, etc. of each local region are mapped to deep features using a multi-layer perceptron (MLP), and a geometry representation of the region is generated through max pooling; in the multi-scale feature aggregation stage, the local and global information is further integrated through the multi-scale grouping (MSG) strategy and feature fusion mechanism to form a comprehensive representation of the welding seam as a whole and in detail.

4. The 3D vision based weld seam identification and trajectory optimization method of claim 1, wherein, The linear welding seam area recognized by the multi-scale welding seam feature recognition network is fitted to obtain a fitted welding trajectory, including: The centroid of the point cloud is calculated as the average position of the point cloud data, and the coordinates of the centroid are: The coordinates of each point are subtracted from the centroid coordinates, so that the data set is distributed with the centroid as the origin: The covariance matrix is calculated to describe the variance of data on each coordinate axis and the covariance between different coordinate axes; the centralized point set is organized into an N*3 matrix X: Eigenvalue decomposition is performed on the covariance matrix C to solve its eigenvalues λ1, λ2, λ3 and their corresponding eigenvectors v1, v2, v3, which satisfy: The eigenvalues are arranged in descending order; the eigenvector corresponding to the largest eigenvalue is selected as the main direction of the data; in linear fitting, the eigenvector v1 corresponding to the largest eigenvalue λ1 is selected as the direction vector of the fitted straight line.

5. The 3D vision based weld seam identification and trajectory optimization method of claim 4, wherein, For three-way intersecting trajectories, the best fitting point search method is used based on the above linear fitting strategy; first, the starting and ending coordinates of each line segment are determined; second, an optimization model is defined to minimize the sum of the squares of the Euclidean distances from a hypothetical intersection point to each line segment; numerical optimization methods are used to iteratively adjust the intersection point position until the point that minimizes the total distance is found, and this point is selected as the best fitting intersection point.

6. The 3D vision based weld seam identification and trajectory optimization method of claim 1, wherein, By extracting the first and last feature points of the welding seam, the welding seam centerline is accurately fitted, and on this basis, the welding gun pose is described and planned in detail using a dihedral structure to ensure accurate alignment of the welding gun pose with the welding seam centerline; the welding gun pose planning and welding seam centerline extraction steps are as follows: First, the weld coordinate system is established, and a local weld coordinate system is first established : The shaft is aligned with the weld axis. The shaft is aligned with the weld centerline. The axis is determined by the right-hand rule from and the axis The direction vector of the axis is calculated by the derivative of a certain point on the weld path and is expressed as: wherein is the rate of change of the welding path in the direction, are the unit vectors in the directions, respectively. fitting the weld centerline directly through the weld start and end feature points performing a linear fit; Next, the normal vector of the welding torch pose is calculated, the normal vector By cross-multiplying the welding seam center line direction and the reference axis, we get: wherein is a unit vector of the reference axis; determining the coordinate system relationship, using the secondary transformation matrix describing the rotational and translational relationship of the welding torch coordinate system relative to the weld seam coordinate system: wherein is an element of the rotation matrix describing the relative rotation of the welding torch coordinate system to the weld seam coordinate system, is the displacement of the welding torch relative to the weld seam coordinate system; defines the welding pose matrix through the weld centerline, normal vector and the homogeneous transformation matrix defines the homogeneous transformation matrix of the welding pose is given by wherein is an arbitrary point on the weld center line; Utilizing homogeneous transformation matrices at the current weld point Euler angles of the welding pose are calculated for precise control of the spatial pose of the welding torch, ensuring that the welding torch always contacts the workpiece at the optimal angle on the welding path.

7. A 3D vision based weld seam identification and trajectory optimization system, characterized in that, It includes: A data acquisition module for acquiring point cloud data of a welding seam to obtain local geometric features of the welding seam, enhancing the local geometric features of the welding seam, and selecting effective edge points; An identification network construction module for collecting welding seam data and constructing a multi-scale welding seam feature recognition network, inputting the selected effective edge points into the multi-scale welding seam feature recognition network for welding seam area recognition to obtain a linear welding seam area; A welding trajectory fitting module for fitting the linear welding seam area recognized by the multi-scale welding seam feature recognition network to obtain a fitted welding trajectory. The supplement optimization module is configured to perform point cloud data supplement optimization on the fitted welding trajectory by using surface reconstruction.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the steps of the 3D vision-based weld seam identification and trajectory optimization method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, the computer-readable storage medium comprising: The processor implements the steps of the 3D vision-based weld seam identification and trajectory optimization method according to any one of claims 1 to 6 when executing the computer program.

Images