A welding seam extraction and robot trajectory generation method based on digital-analog prior guidance and related products

Abstract

This application discloses a method and related products for weld seam extraction and robot trajectory generation based on digital model prior guidance, relating to the field of automated welding technology. The method includes: acquiring downsampled point clouds and undownsampled point clouds, where the downsampled point clouds are obtained by downsampling a coarsely registered point cloud, and the undownsampled point clouds are the coarsely registered point clouds without downsampling; finely registering the downsampled point clouds with the digital model point cloud to obtain a fine registration matrix; projecting the undownsampled point clouds onto the spatial coordinate system of the digital model point cloud using this matrix to obtain a projected point cloud; spatially clipping the projected point cloud using a predefined region of interest (ROI) for the weld seam in the digital model point cloud to obtain a weld seam region point cloud; performing local geometric feature analysis on the weld seam region point cloud, and extracting the weld seam trajectory based on a discrete set of weld seam feature points, and generating a smooth robot welding trajectory. This application can achieve high-precision, low-computing-cost weld seam trajectory generation.

Description

Technical Field

[0001] This application relates to the field of automated welding technology, and in particular to a method for weld seam extraction and robot trajectory generation based on prior digital model guidance, and related products. Background Technology

[0002] Existing technologies mostly consist of traditional weld seam extraction methods based on laser structured light and traditional geometric feature detection, and semantic segmentation methods based on deep learning. Traditional extraction methods are highly dependent on specific lighting environments and regular bevel geometry. In actual welding sites of large components, rust, reflections, and irregular spatter on the workpiece surface can all generate noise interference, resulting in insufficient robustness of traditional algorithms in feature point localization, making them prone to recognition errors or even algorithm failure.

[0003] In contrast, deep learning-based semantic segmentation methods exhibit stronger adaptability, but this approach still faces challenges in practical engineering applications when dealing with ultra-large structural components. Limited by the physical field of view of sensors, a single scan often only acquires local geometric information, failing to capture the complete structure sufficient to support global semantic understanding. Under these conditions of field-of-view deficiency, end-to-end deep segmentation networks struggle to establish a consistent association with the global model through local, fragmented features, and the real-time processing of ultra-large-scale point clouds places extremely high demands on computational resources. Summary of the Invention

[0004] The purpose of this application is to provide a method and related products for weld seam extraction and robot trajectory generation based on prior knowledge of digital models. It can use prior knowledge of digital models for global guidance and local focusing for weld seam extraction, and achieve high-precision weld seam trajectory generation with low computational overhead.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for weld seam extraction and robot trajectory generation based on prior knowledge modeling, including: Obtain downsampled point clouds and undownsampled point clouds; the downsampled point cloud is the point cloud obtained after downsampling the coarse registration point cloud; the undownsampled point cloud is the point cloud that has not been downsampled from the coarse registration point cloud; The downsampled point cloud is precisely registered with the predefined digital model point cloud to obtain a fine registration matrix. Using the fine registration matrix, the unsampling point cloud is projected onto the spatial coordinate system of the digital model point cloud to obtain the projected point cloud; Using the predefined region of interest for the weld in the digital model point cloud, the projected point cloud is spatially clipped to obtain the weld region point cloud; Local geometric feature analysis is performed on the point cloud of the weld area to extract a discrete set of weld feature points; A continuous welding trajectory is generated based on the discrete set of weld feature points.

[0006] Optionally, acquiring the downsampled point cloud and the undownsampled point cloud specifically includes: Obtain the original captured point cloud; The original captured point cloud is coarsely registered with the predefined digital model point cloud to obtain a coarsely registered point cloud. The coarsely registered point cloud is filtered to remove duplicate points, resulting in a filtered point cloud. Save the filtered point cloud as the undownsampled point cloud; The filtered point cloud is then subjected to spatial voxel downsampling to obtain a downsampled point cloud. The downsampled point cloud is clustered to extract the largest cluster and remove discrete points, thus obtaining the downsampled point cloud.

[0007] Optionally, the downsampled point cloud is finely registered with a predefined digital model point cloud to obtain a fine registration matrix, specifically including: The Trimming ICP algorithm is used to iteratively register the downsampled point cloud and the predefined digital model point cloud to obtain a fine registration matrix.

[0008] Optionally, local geometric feature analysis is performed on the point cloud of the weld region to extract a discrete set of weld feature points, specifically including: A local coordinate basis is constructed based on the point cloud of the weld area using principal component analysis. On the local coordinate base, knee point detection is performed by comparing normal vectors to screen out candidate points that constitute the bevel sidewall; Clustering the candidate points yields a cluster of points on the bevel sidewall; Spatial layering and slicing are performed on the bevel sidewall point cluster to extract discrete weld feature point sets.

[0009] Optionally, the candidate points are clustered to obtain a cluster of points on the bevel sidewall, specifically including: The candidate points were clustered using the dual DBSCAN clustering method to obtain the bevel sidewall point cluster.

[0010] Optionally, a continuous welding trajectory is generated based on the discrete set of weld feature points, specifically including: The discrete weld feature point set is interpolated and smoothed using cubic B-splines to obtain a preliminary smooth trajectory; The initial smoothed trajectory is transformed into the robot base coordinate system using the fine registration matrix and the hand-eye calibration matrix to obtain the transformed trajectory; the hand-eye calibration matrix is ​​used to describe the coordinate transformation relationship between the 3D camera that captured the original point cloud and the robot that performs welding. The transformed trajectory is subjected to quadratic B-spline interpolation in the robot base coordinate system to obtain a continuous welding trajectory.

[0011] Optionally, the continuous welding trajectory includes spatial pose information and corresponding weld width information; the weld width information is used to adjust welding process parameters.

[0012] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the weld seam extraction and robot trajectory generation method based on prior guidance of the numerical model as described above.

[0013] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method for weld seam extraction and robot trajectory generation based on prior digital model guidance.

[0014] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method for weld seam extraction and robot trajectory generation based on prior digital model guidance.

[0015] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method and related products for weld seam extraction and robot trajectory generation based on prior digital model guidance. The method includes: firstly, acquiring downsampled point clouds and undownsampled point clouds; the downsampled point cloud is the point cloud obtained after downsampling a coarsely registered point cloud; the undownsampled point cloud is the point cloud without downsampling the coarsely registered point cloud. Then, the downsampled point cloud is finely registered with a predefined digital model point cloud to obtain a fine registration matrix; using the fine registration matrix, the undownsampled point cloud is projected onto the spatial coordinate system of the digital model point cloud to obtain a projected point cloud. Based on this, the projected point cloud is spatially clipped using a predefined region of interest (ROI) for the weld seam in the digital model point cloud to obtain a weld seam region point cloud. This step utilizes a pre-defined ROI in a priori map to clip the measured point cloud, thereby achieving a logical shift from global spatial localization to local feature analysis. Next, local geometric feature analysis is performed on the weld seam region point cloud to extract a discrete set of weld seam feature points. Finally, a continuous welding trajectory is generated based on the discrete weld feature point set. This application adopts a "global guidance-local extraction" strategy, using the spatial transformation matrix obtained through global registration and predefined regions of interest in the digital model to crop the measured point cloud. This strategy effectively avoids the dependence of deep learning methods on the complete global geometry, and uses the topological constraints provided by the 3D model to compensate for the shortcomings of the sensor's field of view, enabling the correct extraction of the corresponding weld region while taking into account the global structure. By cropping the bounding box of the weld region that truly needs to be processed, this application removes a large number of background and main structure point clouds that are irrelevant to the welding task, greatly reducing the waste of computational resources and improving processing efficiency. This application performs feature analysis on the cropped local weld region point cloud, which can better analyze the detailed features of the weld segment and more accurately screen weld groove features, thereby improving the accuracy of weld feature point extraction. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is an application environment diagram of a weld seam extraction and robot trajectory generation method based on numerical model prior guidance in one embodiment of this application; Figure 2 A flowchart illustrating a method for weld seam extraction and robot trajectory generation based on prior digital modeling, provided as an embodiment of this application; Figure 3A flowchart illustrating the preprocessing stage of raw point cloud images provided in an embodiment of this application; Figure 4 A schematic flowchart of the trajectory extraction stage provided in an embodiment of this application; Figure 5 This is a schematic diagram of the cut-out weld area provided in one embodiment of this application; Figure 6 A schematic diagram of a cluster of dots on a bevel sidewall provided in an embodiment of this application; Figure 7 A schematic diagram of the final bevel sidewall provided for an embodiment of this application; Figure 8 A schematic diagram of the preliminary extraction result of the weld trajectory provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0020] The weld seam extraction and robot trajectory generation method based on prior digital modeling provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server.

[0021] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0022] In one exemplary embodiment, such as Figure 2As shown, a method for weld seam extraction and robot trajectory generation based on prior knowledge modeling is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S6. Wherein: S1. Obtain downsampled point cloud and undownsampled point cloud; the downsampled point cloud is the point cloud obtained after downsampling the coarse registration point cloud; the undownsampled point cloud is the point cloud that has not been downsampled from the coarse registration point cloud.

[0023] S2. Perform fine registration between the downsampled point cloud and the predefined digital model point cloud to obtain the fine registration matrix.

[0024] S3. Using the fine registration matrix, project the unsampling point cloud onto the spatial coordinate system of the digital model point cloud to obtain the projected point cloud.

[0025] S4. Using the predefined region of interest for the weld in the digital model point cloud, spatially crop the projected point cloud to obtain the weld region point cloud.

[0026] S5. Perform local geometric feature analysis on the point cloud of the weld area to extract a discrete set of weld feature points.

[0027] S6. Generate a continuous welding trajectory based on the discrete set of weld feature points.

[0028] Due to the limited field of view of cameras, there is a problem of insufficient information acquisition when dealing with large structural components. Only local geometric information can be obtained, failing to capture the complete structure sufficient to support global semantic understanding. This is why deep learning methods cannot be applied in this case. This embodiment utilizes a predefined digital model point cloud to supplement global information. That is, although the camera acquires local information, it is associated with global digital model information through coarse and fine registration. Based on the preset global information, the local information is pruned, thereby locking the actual weld segment region and further reducing the amount of computation. This local-global-local approach effectively solves the problem of utilizing global structural information when the local field of view is limited. Compared with directly analyzing local areas, it greatly reduces the amount of computation and lowers the error probability, thus solving the problem of weld extraction for large components.

[0029] In another exemplary embodiment of this application, such as Figure 3 As shown, step S1 above can be achieved by the following steps: Obtain the original image point cloud; in this embodiment, the original image point cloud is obtained by taking pictures of the welds at the left and right side beams and bottom crossbeam of the real target structural component, which is the data used for weld extraction later.

[0030] The original captured point cloud is coarsely registered with the predefined digital model point cloud to obtain a coarsely registered point cloud. The coarsely registered point cloud is filtered to remove duplicate points, resulting in a filtered point cloud.

[0031] The filtered point cloud is saved as the unsampling point cloud.

[0032] The filtered point cloud is downsampled by spatial voxels (4 mm) to obtain the downsampled point cloud.

[0033] The downsampled point cloud is then subjected to clustering processing (DBSCAN clustering is used in this embodiment), and the largest cluster is extracted to remove discrete points, resulting in a downsampled point cloud.

[0034] In another exemplary embodiment of this application, step S2 specifically involves: using the Trimming ICP algorithm to iteratively register the downsampled point cloud and the predefined digital model point cloud to obtain a fine registration matrix. The cropping ratio k is set to 95%.

[0035] The overall Trimming ICP consists of two main steps: preprocessing and initialization, and iterative registration loop. The preprocessing and initialization phase requires preparing the data and setting the initial state, including the input data (downsampled point cloud P and predefined digital model point cloud Q); followed by the initial transformation and clipping ratio k. Then, the iterative registration loop begins, with each iteration executing the following four core steps: Step 1: Finding the nearest neighbor: For each point pi in the transformed source point cloud (i.e., the downsampled point cloud) P, find the nearest point qi in the target point cloud (i.e., the predefined digital model point cloud) Q. Step 2: Sorting and clipping: This is the core of Trimming ICP. All calculated distances are sorted in ascending order. Based on the set k = 95%, the number of points to retain, m, is calculated. Outliers are removed: only the first m pairs of points with the smallest distance are retained, ignoring the remaining 5%. Principle: These 5% of points are often noise points caused by scan occlusion, model detail differences, or downsampling; forcing them to participate in registration will skew the results. The third step calculates the optimal spatial transformation through singular value decomposition. Finally, updates and convergence checks are performed. After the stopping iteration condition is met, the transformations from each round are accumulated to obtain the final fine registration matrix.

[0036] In another exemplary embodiment of this application, steps S3-S4 described above can be specifically implemented by the following steps: Applying the registration matrix to the unsampling point cloud achieves high-precision pose alignment between the original captured point cloud and the predefined digital model point cloud in a unified coordinate system. This allows for the pre-definition of the weld seam region of interest using prior geometric knowledge from the digital model. Spatial clipping based on this predefined region of interest removes background and main structure point clouds irrelevant to the welding task, retaining only the crucial weld seam area point cloud. This step represents the logical shift from global spatial localization to local feature analysis. By preserving the key weld seam area point cloud, the algorithm's subsequent computational load is reduced, providing a clean data source, thereby enabling high-precision weld seam trajectory fitting and width information recognition.

[0037] The reason for pre-defining the region of interest (ROI) for the weld seam using prior geometric knowledge from the mathematical model is that a standard 3D model of the structural component can be obtained in advance. Therefore, the location of the weld seam can be pre-identified within the point cloud extracted from the 3D model. This doesn't involve directly extracting the specific weld seam, but rather a small area including the weld seam and its surrounding point cloud; the goal is to narrow down the search area and reduce computational load. Thus, the point cloud of this region can be pre-selected in the mathematical model, typically accompanied by a cuboid bounding box, specifying the upper and lower limits of the x, y, and z coordinates. Points in the registered real point cloud that conform to these three coordinate limits are considered to be within the ROI of the weld seam. Therefore, this entire region can be extracted, thus extracting the weld seam segment point cloud from the real point cloud, retaining only the critical weld seam area point cloud.

[0038] The spatial clipping is based on a pre-defined region of interest (ROI) for the weld seam on the mathematical model point cloud. Since this only includes a region of the weld seam, the selection criteria do not need to be very strict. This pre-defined region is usually accompanied by a rectangular virtual bounding box, which defines the extent of this region by specifying the upper and lower limits of the x, y, and z coordinates. Points in the registered downsampled point cloud that meet these three coordinate limits are considered to be within the weld seam ROI. This allows for automatic clipping of the downsampled point cloud by pre-setting coordinate range information on the mathematical model point cloud.

[0039] In another exemplary embodiment of this application, step S5 described above can be replaced by the following step: A local coordinate basis is constructed based on the point cloud of the weld area using principal component analysis.

[0040] On the local coordinate base, knee point detection is performed by comparing normal vectors to screen out candidate points that constitute the bevel sidewall.

[0041] The candidate points are clustered to obtain a cluster of points on the bevel sidewall. In this embodiment, the candidate points can be clustered using the dual DBSCAN clustering method.

[0042] Spatial layering and slicing are performed on the bevel sidewall point cluster to extract discrete weld feature point sets.

[0043] In another exemplary embodiment of this application, step S6 described above can be replaced by the following step: The discrete weld feature point set is interpolated and smoothed using cubic B-splines to obtain a preliminary smooth trajectory; The initial smoothed trajectory is transformed into the robot base coordinate system using the fine registration matrix and the hand-eye calibration matrix to obtain the transformed trajectory; the hand-eye calibration matrix is ​​used to describe the coordinate transformation relationship between the 3D camera that captured the original point cloud and the robot that performs welding. The transformed trajectory is subjected to quadratic B-spline interpolation in the robot base coordinate system to obtain a continuous welding trajectory.

[0044] In another exemplary embodiment of this application, such as Figure 4 As shown, steps S5-S6 above can also be implemented through the following specific steps: In the trajectory extraction stage, by constructing a local geometric basis (which is essentially building a coordinate system to facilitate the analysis of weld region features, using principal component analysis to construct the coordinate axes), adaptive feature filtering and spatial layering slicing are performed to transform the disordered point cloud into a continuous welding trajectory with width attributes. First, a local coordinate basis is constructed using principal component analysis (PCA). After constructing the local basis, the bevel sidewall needs to be located from the complex region of interest point cloud. To ensure the uniqueness and accuracy of trajectory extraction, the algorithm introduces a dual DBSCAN clustering strategy. After obtaining the bevel sidewall point cluster, the trajectory is discretized and its width is quantized using the spatial sliding slicing method. After extracting discrete feature points, the algorithm uses cubic B-splines for initial smoothing. Then, using the coarse and fine registration matrices and hand-eye calibration matrices calculated earlier, the trajectory is transformed from the local coordinate system back to the robot's base coordinate system. In this space, quadratic B-spline interpolation is applied to fuse the trajectory of the entire welding area and resample the width information. Finally, four-dimensional guidance data containing high-precision spatial pose (XYZ) and physical width (Width, i.e., weld width information) is generated. This not only provides the robot with high-precision spatial positioning, but also provides data support for subsequent adaptive adjustment of welding process parameters (such as current magnitude and welding speed) based on weld width information.

[0045] The specific steps for constructing a local coordinate basis using principal component analysis (PCA) include: The bounding box of the weld area point cloud obtained from the previous clipping is usually a cuboid. This step, Principal Component Analysis (PCA), constructs a basis for the clipped point cloud, maintaining a consistent coordinate system for each cluster. For a rectangular (or cuboid) point cloud, PCA extracts three mutually perpendicular eigenvectors, representing: its principal direction (the direction with the largest corresponding eigenvalue, i.e., its length), its second principal direction (its width), and its third principal direction (its normal vector). The bevel sidewall is mainly determined by comparing the angle between the normal vector of each point in the point cloud and the second principal direction. (See also...) Figure 5 , Figure 5 This is a schematic diagram of the cut-out weld area. The turning point in the middle represents the weld area. The normal vectors of points not located at the bevel are approximately perpendicular to the second principal direction, while those at the bevel are approximately parallel to the second principal direction. Therefore, a segmentation threshold can be set; points with angles smaller than this threshold are considered bevel sidewall point clouds. This initial selection of point clouds at the bevel sidewalls is used for further precise filtering later.

[0046] After obtaining the point cluster on the bevel sidewall, the trajectory is discretized and its width is quantized using the spatial sliding slice method, specifically including: The above steps have yielded a preliminary cluster of points on the bevel sidewall, such as... Figure 6 As shown, a dual DBSACAN clustering process is then performed. First, spatial clustering is performed in 3D space to eliminate small clusters. Then, all candidate points are projected onto the second principal direction axis, and on-axis clustering is performed based on the horizontal axis coordinates. Since the trimming is centered on the estimated weld position, the algorithm calculates the centroid of each projected cluster and selects the cluster of points closest to the center of the local coordinate system as the final bevel sidewall. Figure 7 As shown. Once this feature is obtained, the trajectory can be discretized and extracted. The depth axis is actually the third principal direction, which is the direction upwards perpendicular to the point cloud plane.

[0047] After obtaining the point cluster on the bevel sidewall, the trajectory is discretized and the width is quantized by the spatial sliding slice method.

[0048] (1) Axial layering and slicing: The algorithm segments the point cloud along the first principal direction. The point cloud is then sliced ​​according to a set slice width Δ. w =2.0mm is divided into N intervals along the axial direction. For each slice interval Si, the local point set falling within the range is analyzed and projected onto the depth axis, thereby simplifying the three-dimensional search problem into a two-dimensional cross-sectional analysis.

[0049] (2) Boundary extraction based on depth extrema: Within each slice section, the algorithm uses the depth component as the geometric feature discrimination criterion. The local point set is projected onto the depth axis, and the two points with the smallest and largest projected depths are selected as the left boundary point P1 and the right boundary point P2, respectively. Furthermore, the distance between the boundary points can also be calculated as the weld feature width at that location.

[0050] (3) Multiple statistical filtering removal: The initially extracted centerline sequence may contain misidentification interference and needs to be filtered under dual constraints of width and curvature. Since the width variation of the actual weld is continuous, the algorithm performs first-order difference processing on the width sequence. Calculate the width fluctuation ΔD between adjacent slices. i =|D i -D i-1 |, and set the exclusion threshold Tw using statistical criteria: ; in and These are the mean and standard deviation of the width difference sequence, respectively. Any trajectory points exceeding these thresholds are considered outliers and removed to ensure the physical accuracy of the width information.

[0051] To meet the motion smoothness requirements of the welding robot, the algorithm monitors the curvature of the trajectory point sequence. This is achieved by calculating the angle α between the direction vectors formed by three adjacent points. i The system assesses local curvature changes and introduces the median absolute deviation to construct a robust threshold, eliminating "sawtooth" abrupt changes in the trajectory caused by boundary deviations and ensuring the geometric consistency of the path.

[0052] This yields the left and right boundary point sets of the bevel sidewall features. Based on these two point sets, a set of trajectory center points is calculated. Simultaneously, the weld width for each point can be calculated based on its corresponding left and right boundary points, serving as the width attribute of each trajectory center point. For example... Figure 8 As shown, blue and green represent the left and right boundary point groups extracted based on the bevel features, while black dots represent the trajectory center points calculated based on the left and right boundary points, which are the preliminary extraction results of the weld trajectory.

[0053] After extracting discrete feature points, the algorithm uses cubic B-splines for initial smoothing, specifically including: The above describes the initial weld trajectory points calculated based on the left and right boundary points of the bevel features, using the center point calculation method. To eliminate the influence of partial point identification errors, and considering the ultimate goal of smooth welding by the robotic arm, a cubic B-spline interpolation is performed on the initial weld trajectory points obtained above. This is because if the robot were to move in a straight line directly, the weld would be a series of broken lines. The cubic B-spline interpolation generates a perfect, smooth trajectory function for the robot, from which thousands of points can be evenly extracted. Typically, twice the number of initial weld trajectory points is sampled here as the final welding trajectory points.

[0054] The coarse registration matrix is ​​currently predefined by restricting the relative positions of the camera and the workpiece, keeping their relative positions fixed. This coarse registration matrix is ​​calculated and predefined manually in advance. The hand-eye calibration matrix reflects the spatial transformation relationship between the camera fixed at the end of the robotic arm and the welding torch end of the robotic arm. Because their relative positions are also fixed, the hand-eye calibration matrix is ​​also predefined. The question then becomes how the hand-eye calibration matrix is ​​obtained.

[0055] The transformation of the trajectory from the local coordinate system back to the robot's base coordinate system specifically includes: This system primarily comprises four coordinate systems. The first is the camera coordinate system, with the camera's optical center as the origin; the original point cloud coordinates are all in this system. The second is the tool coordinate system, with the robotic arm's welding torch tip as the zero point. The third is the robot base coordinate system, with the robotic arm's base as the zero point. The fourth is the spatial coordinate system of the predefined mathematical model, referred to here as the local coordinate system. After the coarse and fine registration transformations described earlier, the point cloud coordinates were effectively transformed from the camera coordinate system to the local coordinate system. Therefore, the calculated welding trajectory points are also relative to the local coordinate system. By retaining the previously applied coarse and fine registration transformations and applying an inverse transformation to the calculated welding trajectory points, we can obtain the coordinate values ​​of the welding trajectory points in the camera coordinate system. Based on this, we further apply the spatial transformation relationship of the hand-eye calibration matrix, transforming the coordinate values ​​of the welding trajectory points from the camera coordinate system to the tool coordinate system. The transformation from the tool coordinate system to the robot base coordinate system is performed automatically within the robot. This series of transformations ensures that the extracted welding trajectory points are located in the robot's base coordinate system. This gives the weld trajectory points extracted by the camera practical value to the robotic arm. Therefore, the robotic arm can perform welding based on the coordinates of the welding trajectory points in the robot's base coordinate system, thus completing the welding task of the actual workpiece.

[0056] Specifically, quadratic B-spline interpolation is applied in this space to perform trajectory fusion and width information resampling of the entire welding area, including: Because the target structural component in this embodiment is quite large, it is impossible to capture all welds in a single photograph. Therefore, all areas requiring welding are photographed seven times. Based on the principles described above, parallel calculations are performed on the computer to simultaneously obtain seven welding trajectory points at different locations on the structural component. Since welding needs to be completed in one go, these seven trajectories need to be merged. Because these seven trajectories include three different types of welds—left weld, right weld, and bottom weld—secondary B-spline interpolation is performed on segments 1-3 and 4-6 of the seven welding trajectory points. This merges segments 1-3 into one segment and 4-6 into another, ultimately resulting in smooth and complete welding trajectories for the left, right, and bottom welds. These are then used for actual welding tasks.

[0057] This embodiment provides a method for weld seam extraction and robot trajectory generation based on prior mathematical model guidance. By finding the spatial mapping relationship between the actual captured point cloud and a predefined point cloud model, the actual captured point cloud data is projected into the point cloud model space. The actual captured point cloud is cropped using a priori bounding box for the weld seam region selected in the mathematical model, thereby narrowing the feature search range. Finally, on a small area of ​​the weld seam segment point cloud, principal direction analysis is performed to construct a local coordinate basis. Knee point detection is performed based on normal vector comparison to filter the point set. After a series of clustering filtering strategies, B-spline interpolation is performed, followed by inverse spatial transformation, hand-eye calibration conversion, and multi-weld seam trajectory fusion. This yields a weld seam trajectory that can guide the robotic arm in welding, and this trajectory retains the corresponding weld seam width information.

[0058] To address the unique characteristics of large-sized components, the aforementioned method is a local feature extraction scheme guided by numerical modeling. Similar to the "global-to-local" approach of deep learning schemes, the core logic of this type of method lies in: using the spatial transformation matrix obtained from global registration, the complete target model is projected as a prior map onto the local measured point cloud space; the measured point cloud is then cropped by pre-dividing regions of interest (ROIs) using the prior map to achieve local localization and feature extraction, while also reducing computational load.

[0059] This "global guidance-local extraction" strategy effectively avoids the dependence of deep learning on the complete global geometry, using the topological constraints provided by the 3D model to compensate for the limitations of the sensor's field of view. Thus, it can correctly extract the corresponding weld seam region while taking into account the global structure, and by clipping the bounding box of the area that truly needs processing, it removes a large number of irrelevant parts, greatly reducing the waste of computational resources.

[0060] Furthermore, since the subsequent feature analysis is based on the local point cloud, it can better analyze the detailed features of the weld segment, better filter the weld groove features, and obtain more accurate weld width information.

[0061] In addition, for workpiece-oriented automated welding technology, there are two main alternatives. The first is the traditional teach-and-playback mode, which relies on manual point-by-point teaching and recording of weld information to guide the robotic arm in welding sequentially based on the recorded information. However, this method requires a long production preparation cycle, resulting in efficiency bottlenecks. Furthermore, it heavily relies on clamping accuracy and non-rigid constraints, while the actual pose of the workpiece in the above embodiments often has some deviation. Simultaneously, the teaching trajectory, as a static "preset instruction," cannot identify and compensate for spatial offsets caused by random placement. Moreover, the teaching method is essentially "open-loop execution," lacking the dynamic perception capability of the actual weld condition of the base material, making it difficult to meet the stringent requirements of high-quality consistency in modern manufacturing. The second approach is offline programming technology, which can significantly reduce the on-site teaching and repeated debugging process of the welding robot, thereby greatly shortening the on-site debugging cycle, reducing reliance on worker experience, and realizing welding posture planning and trajectory simulation verification through computer-aided kinematics and trajectory optimization algorithms. However, since it is essentially "CAD-based," it highly depends on the absolute alignment of the workpiece in physical and virtual space. When dealing with large, non-standard components, local deformation caused by manufacturing tolerances or random pose shifts due to hoisting often result in significant deviations from the "ideal trajectory" generated offline during actual operation. This "virtual-real mismatch" problem means that when the system encounters large components without precision tooling constraints, it still requires frequent manual intervention for secondary correction, failing to fundamentally eliminate the reliance on manual intervention. Therefore, while the two methods mentioned above can replace the objectives of the aforementioned embodiments under sufficient preconditions, they are not essentially perception-driven weld extraction methods and cannot improve system flexibility.

[0062] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a method for weld seam extraction and robot trajectory generation based on prior knowledge modeling.

[0063] Those skilled in the art will understand that Figure 9 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0064] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0065] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0066] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0067] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0068] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for weld seam extraction and robot trajectory generation based on prior knowledge modeling, characterized in that, include: Obtain downsampled point clouds and non-downsampled point clouds; The downsampled point cloud is the point cloud obtained by downsampling the coarsely registered point cloud; The unsampling point cloud refers to the point cloud that has not been downsampled from the coarse registration point cloud; The downsampled point cloud is precisely registered with the predefined digital model point cloud to obtain a fine registration matrix. Using the fine registration matrix, the unsampling point cloud is projected onto the spatial coordinate system of the digital model point cloud to obtain the projected point cloud; Using the predefined region of interest for the weld in the digital model point cloud, the projected point cloud is spatially clipped to obtain the weld region point cloud; Local geometric feature analysis is performed on the point cloud of the weld area to extract a discrete set of weld feature points; A continuous welding trajectory is generated based on the discrete set of weld feature points.

2. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling as described in claim 1, characterized in that, The acquisition of downsampled point clouds and non-downsampled point clouds specifically includes: Obtain the original captured point cloud; The original captured point cloud is coarsely registered with the predefined digital model point cloud to obtain a coarsely registered point cloud. The coarsely registered point cloud is filtered to remove duplicate points, resulting in a filtered point cloud. Save the filtered point cloud as the undownsampled point cloud; The filtered point cloud is then subjected to spatial voxel downsampling to obtain a downsampled point cloud. The downsampled point cloud is clustered to extract the largest cluster and remove discrete points, thus obtaining the downsampled point cloud.

3. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling according to claim 1 or 2, characterized in that, The downsampled point cloud is then finely registered with a predefined digital model point cloud to obtain a fine registration matrix, specifically including: The Trimming ICP algorithm is used to iteratively register the downsampled point cloud and the predefined digital model point cloud to obtain a fine registration matrix.

4. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling according to claim 1 or 2, characterized in that, Local geometric feature analysis is performed on the point cloud of the weld area to extract a discrete set of weld feature points, specifically including: A local coordinate basis is constructed based on the point cloud of the weld area using principal component analysis. On the local coordinate base, knee point detection is performed by comparing normal vectors to screen out candidate points that constitute the bevel sidewall; Clustering the candidate points yields a cluster of points on the bevel sidewall; Spatial layering and slicing are performed on the bevel sidewall point cluster to extract discrete weld feature point sets.

5. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling according to claim 4, characterized in that, Clustering the candidate points yields a cluster of points on the bevel sidewall, specifically including: The candidate points were clustered using the dual DBSCAN clustering method to obtain the bevel sidewall point cluster.

6. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling according to claim 1, characterized in that, Based on the discrete set of weld feature points, a continuous welding trajectory is generated, specifically including: The discrete weld feature point set is interpolated and smoothed using cubic B-splines to obtain a preliminary smooth trajectory; The initial smoothed trajectory is transformed into the robot base coordinate system using the fine registration matrix and the hand-eye calibration matrix to obtain the transformed trajectory; the hand-eye calibration matrix is ​​used to describe the coordinate transformation relationship between the 3D camera that captured the original point cloud and the robot that performs welding. The transformed trajectory is subjected to quadratic B-spline interpolation in the robot base coordinate system to obtain a continuous welding trajectory.

7. The method for weld seam extraction and robot trajectory generation based on prior knowledge modeling according to claim 1, characterized in that, The continuous welding trajectory includes spatial pose information and corresponding weld width information; the weld width information is used to adjust welding process parameters.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the weld seam extraction and robot trajectory generation method based on digital model prior guidance as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the weld seam extraction and robot trajectory generation method based on prior digital model guidance as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the weld seam extraction and robot trajectory generation method based on prior digital model guidance as described in any one of claims 1-7.

Images