Multi-view welding point cloud registration and splicing method and system based on deep learning
By employing a deep learning-based multi-view welding point cloud registration and stitching method, and utilizing global and local registration networks to obtain the transformation parameters of the welding point cloud, the problem of insufficient three-dimensional shape perception in robotic welding technology is solved, thereby improving the reliability and accuracy of welding.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HEFEI UNIV OF TECH
- Filing Date
- 2026-01-13
- Publication Date
- 2026-06-09
AI Technical Summary
Robotic welding technology lacks accurate perception and understanding of the three-dimensional morphology of the workpiece, resulting in low reliability and accuracy.
A deep learning-based multi-view welding point cloud registration and stitching method is adopted. Welding point clouds from different perspectives are acquired through a line structure laser camera. Rigidity and local transformation parameters are obtained by using a global registration network and a local registration network to achieve accurate registration and stitching of the point clouds.
It improves the reliability and accuracy of robotic welding technology, enabling efficient and accurate acquisition of the three-dimensional morphology of the welded workpiece.
Smart Images
Figure CN122176016A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of welding point cloud registration and splicing technology, specifically to a multi-view welding point cloud registration and splicing method and system based on deep learning. Background Technology
[0002] As a key process in manufacturing, welding is facing an increasingly urgent need for intelligent transformation.
[0003] In order to solve the problems of low efficiency, unstable quality and harsh working environment caused by traditional manual welding operations, robotic welding technology is usually used to replace manual labor with automated equipment. However, because robots lack the ability to accurately perceive and understand the three-dimensional shape of the workpiece, the reliability and accuracy of robotic welding technology are low. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a deep learning-based method for registering and stitching multi-view welding point clouds. This method can efficiently and accurately register and stitch multi-view welding point clouds, thereby accurately obtaining the three-dimensional morphology of the welded workpiece and improving the reliability and accuracy of robotic welding technology.
[0005] The technical solution adopted in this invention is as follows: A deep learning-based multi-view welding point cloud registration and stitching method includes the following steps: S1, using a line-structured laser camera to scan the target weldment from multiple perspectives to obtain welding point clouds from different perspectives; S2, obtaining the target point cloud corresponding to the target coordinate system; S3, using a global registration network to obtain corresponding rigid transformation parameters based on the welding point clouds and the target point clouds from different perspectives, and performing global registration on the corresponding welding point clouds based on the rigid transformation parameters to obtain the corresponding globally registered source point cloud; S4, using a local registration network to obtain corresponding local transformation parameters based on the globally registered source point cloud and the target point cloud from different perspectives, and performing local registration on the corresponding globally registered source point cloud based on the local transformation parameters to obtain the corresponding locally registered source point cloud; S5, based on the locally registered source point clouds from different perspectives, performing downsampling processing on the point clouds in the overlapping area in the target coordinate system.
[0006] In one embodiment of the present invention, step S3 specifically includes the following steps: S31, using a first encoder to extract first global features of the welding point cloud from different perspectives based on the PointNet point cloud network; S32, using a regressor to obtain corresponding prediction transformation parameters based on the first global features of the welding point cloud from different perspectives using a fully connected layer; S33, applying the prediction transformation parameters to the corresponding welding point cloud respectively, and using mean square error to measure the error after alignment, and updating the weight coefficients of the first encoder and the regressor based on the error through backpropagation; S34, using the updated first encoder to extract second global features of the welding point cloud from different perspectives based on the PointNet point cloud network; S35, using the updated regressor to obtain the corresponding rigid transformation parameters based on the second global features of the welding point cloud from different perspectives using a fully connected layer.
[0007] In one embodiment of the present invention, step S4 specifically includes the following steps: S41, using a second encoder to extract local multi-scale features of the globally registered source point cloud from different perspectives based on the PointNet++ point cloud network; S42, using a feature matching unit to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales through cross-scale module attention fusion features, and calculating the cross-attention between the globally registered source point cloud and the target point cloud for each level of features, and outputting the weighted features; S43, using a decoder to upsample the high-level features to the original resolution through deconvolution interpolation, and progressively regressing and predicting the displacement vector for each level of features, and fusing the predicted displacement vectors to generate local transformation parameters.
[0008] A deep learning-based multi-view welding point cloud registration and stitching system includes: a first acquisition module, which uses a line-structured laser camera to scan a target weldment from multiple perspectives to acquire welding point clouds from different perspectives; a second acquisition module, which acquires a target point cloud corresponding to a target coordinate system; a third acquisition module, which uses a global registration network to acquire corresponding rigid transformation parameters based on the welding point clouds and the target point clouds from different perspectives, and performs global registration on the corresponding welding point clouds based on the rigid transformation parameters to acquire corresponding globally registered source point clouds; a fourth acquisition module, which uses a local registration network to acquire corresponding local transformation parameters based on the globally registered source point clouds and the target point clouds from different perspectives, and performs local registration on the globally registered source point clouds based on the local transformation parameters to acquire corresponding locally registered source point clouds; and a processing module, which performs downsampling processing on the point clouds in overlapping areas in the target coordinate system based on the locally registered source point clouds from different perspectives.
[0009] In one embodiment of the present invention, the third acquisition module is specifically used to: extract first global features of the welding point cloud from different perspectives using a first encoder based on a PointNet point cloud network; obtain corresponding prediction transformation parameters based on the first global features of the welding point cloud from different perspectives using a regressor based on a fully connected layer; apply the prediction transformation parameters to the corresponding welding point cloud respectively, and use mean square error to measure the error after alignment, and update the weight coefficients of the first encoder and the regressor based on the error through backpropagation; extract second global features of the welding point cloud from different perspectives using the updated first encoder based on a PointNet point cloud network; and obtain corresponding rigid transformation parameters based on the second global features of the welding point cloud from different perspectives using the updated regressor based on a fully connected layer.
[0010] In one embodiment of the present invention, the fourth acquisition module is specifically used to: use a second encoder to extract local multi-scale features of the globally registered source point cloud from different perspectives based on the PointNet++ point cloud network; use a feature matching unit to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales through cross-scale module attention fusion features, and calculate the cross-attention between the globally registered source point cloud and the target point cloud for each level of features, and output the weighted features; use a decoder to upsample the high-level features to the original resolution through deconvolution interpolation, and progressively regress the predicted displacement vector for each level of features, and fuse the predicted displacement vector to generate local transformation parameters.
[0011] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described deep learning-based multi-view welding point cloud registration and splicing method.
[0012] A non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned deep learning-based multi-view welding point cloud registration and splicing method.
[0013] The beneficial effects of this invention are: This invention can efficiently and accurately register and stitch together welding point clouds from multiple perspectives, thereby accurately obtaining the three-dimensional morphology of the welded workpiece and improving the reliability and accuracy of robotic welding technology. Attached Figure Description
[0014] Figure 1 This is a flowchart of a deep learning-based multi-view welding point cloud registration and splicing method according to an embodiment of the present invention; Figure 2This is a schematic diagram of the network structure of a global registration network according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the network structure of a local registration network according to an embodiment of the present invention; Figure 4 This is a block diagram of a deep learning-based multi-view welding point cloud registration and stitching system, which is an embodiment of the present invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0016] Figure 1 This is a flowchart of a deep learning-based multi-view welding point cloud registration and splicing method according to an embodiment of the present invention.
[0017] like Figure 1 As shown, the deep learning-based multi-view welding point cloud registration and stitching method of this invention may include the following steps: S1 uses a line-structured laser camera to perform multi-view scanning of the target weldment to obtain welding point clouds from different perspectives.
[0018] Specifically, the target weldment can be fixed on the worktable, and the target weldment can be scanned from different angles using a line structure laser camera. Each scan process must ensure that there are repeated scan areas for subsequent registration and stitching to obtain the weldment point cloud from different angles. The position of the weldment must be kept fixed throughout the scanning process.
[0019] S2, obtain the target point cloud corresponding to the target coordinate system.
[0020] The target point cloud (i.e., the target point cloud) is the point cloud to which the welding point cloud (i.e., the source point cloud) needs to be aligned to the target coordinate system. The target point cloud can be represented as... .
[0021] S3 uses a global registration network to obtain the corresponding rigid transformation parameters based on the welding point cloud and target point cloud from different perspectives, and performs global registration on the corresponding welding point cloud based on the rigid transformation parameters to obtain the corresponding globally registered source point cloud.
[0022] Specifically, the network structure of a global registration network may include a first encoder and a regressor, such as... Figure 2As shown, the goal of the global registration network is to learn a mapping function f that maps the weld point cloud (i.e., the source point cloud P) and the target point cloud Q to rigid transformation parameters (R, t), so that the globally registered source point cloud and the target point cloud Q are aligned. The inputs to the global registration network are the source point cloud P ( ) and target point cloud Q ( The output is the rigid transformation parameters, namely the rigid rotation matrix R and the translation vector t.
[0023] Specifically, in one embodiment of the present invention, step S3 specifically includes the following steps: S31 uses a first encoder to extract the first global features of the welding point cloud from different perspectives based on the PointNet point cloud network.
[0024] Specifically, such as Figure 1 As shown, firstly, a first encoder can be used to extract the global feature vector of the source point cloud P based on the PointNet point cloud network. and the global feature vector of the target point cloud Q , specifically, That is, the source point cloud P is first processed by multilayer perceptrons (MPLs), and then by max pooling to output a global feature vector. ; That is, the target point cloud Q is first processed by multilayer perceptrons (MPLs), and then by max pooling to output a global feature vector. Among them, the global feature vector of the source point cloud P. and the global feature vector of the target point cloud Q Both are 1024-dimensional, representing the global feature vector of the source point cloud P. and the global feature vector of the target point cloud Q Perform feature merging to obtain joint features (i.e., the first global feature), that is joint features The dimension is 2048.
[0025] S32 uses a regressor based on a fully connected layer to obtain the corresponding prediction transformation parameters according to the first global features of the welding point cloud from different perspectives.
[0026] Furthermore, such as Figure 2 As shown, a regressor based on a fully connected (FC) layer is used to predict the transformation parameters. .
[0027] S33 applies the predicted transformation parameters to the corresponding welding point clouds, uses mean square error to measure the alignment error, and updates the weight coefficients of the first encoder and regressor based on the error through backpropagation.
[0028] Specifically, the predicted transformation parameters are applied to the corresponding welding point cloud P, and then the mean square error is used to measure the alignment error. , in, The nearest neighbor of the source point cloud in the target point cloud after transformation by the predicted transformation parameters.
[0029] Then, the weight coefficients of the first encoder and regressor are updated based on the error through backpropagation.
[0030] S34, using the updated first encoder to extract the second global features of the welding point cloud from different perspectives based on the PointNet point cloud network; S35 uses an updated regressor to obtain the corresponding rigid transformation parameters based on the second global features of the weld point cloud from different perspectives using a fully connected layer.
[0031] After updating the weight coefficients in the first encoder and regressor, the above S32 and S33 are repeated using the updated first encoder and regressor to output rigid transformation parameters. The output is a 7-dimensional vector, where the first 4 numbers are quaternions used to represent that the rotation can be converted into a rotation matrix R, and the last 3 numbers represent the translation vector t.
[0032] S4. A local registration network is used to obtain the corresponding local transformation parameters of the source point cloud and the target point cloud after global registration based on different viewpoints. Based on the local transformation parameters, the corresponding source point cloud after global registration is locally registered to obtain the corresponding locally registered source point cloud.
[0033] Specifically, the design of the local registration network is as follows: Figure 3 As shown, the goal is to learn a local transformation parameter, namely the displacement field D, to map the globally registered source point cloud to a displacement field, thereby aligning the globally registered source point cloud with the target point cloud. The input to this local registration network is the globally registered source point cloud, and the output is a point-by-point displacement vector d. i , i.e., displacement D, where the network structure of the local registration network may include: a second encoder, a feature matching unit and a decoder.
[0034] Specifically, in one embodiment of the present invention, step S4 specifically includes the following steps: S41 uses a second encoder to extract local multi-scale features of the source point cloud after global registration from different perspectives based on the PointNet++ point cloud network.
[0035] Specifically, such as Figure 3As shown, a second encoder is used to extract local multi-scale features of the globally registered source and target point clouds in a hierarchical manner based on the PointNet++ point cloud network. The input consists of the globally registered source and target point clouds, which can be processed in three levels. The first level extracts 1024 points from the original point cloud (i.e., the input point cloud) with a feature dimension of 64 for each point. The second level downsamples the point cloud to 512 points with a feature dimension of 128 for each point. The third level further downsamples to 256 points with a feature dimension of 256 for each point.
[0036] S42 employs a feature matching unit to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales through cross-scale module attention fusion features. It also calculates the cross-attention between the globally registered source point cloud and the target point cloud for each level of features and outputs the weighted features.
[0037] Furthermore, a feature matching unit is employed to fuse features through cross-scale module attention, capturing the local correspondence between the globally registered source and target point clouds at multiple scales, thus enhancing local matching capabilities. For each level of features, the cross-attention between the transformed source and target point clouds is calculated, and the weighted features are output. Specifically, such as... Figure 3 As shown, the output consists of the first layer weighted feature F1, the second layer weighted feature F2, and the third layer weighted feature F3. Cross-attention can be calculated using the following formula: , in, This represents the feature matrix of the source point cloud after global registration. The feature matrix representing the target point cloud, Indicates the feature dimension.
[0038] S43 employs a decoder to upsample high-level features to the original resolution through deconvolution interpolation, and then progressively regresses and predicts displacement vectors for each level of features, and fuses the predicted displacement vectors to generate local transformation parameters.
[0039] Furthermore, a decoder upsamples high-level features to the original resolution using deconvolution interpolation, predicting the displacement vector point by point. Specifically, inverse distance-weighted interpolation is used to pass high-level features to lower levels, i.e. , in, Represents the features of the k-th layer. This represents the interpolation index and weight parameters corresponding to the k-th layer. This represents the features after upsampling at the k-th layer. This indicates that the search is based on KNN and uses a weighted average of the features of neighboring points.
[0040] Each level of feature is progressively regressed to predict the displacement vector, i.e.
[0041] in, Represents the predicted displacement vector. Represents the regression parameters (weights and biases) of the learning process. This represents a one-dimensional convolution operation.
[0042] Finally, the multi-scale displacements are fused, that is... , in, This represents the local transformation parameter (i.e., displacement D). This represents the learnable weights.
[0043] It should be noted that the local registration network can be constrained using an improved Chamfer Distance, i.e., the loss function is: , in, This represents the source point cloud after global registration.
[0044] S5, based on the source point cloud after local registration corresponding to different viewpoints, performs downsampling processing on the point cloud of the overlapping area in the target coordinate system.
[0045] Specifically, after aligning point clouds from multiple perspectives to the same target coordinate system, directly using the merged point cloud results in redundancy in overlapping areas and a surge in data volume. Therefore, it is necessary to process duplicate points in overlapping areas and then perform optimization operations (smoothing, denoising, hole filling, etc.). Correspondingly, point cloud density can be equalized through downsampling, isolated noise points can be removed using radius filtering, and smooth surfaces can be generated based on the point cloud normal vectors to balance detail preservation and computational efficiency.
[0046] In summary, the deep learning-based multi-view welding point cloud registration and stitching method according to embodiments of the present invention employs a line-structured laser camera to scan the target weldment from multiple perspectives to obtain welding point clouds from different viewpoints and target point clouds corresponding to the target coordinate system. A global registration network is used to obtain corresponding rigid transformation parameters based on the welding point clouds and target point clouds from different viewpoints, and global registration is performed on the corresponding welding point clouds based on the rigid transformation parameters to obtain globally registered source point clouds. Similarly, a local registration network is used to obtain corresponding local transformation parameters based on the globally registered source point clouds and target point clouds from different viewpoints, and local registration is performed on the globally registered source point clouds based on the local transformation parameters to obtain locally registered source point clouds. Finally, based on the locally registered source point clouds from different viewpoints, downsampling processing is performed on the point clouds in overlapping areas within the target coordinate system. Therefore, multi-view welding point cloud registration and stitching can be performed efficiently and accurately, thereby accurately obtaining the three-dimensional morphology of the welded workpiece and improving the reliability and accuracy of robotic welding technology.
[0047] Corresponding to the deep learning-based multi-view welding point cloud registration and stitching method in the above embodiments, the present invention also proposes a deep learning-based multi-view welding point cloud registration and stitching system.
[0048] like Figure 4 As shown, the deep learning-based multi-view welding point cloud registration and stitching system of the present invention may include: a first acquisition module 100, a second acquisition module 200, a third acquisition module 300, a fourth acquisition module 400, and a processing module 500.
[0049] The system comprises the following modules: a first acquisition module 100, which uses a line-structured laser camera to perform multi-view scanning of the target weldment to obtain welding point clouds from different perspectives; a second acquisition module 200, which acquires the target point cloud corresponding to the target coordinate system; a third acquisition module 300, which uses a global registration network to acquire corresponding rigid transformation parameters based on the welding point clouds and target point clouds from different perspectives, and performs global registration on the corresponding welding point clouds based on the rigid transformation parameters to obtain the corresponding globally registered source point cloud; a fourth acquisition module 400, which uses a local registration network to acquire corresponding local transformation parameters based on the globally registered source point cloud and target point cloud from different perspectives, and performs local registration on the globally registered source point cloud based on the local transformation parameters to obtain the corresponding locally registered source point cloud; and a processing module 500, which performs downsampling processing on the point cloud of the overlapping region in the target coordinate system based on the locally registered source point cloud from different perspectives.
[0050] In one embodiment of the present invention, the third acquisition module 300 is specifically configured to: use a first encoder to extract first global features of welding point clouds from different perspectives based on a PointNet point cloud network; use a regressor to obtain corresponding prediction transformation parameters based on the first global features of welding point clouds from different perspectives based on a fully connected layer; apply the prediction transformation parameters to the corresponding welding point clouds respectively, and use mean square error to measure the error after alignment, and update the weight coefficients of the first encoder and the regressor based on the error through backpropagation; use the updated first encoder to extract second global features of welding point clouds from different perspectives based on a PointNet point cloud network; and use the updated regressor to obtain corresponding rigid transformation parameters based on the second global features of welding point clouds from different perspectives based on a fully connected layer.
[0051] In one embodiment of the present invention, the fourth acquisition module 400 is specifically used to: use a second encoder to extract local multi-scale features of the globally registered source point cloud from different perspectives based on the PointNet++ point cloud network; use a feature matching unit to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales through cross-scale module attention fusion features, and calculate the cross-attention between the globally registered source point cloud and the target point cloud for each level of features, and output the weighted features; use a decoder to upsample the high-level features to the original resolution through deconvolution interpolation, and progressively regress the predicted displacement vector for each level of features, and fuse the predicted displacement vector to generate local transformation parameters.
[0052] It should be noted that for details not disclosed in the deep learning-based multi-view welding point cloud registration and stitching system of the present invention embodiments, please refer to the details disclosed in the above-described deep learning-based multi-view welding point cloud registration and stitching method, which will not be elaborated here.
[0053] According to an embodiment of the present invention, a deep learning-based multi-view welding point cloud registration and stitching system utilizes a first acquisition module to scan the target weldment from multiple perspectives using a line-structured laser camera to acquire welding point clouds from different viewpoints. A second acquisition module acquires the target point cloud corresponding to the target coordinate system. A third acquisition module uses a global registration network to acquire corresponding rigid transformation parameters based on the welding point clouds and target point clouds from different viewpoints, and performs global registration on the corresponding welding point clouds based on the rigid transformation parameters to acquire globally registered source point clouds. A fourth acquisition module uses a local registration network to acquire corresponding local transformation parameters based on the globally registered source point clouds and target point clouds from different viewpoints, and performs local registration on the globally registered source point clouds based on the local transformation parameters to acquire locally registered source point clouds. A processing module downsamples the point clouds in overlapping areas within the target coordinate system based on the locally registered source point clouds from different viewpoints. This allows for efficient and accurate registration and stitching of multi-view welding point clouds, thereby accurately determining the three-dimensional morphology of the welded workpiece and improving the reliability and accuracy of robotic welding technology.
[0054] Corresponding to the above embodiments, the present invention also proposes a computer device.
[0055] The computer device of this invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the deep learning-based multi-view welding point cloud registration and splicing method of the above embodiments.
[0056] The computer device according to embodiments of the present invention can efficiently and accurately register and stitch together multi-view welding point clouds, thereby accurately obtaining the three-dimensional morphology of the welded workpiece, and thus improving the reliability and accuracy of robotic welding technology.
[0057] Corresponding to the above embodiments, the present invention also proposes a non-transitory computer-readable storage medium.
[0058] The non-transitory computer-readable storage medium of this invention stores a computer program that, when executed by a processor, implements the above-described deep learning-based multi-view welding point cloud registration and splicing method.
[0059] According to embodiments of the present invention, a non-transitory computer-readable storage medium can efficiently and accurately register and stitch together multi-view welding point clouds, thereby accurately obtaining the three-dimensional morphology of the welded workpiece, and thus improving the reliability and accuracy of robotic welding technology.
[0060] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.
[0061] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0062] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.
[0063] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.
[0064] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.
[0065] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for multi-view welding point cloud registration and stitching based on deep learning, characterized in that, Includes the following steps: S1 uses a line-structured laser camera to perform multi-view scanning of the target weldment to obtain welding point clouds from different perspectives; S2, obtain the target point cloud corresponding to the target coordinate system; S3, a global registration network is used to obtain the corresponding rigid transformation parameters of the welding point cloud and the target point cloud from different perspectives, and the corresponding welding point cloud is globally registered based on the rigid transformation parameters to obtain the corresponding globally registered source point cloud. S4, using a local registration network, obtains corresponding local transformation parameters for the source point cloud and the target point cloud after global registration based on different viewpoints, and performs local registration on the corresponding source point cloud after global registration based on the local transformation parameters to obtain the corresponding locally registered source point cloud. S5, based on the source point cloud after local registration corresponding to different viewpoints, the point cloud of the overlapping area is downsampled in the target coordinate system.
2. The method for multi-view welding point cloud registration and stitching based on deep learning according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31, the first encoder is used to extract the first global features of the welding point cloud from different perspectives based on the PointNet point cloud network; S32, a regressor is used to obtain the corresponding prediction transformation parameters based on the first global features of the welding point cloud from different perspectives using a fully connected layer; S33, apply the predicted transformation parameters to the corresponding welding point clouds respectively, use mean square error to measure the error after alignment, and update the weight coefficients of the first encoder and the regressor according to the error through backpropagation; S34, using the updated first encoder to extract the second global features of the welding point cloud from different perspectives based on the PointNet point cloud network; S35, using an updated regressor based on a fully connected layer, the corresponding rigid transformation parameters are obtained according to the second global features of the weld point cloud from different perspectives.
3. The method for multi-view welding point cloud registration and stitching based on deep learning according to claim 2, characterized in that, Step S4 specifically includes the following steps: S41, using a second encoder based on the PointNet++ point cloud network to extract local multi-scale features of the source point cloud after global registration from different perspectives; S42, the feature matching unit uses cross-scale module attention fusion features to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales, and calculates the cross attention between the globally registered source point cloud and the target point cloud for each level of features, and outputs the weighted features; S43 employs a decoder to upsample high-level features to the original resolution through deconvolution interpolation, and then progressively regresses and predicts displacement vectors for each level of features, and fuses the predicted displacement vectors to generate local transformation parameters.
4. A multi-view welding point cloud registration and stitching system based on deep learning, characterized in that, include: The first acquisition module is used to perform multi-view scanning of the target weldment using a line structure laser camera to obtain welding point clouds from different viewpoints. The second acquisition module is used to acquire the target point cloud corresponding to the target coordinate system. The third acquisition module is used to acquire corresponding rigid transformation parameters based on the welding point cloud and the target point cloud from different perspectives using a global registration network, and to perform global registration on the corresponding welding point cloud based on the rigid transformation parameters to obtain the corresponding globally registered source point cloud. The fourth acquisition module is used to acquire corresponding local transformation parameters for the source point cloud and the target point cloud after global registration based on different viewpoints using a local registration network, and to perform local registration on the corresponding source point cloud after global registration based on the local transformation parameters to obtain the corresponding locally registered source point cloud. The processing module is used to downsample the point cloud of the overlapping region in the target coordinate system based on the source point cloud after local registration corresponding to different viewpoints.
5. The deep learning-based multi-view welding point cloud registration and stitching system according to claim 4, characterized in that, The third acquisition module is specifically used for: The first global features of the welding point cloud from different perspectives are extracted using a first encoder based on the PointNet point cloud network. A regressor is used to obtain the corresponding prediction transformation parameters based on the first global features of the welding point cloud from different perspectives using a fully connected layer; The predicted transformation parameters are applied to the corresponding welding point clouds, and the mean square error is used to measure the error after alignment. The weight coefficients of the first encoder and the regressor are updated according to the error through backpropagation. The updated first encoder is used to extract the second global features of the welding point cloud from different perspectives based on the PointNet point cloud network; The updated regressor is used to obtain the corresponding rigid transformation parameters based on the second global features of the weld point cloud from different perspectives using a fully connected layer.
6. The deep learning-based multi-view welding point cloud registration and stitching system according to claim 5, characterized in that, The fourth acquisition module is specifically used for: A second encoder is used to extract local multi-scale features of the source point cloud after global registration from different perspectives based on the PointNet++ point cloud network in a hierarchical manner. The feature matching unit uses cross-scale module attention fusion features to capture the local correspondence between the globally registered source point cloud and the target point cloud at multiple scales. It also calculates the cross attention between the globally registered source point cloud and the target point cloud for each level of features and outputs the weighted features. The decoder upsamples high-level features to the original resolution through deconvolution interpolation, and then progressively regresses and predicts displacement vectors for each level of features. Finally, the predicted displacement vectors are fused to generate local transformation parameters.
7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the deep learning-based multi-view welding point cloud registration and splicing method according to any one of claims 1-3.
8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the deep learning-based multi-view welding point cloud registration and splicing method according to any one of claims 1-3.