Three-dimensional shape stitching method, device and equipment based on non-overlapping point cloud data

Abstract

Embodiments of the present application provide a three-dimensional shape splicing method, device and equipment based on non-overlapping point cloud data, the method comprising: obtaining first point cloud data of a first target object and second point cloud data of a second target object; there is no overlap between the first point cloud data and the second point cloud data; inputting the first point cloud data and the second point cloud data into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix represents pose information of point cloud data splicing; the target neural network is trained based on point cloud data samples to be spliced; and splicing the first point cloud data and the second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain a target spliced object. The method of embodiments of the present application improves the splicing efficiency and accuracy of point cloud data.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus and device for stitching three-dimensional shapes based on non-overlapping point cloud data. Background Technology

[0002] Based on the simplicity, effectiveness, and flexibility of 3D shape (point cloud model) technology, it has become a widely used form of discrete representation of 3D data in the field of computer vision. With the development of 3D scanning equipment and computer vision reconstruction technology, obtaining detailed 3D shape (point cloud data) has become increasingly easier. Therefore, the importance of 3D shape analysis (such as shape matching, segmentation, correspondence, and retrieval) has significantly increased. Designing an algorithm and system for assembling the surface shapes of multiple objects in cultural relic restoration, manufacturing, and reverse engineering is an important problem in computer vision and a key foundation for other advanced applications.

[0003] In related technologies, stitching is performed using methods based on manually designed descriptors. These methods primarily involve modeling points in a point cloud and their geometric features, such as normals, density, directed distance fields, and corresponding features at neighboring points. This results in a descriptor describing the local or global geometric features of a portion of the point cloud, which is then used to stitch together multiple point clouds. Clearly, these methods are based on spatial domain features and are easily affected by data acquisition methods, resolution, shape transformations, and deformations, leading to poor stitching results. Summary of the Invention

[0004] To address the problems in the prior art, embodiments of the present invention provide a method, apparatus, and device for three-dimensional shape stitching based on non-overlapping point cloud data. Specifically, embodiments of the present invention provide the following technical solutions:

[0005] In a first aspect, embodiments of the present invention provide a method for stitching three-dimensional shapes based on non-overlapping point cloud data, including:

[0006] Acquire first point cloud data of a first target object and second point cloud data of a second target object; there is no overlap between the first point cloud data and the second point cloud data.

[0007] The first point cloud data and the second point cloud data are input into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data splicing; the target neural network is trained based on the point cloud data samples to be spliced.

[0008] Based on the target transformation matrix and the boundary point information of the point cloud data, the first point cloud data and the second point cloud data are stitched together to obtain the target stitching object.

[0009] Furthermore, the target neural network includes:

[0010] A first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; the first encoder, the second encoder, and the attitude decoder are connected; the first encoder, the second encoder, and the target boundary point classification decoder are connected; the first encoder and the second encoder have the same structure;

[0011] The first encoder is used to extract global shape features and local point-by-point features from the first point cloud data;

[0012] The second encoder is used to extract global shape features and local point-by-point features from the second point cloud data;

[0013] The attitude decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data;

[0014] The target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data.

[0015] Further, the pose decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data, including:

[0016] The attitude decoder is used to connect the global shape features of the first point cloud data and the global shape features of the second point cloud data to obtain the target global shape features;

[0017] Determine the pose transformation Lie algebra based on the global shape features of the target;

[0018] The target transformation matrix is ​​determined based on the attitude transformation Lie algebra; the target transformation matrix includes an attitude rotation matrix and an attitude displacement vector.

[0019] Further, the target boundary point classification decoder includes a first boundary point classification decoder and a second boundary point classification decoder; the target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data, including:

[0020] The first boundary point classification decoder in the target boundary point classification decoder is used to perform max pooling on the local point-by-point features of the second point cloud data, and then obtain the high-level feature tensor of the second point cloud data through repeated element terms.

[0021] After concatenating the high-level feature tensor of the second point cloud data and the local point-by-point features of the first point cloud data, the data is decoded and classified to obtain the boundary point information in the first point cloud data.

[0022] The second boundary point classification decoder in the target boundary point classification decoder is used to perform max pooling on the local point-by-point features of the first point cloud data, and then obtain the high-level feature tensor of the first point cloud data through repeated element terms.

[0023] After concatenating the high-level feature tensor of the first point cloud data and the local point-by-point features of the second point cloud data, the boundary point information in the second point cloud data is obtained by decoding and classification.

[0024] Furthermore, the 3D shape stitching method based on non-overlapping point cloud data also includes:

[0025] If the number of target objects to be concatenated in the target set is N, and N is greater than 2, then...

[0026] Step a: Determine the first target object from the target set;

[0027] Step b: Remove the first target object from the target set to obtain a candidate splicing set;

[0028] Step c: Determine the target transformation matrix and point cloud boundary point information corresponding to each target object in the candidate splicing set and the first target object, respectively;

[0029] Based on the target transformation matrix and point cloud boundary point information corresponding to each target object in the candidate stitching set and the first target object, respectively, the target distance between each target object in the candidate stitching set and the first target object is obtained using a preset distance calculation method.

[0030] Determine the minimum distance among all the target distances; and designate the target object corresponding to the minimum distance as the second target object;

[0031] Step d: Combine the first point cloud data of the first target object and the second point cloud data of the second target object to obtain the target composite object;

[0032] Step e: Remove the second target object from the candidate splicing set, add the target splicing object to the candidate splicing set, and obtain an updated candidate splicing set; use the updated candidate splicing set as the target set;

[0033] Repeat step ae until the number of target objects to be spliced ​​in the candidate splicing set is 0.

[0034] Furthermore, the attitude decoder is trained based on a first target hybrid loss function; the first target hybrid loss function is determined based on a homogeneous second-order difference loss function, a bidirectional nearest neighbor distance loss function, and an invertible minimum distance loss function.

[0035] The target boundary point classification decoder is trained based on a second target hybrid loss function; the second target hybrid loss function is determined based on the cross-entropy loss function and the bidirectional nearest neighbor distance loss function.

[0036] Secondly, embodiments of the present invention also provide a three-dimensional shape stitching device based on non-overlapping point cloud data, comprising:

[0037] The acquisition module is used to acquire first point cloud data of a first target object and second point cloud data of a second target object; there is no overlap between the first point cloud data and the second point cloud data.

[0038] The processing module is used to input the first point cloud data and the second point cloud data into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data splicing; the target neural network is trained based on the point cloud data samples to be spliced.

[0039] The stitching module is used to stitch the first point cloud data and the second point cloud data together based on the target transformation matrix and the boundary point information of the point cloud data to obtain the target stitching object.

[0040] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the three-dimensional shape stitching method based on non-overlapping point cloud data as described in the first aspect.

[0041] Fourthly, embodiments of the present invention also provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the three-dimensional shape stitching method based on non-overlapping point cloud data as described in the first aspect.

[0042] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the three-dimensional shape stitching method based on non-overlapping point cloud data as described in the first aspect.

[0043] The three-dimensional shape stitching method, apparatus, and device based on non-overlapping point cloud data provided in this invention, after inputting first and second point cloud data into a pre-trained target neural network to obtain the target transformation matrix and boundary point information of the point cloud data, can determine the rotation direction, rotation angle, displacement, and boundary points of the second point cloud data of the second target object, based on the target transformation matrix and boundary point information of the point cloud data. This allows for accurate stitching of the first and second point cloud data. In other words, the target neural network predicts the transformation matrix (rotation and displacement) for stitching the two point clouds and the subset of boundary points of the two point clouds. By utilizing the point cloud stitching transformation matrix and boundary information, the stitching of the point clouds can be achieved, thus avoiding the influence of data acquisition methods, resolution, and shape transformation on the stitching effect and improving the efficiency and accuracy of point cloud data stitching. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating the three-dimensional shape stitching method based on non-overlapping point cloud data provided in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of the target neural network structure provided in an embodiment of the present invention;

[0047] Figure 3 This is one of the schematic diagrams of a three-dimensional shape stitching method based on non-overlapping point cloud data provided in an embodiment of the present invention;

[0048] Figure 4 This is a second schematic diagram of the three-dimensional shape stitching method based on non-overlapping point cloud data provided in the embodiments of the present invention;

[0049] Figure 5 This is the third schematic diagram of the three-dimensional shape stitching method based on non-overlapping point cloud data provided in the embodiments of the present invention;

[0050] Figure 6 This is the fourth schematic diagram of the three-dimensional shape stitching method based on non-overlapping point cloud data provided in the embodiments of the present invention;

[0051] Figure 7 This is one of the schematic diagrams of a method for stitching together multiple point cloud data provided in an embodiment of the present invention;

[0052] Figure 8 This is a second schematic diagram of the method for stitching together multiple point cloud data provided in the embodiments of the present invention;

[0053] Figure 9 This is the third schematic diagram of the method for stitching together multiple point cloud data provided in the embodiments of the present invention;

[0054] Figure 10 This is a schematic diagram of the structure of the three-dimensional shape stitching device based on non-overlapping point cloud data provided in an embodiment of the present invention;

[0055] Figure 11 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] The method of this invention can be applied to computer vision technology scenarios, improving the efficiency and accuracy of point cloud data stitching.

[0058] In related technologies, stitching is performed using methods based on manually designed descriptors. These methods primarily involve modeling points in a point cloud and their geometric features, such as normals, density, directed distance fields, and corresponding features at neighboring points. This results in a descriptor describing the local or global geometric features of a portion of the point cloud, which is then used to stitch together multiple point clouds. Clearly, these methods are based on spatial domain features and are easily affected by data acquisition methods, resolution, shape transformations, and deformations, leading to poor stitching results.

[0059] The 3D shape stitching method based on non-overlapping point cloud data in this invention, after inputting the first and second point cloud data into a pre-trained target neural network to obtain the target transformation matrix and the boundary point information of the point cloud data, can determine the rotation direction, rotation angle, displacement, and boundary points of the second point cloud data of the second target object, based on the target transformation matrix and the boundary point information of the point cloud data. This allows for accurate stitching of the first and second point cloud data. In other words, the target neural network predicts the transformation matrix (rotation and displacement) for stitching the two point clouds and the subset of boundary points of the two point clouds. By utilizing the point cloud stitching transformation matrix and boundary information, the stitching of the point clouds can be achieved, thus avoiding the influence of data acquisition methods, resolution, and shape transformation on the stitching effect and improving the efficiency and accuracy of point cloud data stitching.

[0060] The following is combined Figures 1-11 The technical solution of the present invention will be described in detail with reference to specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0061] Figure 1 This is a flowchart illustrating an embodiment of a three-dimensional shape stitching method based on non-overlapping point cloud data provided by this invention. Figure 1 As shown, the method provided in this embodiment includes:

[0062] Step 101: Obtain the first point cloud data of the first target object and the second point cloud data of the second target object; there is no overlap between the first point cloud data and the second point cloud data;

[0063] Specifically, in the relevant technologies, the focus in object point cloud assembly tasks is on assembling based on semantic information. This mainly includes methods based on manually designed descriptors and data-driven methods based on deep learning. These methods are easily affected by data acquisition methods, resolution, shape transformation, and deformation, resulting in poor stitching results.

[0064] To address the aforementioned issues, in this embodiment of the invention, first point cloud data of a first target object and second point cloud data of a second target object are acquired; wherein, the first target object and the second target object are objects to be stitched together; optionally, the first target object and the second target object are two adjacent but non-overlapping shapes that need to be stitched together after the same object has been segmented; optionally, after sampling the first target object and the second target object, the first point cloud data of the first target object and the second point cloud data of the second target object can be obtained.

[0065] For example, after sampling the first target object and the second target object, the length of the first point cloud data of the first target object and the second point cloud data of the second target object is 1024. The point cloud data only contains xyz axis coordinate information. Optionally, the second point cloud data needs to be aligned with the first point cloud data, that is, the first target object remains stationary and the second target object is spliced ​​with the first target object.

[0066] Step 102: Input the first point cloud data and the second point cloud data into the pre-trained target neural network to obtain the target transformation matrix and the boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data splicing; the target neural network is trained based on the point cloud data samples to be spliced.

[0067] Specifically, after acquiring the first point cloud data of the first target object and the second point cloud data of the second target object, in this embodiment of the invention, the first point cloud data and the second point cloud data are input into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data. The target transformation matrix represents the attitude information of the point cloud data splicing. The target transformation matrix includes an attitude rotation matrix and an attitude displacement vector. That is, based on the target transformation matrix, the direction, rotation angle, and displacement of the second point cloud data of the second target object to be rotated can be determined. Optionally, the points in the point cloud can be divided into boundary points and interior points. The boundary point information of the point cloud data can be used to indicate which points in the first point cloud data are on the boundary between the first and second target objects, and which points in the second point cloud data are on the boundary between the first and second target objects. That is, the boundary point information of the point cloud data can include boundary points from two point cloud data sets to be spliced. Therefore, after determining the attitude rotation matrix, attitude displacement vector, and boundary points in the point cloud data, the splicing of the first and second point cloud data sets to be spliced ​​can be accurately performed based on the attitude rotation matrix, attitude displacement vector, and the boundaries in the point cloud data.

[0068] Optionally, the target neural network is trained based on the point cloud data samples to be stitched and the label information of the point cloud data; optionally, the point cloud data samples may include first point cloud data samples of the first target object and second point cloud data samples of the second target object to be stitched; the label information of the point cloud data includes the real target transformation matrix between the first point cloud data samples and the second point cloud data samples and the boundary point information of the point cloud data; that is, by inputting the first point cloud data samples and the second point cloud data samples to be stitched into the initial target neural network, the target transformation matrix predicted by the target neural network and the boundary point information of the point cloud data are obtained. Then, by using the real target transformation matrix and the real boundary point information of the point cloud data in the label information and the target transformation matrix and the boundary point information of the point cloud data predicted by the target neural network, the target neural network is trained under supervision, and the trained target neural network can be obtained, so that the trained target neural network can accurately output the target transformation matrix and the boundary point information.

[0069] Step 103: Based on the target transformation matrix and the boundary point information of the point cloud data, stitch the first point cloud data and the second point cloud data together to obtain the target stitching object.

[0070] Specifically, after inputting the first and second point cloud data into a pre-trained target neural network to obtain the target transformation matrix and the boundary point information of the point cloud data, the direction of rotation, rotation angle, displacement, and boundary points of the second point cloud data of the second target object can be determined based on the target transformation matrix and the boundary point information of the point cloud data. Thus, the stitching of the first and second point cloud data can be performed accurately. In other words, the target neural network predicts the transformation matrix (rotation and displacement) for stitching the two point clouds and the subset of boundary points of the two point clouds. Then, the stitching of the point clouds can be achieved by using the point cloud stitching transformation matrix and boundary information. This avoids the influence of data acquisition method, resolution, and shape transformation on the stitching effect, and improves the stitching efficiency and accuracy of point cloud data.

[0071] The method described in the above embodiment, by inputting the first point cloud data and the second point cloud data into a pre-trained target neural network, obtains the target transformation matrix and the boundary point information of the point cloud data. This allows the determination of the rotation direction, rotation angle, displacement, and boundary points in the two point cloud data to be stitched together for the second target object. Consequently, the stitching of the first point cloud data and the second point cloud data can be performed accurately, avoiding the influence of data acquisition methods, resolution, and shape transformation on the stitching effect and improving the efficiency and accuracy of point cloud data stitching.

[0072] In one embodiment, the target neural network includes:

[0073] A first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; the first encoder, the second encoder, and the attitude decoder are connected; the first encoder, the second encoder, and the target boundary point classification decoder are connected; the first encoder and the second encoder have the same structure;

[0074] The first encoder is used to extract global shape features and local point-by-point features from the first point cloud data;

[0075] The second encoder is used to extract global shape features and local point-by-point features from the second point cloud data;

[0076] The attitude decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data;

[0077] The target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data.

[0078] Specifically, in this embodiment of the invention, the target neural network is used to obtain the target transformation matrix and boundary point information of the point cloud data based on the point cloud data of the first target object and the second point cloud data of the second target object. Optionally, the target neural network in this embodiment of the invention includes a first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; the first encoder, the second encoder, and the attitude decoder are connected; the first encoder, the second encoder, and the target boundary point classification decoder are connected.

[0079] For example, Figure 2 Here is a schematic diagram of the target neural network structure:

[0080] Optionally, the first encoder is used to extract global shape features and local point-by-point features of the first point cloud data; the second encoder is used to extract global shape features and local point-by-point features of the second point cloud data; optionally, the first encoder and the second encoder have the same structure; optionally, the global shape features of the point cloud data include the overall contour and shape features of the target object corresponding to the point cloud data, that is, the global shape features of the point cloud data are used to describe the global shape of the point cloud of the first target object; optionally, the global shape features of the point cloud data are tensors of shape 1×1024, and the global shape features of the point cloud data can be used to determine the target transformation matrix for aligning two input point clouds. Optionally, the local point-by-point features of the point cloud data include the semantic information and boundary information of each point in the point cloud data, and the local features are used to extract a subset of boundary points from two point clouds; optionally, the local features of the point cloud data are tensors of shape 1024×64.

[0081] For example, such as Figure 2As shown, the first encoder and the second encoder extract local point-by-point features of 1024×64 and global shape features of 1×1024 from the first point cloud data and the second point cloud, respectively. The encoder first inputs the input point cloud into two linear layers to obtain local point-by-point features. In order to obtain global features, the features are then processed sequentially through a sampling module, an aggregation module, four attention modules, and a final linear layer to transform the shape. Finally, the global shape features are obtained through a max pooling operation.

[0082] Optionally, the attitude decoder uses the global shape features of the two point clouds to be stitched, output by the first encoder and the second encoder, as input, and outputs a target transformation matrix. That is, by combining the global shape features of the two point clouds and inputting them into the attitude decoder, attitude information described by Lie algebra is obtained, describing how to stitch the two point clouds. Optionally, the target transformation matrix represents the attitude information of the point cloud data stitching. The target transformation matrix includes an attitude rotation matrix and an attitude displacement vector. Based on the target transformation matrix, the direction, rotation angle, and displacement of the second point cloud data of the second target object can be determined.

[0083] Optionally, the attitude decoder connects the global shape features of the first point cloud data and the global shape features of the second point cloud data to obtain the target global shape features; determines the attitude transformation Lie algebra based on the target global shape features; determines the target transformation matrix based on the attitude transformation Lie algebra; the target transformation matrix includes the attitude rotation matrix and the attitude displacement vector.

[0084] For example, such as Figure 2 As shown, in this embodiment of the invention, the target transformation matrix is ​​determined by an attitude decoder. The attitude decoder first concatenates two point cloud vectors of length 1×1024 (global shape features) and then inputs them into five different linear layers to obtain a 1×6 Lie algebra, thereby obtaining the target transformation matrix. This achieves the effect of accurately determining the target transformation matrix based on the global shape features of point cloud data.

[0085] Optionally, unlike traditional point cloud registration tasks, in point cloud assembly tasks, especially in cases of complete non-overlapping points, it is difficult to calculate the relationship between point pairs and thus estimate the transformation matrix by finding overlapping points or domains. Therefore, the approach in this embodiment of the invention is to link the global shape features of two point clouds and then perform a process such as... Figure 2 The several fully connected layers with the parameters shown serve as attitude estimation decoders. The decoder outputs a Lie algebra of length 6, representing the rigid transformation matrix T. The Lie algebra to the transformation matrix does not affect the backpropagation process of the network.

[0086] Optionally, the target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the second point cloud data. That is, the target boundary point classification decoder uses the local point-by-point features of the two point clouds to be stitched, output by the first encoder and the second encoder, as input, and outputs the boundary point information of the point cloud data. Optionally, the number of boundary points can be defined as 128 by defining the number of boundary point clouds. For a point cloud pair, the 128 boundary points are obtained by taking the 128 nearest points from point clouds A and B respectively.

[0087] Optionally, the target boundary point classification decoder includes a first boundary point classification decoder and a second boundary point classification decoder. The first boundary point classification decoder performs max pooling on the local point-by-point features of the second point cloud data, then obtains a high-level feature tensor of the second point cloud data after passing through repeated element terms. The high-level feature tensor of the second point cloud data and the local point-by-point features of the first point cloud data are concatenated and then decoded and classified to obtain the boundary point information in the first point cloud data. The second boundary point classification decoder performs max pooling on the local point-by-point features of the first point cloud data, then obtains a high-level feature tensor of the first point cloud data after passing through repeated element terms. The high-level feature tensor of the first point cloud data and the local point-by-point features of the second point cloud data are concatenated and then decoded and classified to obtain the boundary point information in the second point cloud data. Optionally, the first boundary point classification decoder and the second boundary point classification decoder are two independent decoders with the same structure.

[0088] For example, such as Figure 2As shown, the second boundary point classifier decoder performs max pooling on the local point-by-point features of the first point cloud data, and then passes the data through repeated elements to form a tensor of shape 1024×64. This tensor is used as the high-level feature tensor of the first point cloud data. The high-level feature tensor of the first point cloud data is then concatenated with the local point-by-point features of the second point cloud data, and input into multiple linear layers of gradually decreasing size and a softmax function to obtain the point-by-point classification result of the second point cloud data. Optionally, the first boundary point classifier decoder employs a symmetrical operation on the second and first point cloud data, that is, it uses the local point-by-point features of the first point cloud data and the high-level feature tensor of the second point cloud data to obtain the point-by-point classification result of the first point cloud data. Optionally, the classification results of each point in the point cloud data are divided into only two categories: boundary points and non-boundary points. Based on the local point-by-point features of the point cloud data, each point in the point cloud data can be assigned a value of 0 (non-boundary point) and 1 (boundary point). This achieves the effect of accurately determining the boundary point information of the point cloud data based on the local point-by-point features of the point cloud data. Then, based on the target transformation matrix determined by the attitude decoder and the boundary point information of the point cloud data determined by the target boundary point classification decoder, the direction of rotation, rotation angle, displacement, and boundary points in the two point cloud data to be stitched can be determined. Thus, the stitching of the first point cloud data and the second point cloud data can be performed accurately.

[0089] In the above embodiments, the target neural network includes: a first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; wherein, the first encoder and the second encoder are used to extract the global shape features and local point-by-point features of the first point cloud data, and the global shape features and local point-by-point features of the second point cloud data, respectively; the attitude decoder takes the global shape features of the first point cloud data and the global shape features of the second point cloud data as input, thereby accurately determining the target transformation matrix; the target boundary point classification decoder takes the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data as input, thereby accurately determining the boundary point information of the point cloud data; and then, based on the target transformation matrix determined by the attitude decoder and the boundary point information of the point cloud data determined by the target boundary point classification decoder, the direction of rotation, rotation angle, displacement, and boundary points in the two point cloud data to be stitched can be determined, thereby accurately stitching the first point cloud data and the second point cloud data can be performed, thus realizing the task of stitching two point cloud fragments based on the combined use of the encoder and the two decoders in the target neural network.

[0090] In one embodiment, the 3D shape stitching method based on non-overlapping point cloud data further includes:

[0091] If the number of target objects to be concatenated in the target set is N, and N is greater than 2, then...

[0092] If the number of target objects to be concatenated in the target set is N, and N is greater than 2, then...

[0093] Step a: In the target set, determine the first target object;

[0094] Step b: Remove the first target object from the target set to obtain the candidate splicing set;

[0095] Step c: Determine the target transformation matrix and point cloud boundary point information of each target object in the candidate splicing set corresponding to the first target object;

[0096] Based on the target transformation matrix and point cloud boundary point information of each target object in the candidate stitching set and the first target object respectively, the target distance between each target object in the candidate stitching set and the first target object is obtained using a preset distance calculation method.

[0097] Determine the minimum distance among all target distances; and designate the target object corresponding to the minimum distance as the second target object;

[0098] Step d: Combine the first point cloud data of the first target object and the second point cloud data of the second target object to obtain the target composite object;

[0099] Step e: Remove the second target object from the candidate splicing set, add the target splicing object to the candidate splicing set, and obtain the updated candidate splicing set; use the updated candidate splicing set as the target set.

[0100] Repeat step ae until the number of target objects to be spliced ​​in the candidate splicing set is 0.

[0101] Specifically, when the number of target objects to be stitched in the target set is N, and N is greater than 2 (meaning multiple fragments (point cloud data of the target objects) of the same object need to be stitched together), only two adjacent point cloud data of the target objects are stitched together at a time. This process is iterated to stitch together all the target objects to be stitched in the target set. In other words, when the target neural network needs to stitch together multiple target objects in the target set, each iteration uses two complete, adjacent, but non-overlapping point clouds as input, stitching together only two adjacent point clouds at a time. Through multiple iterations, all the fragments (target objects) in the target set can be stitched together.

[0102] Optionally, the stitching order of point clouds can be determined by calculating the boundary distance between any two point clouds from multiple point clouds to be stitched; alternatively, for better stitching results, the distance between point cloud pairs should be as small as possible. Optionally, the stitching order of each target object in the target set can be determined in the following way:

[0103] First, according to step a, a first target object is determined from the target objects in the target set. Optionally, when determining the first target object, a target object can be randomly selected from the target set as the first target object, or it can be determined in other ways. No limitation is made in this embodiment of the invention.

[0104] Then, according to step b, the first target object is removed from the target set, resulting in a candidate concatenation set; that is, the candidate concatenation set does not include the first target object determined in step a. For example, if the target set includes target object 1, target object 2, target object 3, and target object 4, and target object 4 is taken as the first target object, then the candidate concatenation set includes target object 1, target object 2, and target object 3, but does not include target object 4.

[0105] Next, according to step c, determine the target transformation matrix and point cloud boundary point information corresponding to the first target object for each target object in the candidate stitching set; based on the target transformation matrix and point cloud boundary point information corresponding to the first target object for each target object in the candidate stitching set, use a preset distance calculation method to obtain the target distances corresponding to the first target object for each target object in the candidate stitching set; determine the minimum distance among all target distances; and take the target object corresponding to the minimum distance as the second target object; optionally, the preset distance calculation method can be a bidirectional nearest neighbor distance calculation method, or other distance calculation methods, which are not limited in this embodiment of the invention; that is, respectively select the candidate In the concatenation and combination process, target object 1 and the first target object (target object 4) are used as inputs to the target neural network to obtain the target transformation matrix and point cloud boundary information corresponding to target object 1 and the first target object (target object 4). Similarly, in the candidate concatenation and combination process, target object 2 and the first target object (target object 4) are used as inputs to the target neural network to obtain the target transformation matrix and point cloud boundary information corresponding to target object 2 and the first target object (target object 4). Likewise, in the candidate concatenation and combination process, target object 3 and the first target object (target object 4) are used as inputs to the target neural network to obtain the target transformation matrix and point cloud boundary information corresponding to target object 3 and the first target object (target object 4). Then, based on the candidate concatenation... By combining the target transformation matrices and point cloud boundary information corresponding to target object 1 and the first target object (target object 4), the bidirectional nearest neighbor distance between target object 1 and the first target object (target object 4) is obtained. Based on the target transformation matrices and point cloud boundary information corresponding to target object 2 and the first target object (target object 4) in the candidate splicing combination, the bidirectional nearest neighbor distance between target object 2 and the first target object (target object 4) is obtained. Based on the target transformation matrices and point cloud boundary information corresponding to target object 3 and the first target object (target object 4) in the candidate splicing combination, the bidirectional nearest neighbor distance between target object 3 and the first target object (target object 4) is obtained. Finally, from target object 1 and the first target object (target object 4)... The minimum bidirectional nearest neighbor distance is determined from the bidirectional nearest neighbor distance between target object 1 (target object 4), the bidirectional nearest neighbor distance between target object 2 and the first target object (target object 4), and the bidirectional nearest neighbor distance between target object 3 and the first target object (target object 4). Optionally, if the bidirectional nearest neighbor distance between target object 3 and the first target object (target object 4) is the minimum, then target object 3 is taken as the second target object. Thus, the first target object (target object 4) and the second target object (target object 3) are determined from multiple target objects to be spliced ​​in the target set using the target neural network, thereby accurately determining the splicing order of each target object in the target set.Then, by inputting the point cloud data of the first target object (target object 4) and the point cloud data of the second target object (target object 3) into the target neural network, the concatenated result of the point cloud data of the first target object and the point cloud data of the second target object (the concatenated result of target object 4 and target object 3) can be obtained.

[0106] Optionally, in this embodiment of the invention, the distance between target objects is determined based on the bidirectional nearest neighbor distance. It should be noted that the distance between the first target object and the second target object can also be determined based on any other distance determination method, and this embodiment of the invention does not impose any limitation on this method. Optionally, the bidirectional nearest neighbor distance can be calculated and determined based on the definition in related technologies, and this embodiment of the invention does not impose any limitation on this method. Optionally, the bidirectional nearest neighbor distance can be determined based on the following method:

[0107] Where, d bn (S1, S2) is used to represent the bidirectional nearest neighbor distance between target objects S1 and S2, where x represents the point in the cloud point data corresponding to target object S1; and y represents the point in the cloud point data corresponding to target object S2.

[0108] Step d: The first point cloud data of the first target object and the second point cloud data of the second target object are concatenated to obtain the target concatenated object; for example, the point cloud data of the first target object (target object 4) and the point cloud data of the second target object (target object 3) are input into the target neural network to obtain the concatenation result of the point cloud data of the first target object and the point cloud data of the second target object (the concatenation result of target object 4 and target object 3); the concatenation process of the first target object (target object 4) and the second target object (target object 3) is as described in the foregoing embodiments of this application and will not be repeated here.

[0109] Step e: Remove the second target object from the candidate splicing set, add the target splicing object to the candidate splicing set, and obtain the updated candidate splicing set; use the updated candidate splicing set as the target set.

[0110] Repeat step ae until the number of target objects to be spliced ​​in the candidate splicing set is 0.

[0111] Specifically, after stitching the first and second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain the target stitching object, that is, after stitching the first target object (target object 4) and the second target object (target object 3) in the target set, the second target object (target object 3) can be deleted from the candidate stitching set, and the target stitching object (the stitching result of target object 3 and target object 4) can be added to the candidate stitching set to obtain the updated candidate stitching set. The updated candidate stitching set is used as the target set; that is, the updated candidate stitching set includes the stitching results of target object 1, target object 2, target object 3 and target object 4 to be stitched. The updated candidate stitching set is used as the target set, that is, the target set includes 3 objects, namely: the stitching results of target object 1, target object 2, target object 3 and target object 4.

[0112] Then, step ae is repeated until the number of target objects to be spliced ​​in the candidate splicing set is 0. Optionally, during the iteration process, a first target object can be determined from the target set. Any target object in the target set can be used as the first target object, that is, target object 1 can be used as the first target object, target object 2 can be used as the first target object, or the splicing result of target object 3 and target object 4 can be used as the first splicing object. For example, the splicing result of target object 4 and target object 3 can be used as the first target object, and a second target object can be determined from the candidate splicing set (target object 1 and target object 2) for splicing.

[0113] Optionally, the process of determining the second target object from the candidate splicing set (target object 1 and target object 2) can be determined by repeatedly executing step c, that is, by determining the second target object based on the bidirectional nearest neighbor distance between target object 1 and the first target object (the splicing result of target object 4 and target object 3), and the bidirectional nearest neighbor distance between target object 2 and the first target object (the splicing result of target object 4 and target object 3); if the bidirectional nearest neighbor distance between target object 1 and the first target object (the splicing result of target object 4 and target object 3) is less than the bidirectional nearest neighbor distance between target object 2 and the first target object (the splicing result of target object 4 and target object 3), then step d is executed, and target object 1 is taken as the second target object and spliced ​​with the first target object (the splicing result of target object 4 and target object 3) to obtain the target splicing object (the splicing result of target object 4, target object 3, and target object 1), and then step e is repeated. The second target object (target object 1) is removed from the candidate splicing set. The target splicing object (the splicing result of target object 4, target object 3, and target object 1) is added to the candidate splicing set, resulting in an updated candidate splicing set. This updated candidate splicing set is used as the target set. In other words, the updated candidate splicing set includes the target object 2 to be spliced ​​and the splicing results of target object 3, target object 4, and target object 1. The updated candidate splicing set is used as the target set, which includes two objects: target object 2 and the splicing results of target object 3, target object 4, and target object 1. Since the candidate splicing set only contains target object 2, the target splicing object (the splicing result of target object 4, target object 3, and target object 1) is used as the first target object, and target object 2 is used as the second target object for splicing, thus obtaining the target splicing result (the splicing result of target object 4, target object 3, target object 1, and target object 2).

[0114] In other words, when the target set includes N target objects to be spliced, it is necessary to iteratively execute step ac to select the second target object from the candidate splicing set, the splicing of the first and second target objects, and the updating of the target set in step e, until the number of target objects in the candidate splicing set is 0, that is, when there are no target objects to be spliced ​​in the candidate splicing set, the splicing of all N target objects to be spliced ​​in the target set is completed.

[0115] In this embodiment of the invention, based on the target transformation matrix and point cloud boundary point information corresponding to each target object in the candidate splicing set and the first target object, and using a preset distance calculation method, the target distances between each target object in the candidate splicing set and the first target object are obtained; the minimum distance among all target distances is determined; and the target object corresponding to the minimum distance is taken as the second target object; optionally, the splicing effect of the two target objects with the smallest distance is the best; that is, by determining the minimum distance, the two fragments with the best splicing effect are determined, and then the splicing of multiple fragments can be completed through iteration. In the process of splicing four fragments in this embodiment of the invention, the best result is selected each time and added to the already spliced ​​position until all four fragments are spliced.

[0116] In the method of this embodiment of the invention, each iteration determines a point cloud as the target point cloud, and all the remaining point clouds are used as source point clouds to assemble with the target point cloud. The best result is taken as the point cloud assembly result for that iteration. The selection is based on the bidirectional nearest neighbor distance between the source point cloud and the target point cloud. The smaller the distance, the better the result. Then, the two point clouds with the smallest distance and the shapes they represent are merged until all shapes are merged into a final complete shape.

[0117] For example, the specific steps are as follows:

[0118] (1) For multiple shapes that need to be assembled, iteratively, in each round, a point cloud is randomly selected from the set S of all shape point clouds as the target point cloud A.

[0119] The stopping condition for iteration is when there is only one element left in the set.

[0120] (3) In the remaining point cloud, in this round, take each point cloud as the source point cloud B in turn.

[0121] (4) Sample both point clouds to 1024 points.

[0122] (5) Assemble B onto A and calculate the distance between the boundary points of A and B using an artificial deep neural network.

[0123] (6) Use the bidirectional nearest neighbor distance between the two point cloud boundaries.

[0124] (7) Select the combination AB that minimizes the distance between the source point cloud B and the target point cloud A in this round, and then stitch and merge the shapes of AB according to the predicted transformation matrix.

[0125] (8) Update the merged shape to the original set S.

[0126] (9) Iterate from step 1 until only one element remains in the set, which is the final shape of all the point clouds.

[0127] The method described in the above embodiment, when it is necessary to stitch together multiple fragments (point cloud data of the target object) of the same object, iteratively executes the process of selecting a second target object from the candidate stitching set, stitching the first target object and the second target object, and updating the candidate stitching set and the first target object through the target neural network. This accurately determines the stitching order of each target object in the target set and iteratively realizes the stitching of all target objects to be stitched in the target set, thereby extending the stitching task of point cloud data of two target objects to the stitching task of point cloud data of multiple target objects.

[0128] In one embodiment, the attitude decoder is trained based on a first target mixture loss function; the first target mixture loss function is determined based on a homogeneous second-order difference loss function, a bidirectional nearest neighbor distance loss function, and an invertible minimum distance loss function.

[0129] The target boundary point classification decoder is trained based on the second target hybrid loss function; the second target hybrid loss function is determined based on the cross-entropy loss function and the bidirectional nearest neighbor distance loss function.

[0130] Specifically, in this embodiment of the invention, the attitude decoder is used to output a target transformation matrix based on the global shape features of the point cloud; that is, by combining the global shape features of two point clouds and inputting them into the attitude decoder, attitude information described by Lie algebra describing how to assemble the two point clouds is obtained; the target boundary point classification decoder is used to output the boundary point information of the point cloud data based on the local point-by-point features of the point cloud data; and then, based on the target transformation matrix determined by the attitude decoder and the boundary point information of the point cloud data determined by the target boundary point classification decoder, the direction of rotation, rotation angle, displacement of the second point cloud data of the second target object and the boundary points in the two point cloud data to be spliced ​​can be determined, and thus the task of splicing two point cloud fragments can be accurately performed.

[0131] To enable the attitude encoder to accurately determine the target transformation matrix, the attitude decoder is trained based on a first target hybrid loss function in this invention. Optionally, the first target hybrid loss function is determined based on a homogeneous second-order difference loss function, a bidirectional nearest neighbor distance loss function, and an invertible minimum distance loss function. Optionally, the first target hybrid loss function is determined based on the following formula:

[0132] L pose =L M2 +L bn +L rn

[0133] Among them, L pose Let L represent the first objective mixture loss function. M2 Let L represent the homogeneous second-order difference loss function. bn L represents the bidirectional nearest neighbor distance loss function. rn This represents the invertible minimum distance loss function.

[0134] Alternatively, the homogeneous second-order difference loss function is determined based on the following formula:

[0135]

[0136] Where M2 is defined as Where T is the target transformation matrix predicted by the target neural network. It is the actual target transformation matrix, that is It is tag information.

[0137] Optionally, the bidirectional nearest neighbor distance loss function is determined based on the following formula:

[0138] L bn =d bn (S1, S2)

[0139]

[0140] Where S2 is the predicted value, that is, the point cloud data after the second point cloud data is moved by the transformation matrix predicted by the target neural network; S1 is the point cloud data after the second point cloud data is moved by the actual transformation matrix.

[0141] Alternatively, the invertible minimum distance loss function is determined based on the following formula:

[0142] L rn =d rn (S1, S2)

[0143]

[0144] Where S2 is the predicted value, that is, the point cloud data after the second point cloud data is moved by the transformation matrix predicted by the target neural network; S1 is the point cloud data after the second point cloud data is moved by the actual transformation matrix; φ represents bijection.

[0145] Optionally, to enable the target boundary point classification decoder to accurately determine the boundary point information of the point cloud data, the target boundary point classification decoder is trained based on a second target hybrid loss function in this embodiment of the invention. Optionally, the second target hybrid loss function is determined based on the cross-entropy loss function and the bidirectional nearest neighbor distance loss function. Optionally, the second target hybrid loss function is determined based on the following formula:

[0146] L pose =L CE +L bn

[0147] Among them, L pose Let L represent the second objective mixed loss function. CE L′ represents the cross-entropy loss function. bn This represents the bidirectional nearest neighbor distance loss function.

[0148] Alternatively, the cross-entropy loss function is determined based on the following formula:

[0149]

[0150] in, Bo represents the ground truth values ​​of the boundary points of the first and second point cloud data, respectively. A and Bo B Let represent the boundary point prediction values ​​of the first point cloud data and the second point cloud data, respectively. CE(·) is the cross-entropy loss function, and R and t are the attitude rotation matrix and attitude displacement vector in the target transformation matrix predicted by the target neural network, respectively.

[0151] Optionally, the bidirectional nearest neighbor distance loss function is determined based on the following formula:

[0152]

[0153] in, Bo represents the truth value of the boundary point of the second point cloud data. B Let R represent the predicted boundary point value of the second point cloud data, and let R and t be the attitude rotation matrix and attitude displacement vector in the target transformation matrix predicted by the target neural network, respectively.

[0154] The method described above trains the attitude decoder using the homogeneous second-order difference loss function, the bidirectional nearest neighbor distance loss function, and the invertible minimum distance loss function in the first target mixture loss function, enabling the attitude decoder to accurately output the target transformation matrix. It also trains the target boundary point classification decoder using the cross-entropy loss function and the bidirectional nearest neighbor distance loss function in the second target mixture loss function, enabling the target boundary point classification decoder to accurately output boundary point information. Based on the accurate target transformation matrix and boundary point information, the rotation direction, rotation angle, displacement, and boundary points in the two point cloud data sets to be stitched can be determined, allowing for accurate stitching of the first and second point cloud data sets. This avoids the influence of data acquisition methods, resolution, and shape transformation on the stitching effect, improving the efficiency and accuracy of point cloud data stitching.

[0155] For example, to verify the accuracy of the 3D shape stitching method based on non-overlapping point cloud data in this embodiment of the invention, the CAD dataset and the real-world scene dataset DublinCity provided by the ModelNet dataset were used. In the experiments, to verify the effectiveness of the method, the real-world CAD dataset was used. The CAD dataset was segmented using different 3D primitives, including: plane, cone, cylinder, and sphere. The real-world scene dataset was segmented using different semantic labels, including: roof and building facade. In the experiments, previous methods struggled to achieve non-overlapping stitching, while the method of this invention can perform completely non-overlapping assembly tasks.

[0156] In practical applications, the accuracy of the assembly method provided by this invention is evaluated through experiments. The experimental platform is a computer with an Intel 6226R CPU, 256GB RAM, and a 64-bit Ubuntu 20.04 operating system, and the PyTorch open-source deep learning framework is used on this platform. Offline training is run on an NVIDIA GeForce GTX 2080 Ti GPU (11GB VRAM).

[0157] Figure 3 This invention addresses the splicing of multiple adjacent shapes with multiple fragments. The baseline method of this invention can splice two objects and explicitly indicate the set of adjacent contact points at the splice point. In an iterative algorithm, the distance between different combinations is calculated based on the set of adjacent points, and the two fragments with the best splicing effect are selected to complete the splicing of multiple fragments. Figure 3 The basic steps for piecing together the four pieces (a) are shown, (b) the best result is selected each time and added to the already pieced position, until (c) all pieces are pieced together, and the final assembly result is shown in (c).

[0158] Figure 4 This is the result of stitching together only in the case of two objects. As shown in the figure, the results of stitching together the same model using four different primitives and the results of boundary prediction are displayed, including the input, the predicted stitched result, the ground truth, the predicted boundary result, and the ground truth. The results fully demonstrate the excellent robustness and accuracy of this invention in the case of two-object stitching.

[0159] Figure 5-6 The comparative experiments were conducted with only two shapes, including the input, prediction results of different methods, and the true value. The results fully demonstrate the superior robustness and accuracy of the present invention compared to existing methods.

[0160] To fully demonstrate the robustness of multi-fragmentation scenarios, this invention tested experiments with different numbers of fragments under different model categories.

[0161] Figure 7 This is an iterative diagram illustrating the piecing together of multiple fragments. In each iteration, the two shapes with the smallest distance between their boundary points, predicted by the network, are selected as the results for that iteration. After multiple iterations, the multiple fragments can be pieced together into a complete shape.

[0162] Figure 8 This invention presents assembly experiments using the same artificial neural network model with varying numbers of fragments on different models. In tasks involving multiple fragments and point clouds segmented using geometric primitives, this invention demonstrates results surpassing those of current state-of-the-art methods.

[0163] Figure 9 The results show recall curves using the same artificial neural network model on the same dataset, with different numbers of fragments and different rotation and displacement thresholds. The results fully demonstrate the excellent generalization ability of this invention, which uses a multi-task network to learn the assembly matrix and boundary value approach separately.

[0164] The following describes the three-dimensional shape stitching device based on non-overlapping point cloud data provided by the present invention. The three-dimensional shape stitching device based on non-overlapping point cloud data described below and the three-dimensional shape stitching method based on non-overlapping point cloud data described above can be referred to in correspondence with each other.

[0165] Figure 10 This is a schematic diagram of the structure of the three-dimensional shape stitching device based on non-overlapping point cloud data provided by the present invention. The three-dimensional shape stitching device based on non-overlapping point cloud data provided in this embodiment includes:

[0166] The acquisition module 710 is used to acquire the first point cloud data of the first target object and the second point cloud data of the second target object;

[0167] The processing module 720 is used to input the first point cloud data and the second point cloud data into the pre-trained target neural network to obtain the target transformation matrix and the boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data splicing; the target neural network is trained based on the point cloud data samples to be spliced.

[0168] The stitching module 730 is used to stitch together the first point cloud data and the second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain the target stitching object.

[0169] The apparatus of this invention is used to execute the method in any of the foregoing method embodiments, and its implementation principle and technical effect are similar, so they will not be described again here.

[0170] Figure 11A schematic diagram of the physical structure of an electronic device is provided. This electronic device may include a processor 810, a communication interface 820, a memory 830, and a communication bus 840. The processor 810, communication interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a 3D shape stitching method based on non-overlapping point cloud data. This method includes: acquiring first point cloud data of a first target object and second point cloud data of a second target object; ensuring no overlap between the first and second point cloud data; inputting the first and second point cloud data into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data stitching; the target neural network is trained based on the point cloud data samples to be stitched; and stitching the first and second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain the target stitched object.

[0171] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the three-dimensional shape stitching method based on non-overlapping point cloud data provided by the above methods, the method comprising: acquiring first point cloud data of a first target object and second point cloud data of a second target object, wherein the first point cloud data and the second point cloud data are non-overlapping; inputting the first point cloud data and the second point cloud data into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix representing the pose information of the point cloud data stitching; the target neural network being trained based on the point cloud data samples to be stitched; and stitching the first point cloud data and the second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain a target stitching object.

[0172] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned three-dimensional shape stitching methods based on non-overlapping point cloud data. The method includes: acquiring first point cloud data of a first target object and second point cloud data of a second target object; ensuring no overlap between the first and second point cloud data; inputting the first and second point cloud data into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix representing the pose information of the point cloud data stitching; the target neural network being trained based on the point cloud data samples to be stitched; and stitching the first and second point cloud data according to the target transformation matrix and the boundary point information of the point cloud data to obtain a target stitching object.

[0173] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for stitching three-dimensional shapes based on non-overlapping point cloud data, characterized in that, include: Acquire the first point cloud data of the first target object and the second point cloud data of the second target object; There is no overlap between the first point cloud data and the second point cloud data; The first point cloud data and the second point cloud data are input into a pre-trained target neural network to obtain a target transformation matrix and boundary point information of the point cloud data; the target transformation matrix represents the pose information of the point cloud data splicing; the target neural network is trained based on the point cloud data samples to be spliced. Based on the target transformation matrix and the boundary point information of the point cloud data, the first point cloud data and the second point cloud data are stitched together to obtain the target stitching object; The target neural network includes: A first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; the first encoder, the second encoder, and the attitude decoder are connected; the first encoder, the second encoder, and the target boundary point classification decoder are connected; the first encoder and the second encoder have the same structure; The first encoder is used to extract global shape features and local point-by-point features from the first point cloud data; The second encoder is used to extract global shape features and local point-by-point features from the second point cloud data; The attitude decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data; The target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data. The attitude decoder is trained based on a first target hybrid loss function; the first target hybrid loss function is determined based on a homogeneous second-order difference loss function, a bidirectional nearest neighbor distance loss function, and an invertible minimum distance loss function. The target boundary point classification decoder is trained based on a second target hybrid loss function; the second target hybrid loss function is determined based on the cross-entropy loss function and the bidirectional nearest neighbor distance loss function.

2. The three-dimensional shape stitching method based on non-overlapping point cloud data according to claim 1, characterized in that, The attitude decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data, including: The attitude decoder is used to connect the global shape features of the first point cloud data and the global shape features of the second point cloud data to obtain the target global shape features; Determine the pose transformation Lie algebra based on the global shape features of the target; The target transformation matrix is ​​determined based on the attitude transformation Lie algebra; the target transformation matrix includes an attitude rotation matrix and an attitude displacement vector.

3. The three-dimensional shape stitching method based on non-overlapping point cloud data according to claim 2, characterized in that, The target boundary point classification decoder includes a first boundary point classification decoder and a second boundary point classification decoder; the target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data, including: The first boundary point classification decoder in the target boundary point classification decoder is used to perform max pooling on the local point-by-point features of the second point cloud data, and then obtain the high-level feature tensor of the second point cloud data through repeated element terms. After concatenating the high-level feature tensor of the second point cloud data and the local point-by-point features of the first point cloud data, the data is decoded and classified to obtain the boundary point information in the first point cloud data. The second boundary point classification decoder in the target boundary point classification decoder is used to perform max pooling on the local point-by-point features of the first point cloud data, and then obtain the high-level feature tensor of the first point cloud data through repeated element terms. After concatenating the high-level feature tensor of the first point cloud data and the local point-by-point features of the second point cloud data, the boundary point information in the second point cloud data is obtained by decoding and classification.

4. The three-dimensional shape stitching method based on non-overlapping point cloud data according to any one of claims 1-3, characterized in that, Also includes: If the number of target objects to be concatenated in the target set is N, and N is greater than 2, then... Step a: Determine the first target object from the target set; Step b: Remove the first target object from the target set to obtain a candidate splicing set; Step c: Determine the target transformation matrix and boundary point information of the point cloud data corresponding to each target object in the candidate splicing set and the first target object, respectively; Based on the target transformation matrix corresponding to the first target object and the boundary point information of the point cloud data for each target object in the candidate stitching set, the target distance between each target object in the candidate stitching set and the first target object is obtained using a preset distance calculation method. Determine the minimum distance among all the target distances; and designate the target object corresponding to the minimum distance as the second target object; Step d: Combine the first point cloud data of the first target object and the second point cloud data of the second target object to obtain the target composite object; Step e: Remove the second target object from the candidate splicing set, add the target splicing object to the candidate splicing set, and obtain an updated candidate splicing set; use the updated candidate splicing set as the target set; Repeat step ae until the number of target objects to be spliced ​​in the candidate splicing set is 0.

5. A three-dimensional shape stitching device based on non-overlapping point cloud data, characterized in that, include: The acquisition module is used to acquire the first point cloud data of the first target object and the second point cloud data of the second target object; There is no overlap between the first point cloud data and the second point cloud data; The processing module is used to input the first point cloud data and the second point cloud data into a pre-trained target neural network to obtain the target transformation matrix and the boundary point information of the point cloud data; The target transformation matrix represents the attitude information of the point cloud data stitching; The target neural network is trained based on point cloud data samples to be stitched together; the target neural network includes: A first encoder, a second encoder, an attitude decoder, and a target boundary point classification decoder; the first encoder, the second encoder, and the attitude decoder are connected; the first encoder, the second encoder, and the target boundary point classification decoder are connected; the first encoder and the second encoder have the same structure; The first encoder is used to extract global shape features and local point-by-point features from the first point cloud data; The second encoder is used to extract global shape features and local point-by-point features from the second point cloud data; The attitude decoder is used to obtain the target transformation matrix based on the global shape features of the first point cloud data and the global shape features of the second point cloud data; The target boundary point classification decoder is used to obtain the boundary point information of the point cloud data based on the local point-by-point features of the first point cloud data and the local point-by-point features of the second point cloud data. The attitude decoder is trained based on a first target hybrid loss function; the first target hybrid loss function is determined based on a homogeneous second-order difference loss function, a bidirectional nearest neighbor distance loss function, and an invertible minimum distance loss function. The target boundary point classification decoder is trained based on a second target hybrid loss function; the second target hybrid loss function is determined based on the cross-entropy loss function and the bidirectional nearest neighbor distance loss function. The stitching module is used to stitch the first point cloud data and the second point cloud data together based on the target transformation matrix and the boundary point information of the point cloud data to obtain the target stitching object.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the three-dimensional shape stitching method based on non-overlapping point cloud data as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the three-dimensional shape stitching method based on non-overlapping point cloud data as described in any one of claims 1 to 4.

Images