A 3D point cloud registration method and system based on a probability model, equipment and medium

Abstract

The application relates to the technical field of computer vision and computer graphics, and discloses a 3D point cloud registration method and system based on a probability model, which comprises the following steps: after a HybridMatch data set is constructed and a HybirdReg probability model is constructed, the HybirdReg probability model is trained in the HybridMatch data set, a loss function is used to guide the HybirdReg probability model, and a trained HybirdReg probability model is output; a point cloud pair is selected in the HybridMatch data set and input into the trained diffusion network model, a super point pair match is predicted, a 3D transformation matrix of the point cloud is calculated according to the super point pair match, and 3D point cloud registration is completed through 3D transformation of the point cloud. The system comprises a data set construction unit, a model construction unit, a model training unit and a registration transformation unit. The application also discloses an electronic device and a computer readable storage medium. The application is used for robustly processing 3D point cloud registration in a mixed motion scene.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision and computational graphics, and in particular to a 3D point cloud registration method and system, device, and medium based on a probability model, which are used for robustly processing 3D point cloud registration in mixed motion scenes. Background Art

[0002] 3D point cloud registration is the process of aligning point cloud data collected at different times, viewpoints, or sensors. It has widespread applications in fields such as robotics, computer graphics, and computer vision. Existing algorithms rely on the rigid body assumption, assuming only rigid body motion in the scene. However, in real indoor scenes, point clouds often contain two types of motion: rigid background motion caused by the camera; and non-rigid foreground motion caused by object movement or deformation.

[0003] Current point cloud registration algorithms often ignore the impact of non-rigid foreground motion on their results, especially in partially overlapping scenes, which can easily lead to erroneous correspondences and transformation calculations. Furthermore, existing large-scale datasets often assume rigid scenes and lack large-scale datasets that include mixed motion scenes, which limits the generalization and robustness of the models.

[0004] Although some works have attempted to alleviate the problem of non-rigid matching by using simulated data, such as Lepard introducing an object-level non-rigid point cloud registration dataset, the simple shape prior at the object level is still significantly different from the complex indoor structures in real scenes, making it difficult to generalize to complex scenes.

[0005] In addition, points with similar local geometric structures in a dynamic foreground may become outliers due to their inconsistent motion with the background, making the matching of correspondences more difficult. Therefore, the performance of existing methods will be significantly degraded when directly applied to mixed motion scenes.

[0006] In summary, the present invention provides a 3D point cloud registration method, system, device, and medium based on a probability model, which are used to process highly robust registration of complex mixed motion scenes in 3D point clouds. Summary of the Invention

[0007] The present invention provides a 3D point cloud registration method, system, device, and medium based on a probability model, which are used to process highly robust registration of complex mixed motion scenes in 3D point clouds.

[0008] The present invention is implemented by the following technical solution: a 3D point cloud registration method based on a probability model, comprising the following steps:

[0009] Step S1, constructing the HybridMatch dataset;

[0010] Step S2, constructing a HybirdReg probability model, which includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud coarse matching module using the uncertainty mask to filter the point cloud, and a final transformation estimation module;

[0011] Step S3, training the HybridReg probability model in the HybridMatch dataset, using a loss function to guide the HybridReg probability model, and outputting the trained HybridReg probability model;

[0012] In step S4, point cloud pairs are selected from the HybridMatch dataset and input into the trained diffusion network model to predict super point pair matching. The point cloud 3D transformation matrix is ​​calculated based on the super point pair matching, and the point cloud is transformed into 3D to complete the 3D point cloud registration.

[0013] In order to better implement the present invention, further, step S1 includes:

[0014] Step S11, constructing a background scene;

[0015] Step S12, introducing foreground moving objects into the background scene to construct a mixed motion scene, wherein the foreground moving objects include rigid objects and non-rigid objects;

[0016] Step S13, using a virtual camera to generate a depth map of the mixed motion scene;

[0017] Step S14, converting the depth map into a point cloud to generate registration data;

[0018] Step S15: Divide the registration data according to the point cloud overlap ratio to construct a HybridMatch dataset.

[0019] In order to better implement the present invention, further, step S2 includes:

[0020] The point cloud pair input module, point cloud downsampling and feature extraction module, geometric transformation module, point cloud patch construction module, uncertainty mask estimation module, point cloud coarse matching module using uncertainty mask filtering, and final transformation estimation module are connected in sequence from front to back;

[0021] The geometric transformation module includes a self-attention module and a cross-attention module.

[0022] In order to better implement the present invention, further, the point cloud input module inputs a point cloud pair into the point cloud downsampling and feature extraction module, the point cloud pair including a source point cloud and a target point cloud;

[0023] The point cloud downsampling and feature extraction module downsamples the source point cloud and the target point cloud to obtain superpoints corresponding to the source point cloud and the target point cloud, performs feature extraction on the superpoints to obtain feature values ​​of the superpoints, upsamples the superpoints and extracts features to obtain features of the upsampled superpoints and their corresponding feature values; the point cloud downsampling and feature extraction module inputs the superpoints and the feature values ​​of the superpoints into the geometric transformation module;

[0024] The geometric transformation module obtains a coarse match between the source point cloud superpoints and the target point cloud superpoints;

[0025] The point cloud patch construction module upsamples the source point cloud superpoints and matches them with the source point cloud superpoints according to the nearest neighbor algorithm to form a source point cloud patch; upsamples the target point cloud superpoints and matches them with the target point cloud superpoints according to the nearest neighbor algorithm to form a target point cloud patch;

[0026] The uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density;

[0027] The method uses the uncertainty mask to filter the point cloud coarse matching module to obtain the final super-point correspondence;

[0028] The estimated final transformation module uses the RANSAC or LGR algorithm to calculate the final 3D transformation matrix from the filtered matches and applies it to the target point cloud to achieve alignment between the source point cloud and the target point cloud.

[0029] In order to better implement the present invention, further, the method for the geometric transformation module to obtain a rough match between the source point cloud and the target point cloud includes:

[0030] The geometric transformation module encodes the super-points and their eigenvalues ​​in the spatial domain, and outputs point cloud eigenvalues ​​with position information. The self-attention module performs self-attention calculation on the super-points and their eigenvalues ​​after position encoding; the cross-attention module performs cross-attention calculation on the super-points and their eigenvalues ​​calculated by the self-attention module, and finds the cross information of the source point cloud and the target point cloud by combining the features of the source point cloud and the target point cloud; the Gaussian correlation of the super-point eigenvalues ​​after the cross-attention calculation is calculated to obtain a rough match between the source point cloud and the target point cloud.

[0031] In order to better implement the present invention, further, the method in which the uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density includes:

[0032] Design a network to predict the parameters of a probability distribution Among them, the parameters include the matching relationship between the source point cloud and the target point cloud and the uncertainty of the relationship σ 2 ;

[0033] By estimating the conditional probability density Get the uncertainty of the forecast, where is the matching pair of the true source point cloud and the target point cloud, are the input source point cloud and target point cloud, θ is the parameter of the neural network;

[0034] Among them, the uncertainty of the relationship σ 2 Used as uncertainty mask to indicate the confidence of the matching pair, the variance σ 2 Constrained to the range [0, 1], values ​​close to 1 represent high-confidence matches corresponding to possible overlapping point pairs, and values ​​close to 0 represent low-confidence matches corresponding to non-overlapping or foreground point pairs.

[0035] In order to better implement the present invention, further, the loss function in step S3 is composed of an uncertainty mask loss function, an overlap-based circle loss function and a point matching loss function.

[0036] The present invention also provides a 3D point cloud registration system based on a probability model, comprising a data set construction unit, a model construction unit, a model training unit and a registration transformation unit, wherein:

[0037] Dataset construction unit, used to construct the HybridMatch dataset;

[0038] A model construction unit is used to construct a HybirdReg probability model, which includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud coarse matching module using the uncertainty mask to filter the point cloud, and a final transformation estimation module;

[0039] a model training unit, configured to train the HybridReg probability model in the HybridMatch dataset, guide the HybridReg probability model using a loss function, and output a trained HybridReg probability model;

[0040] The registration transformation unit is used to select point cloud pairs in the HybridMatch dataset and input them into the trained diffusion network model, predict super point pair matching, calculate the point cloud 3D transformation matrix based on the super point pair matching, and perform 3D transformation on the point cloud to complete 3D point cloud registration.

[0041] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the 3D point cloud registration system based on the probability model described in the second aspect above.

[0042] The present invention also provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the third aspect, the electronic device executes the method described in the first aspect.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] (1) This paper provides a 3D point cloud registration method, system, device, and medium based on a probabilistic model. It constructs the first large-scale indoor hybrid motion synthetic dataset, named HybridMatch. This dataset combines the rigid background of the 3D-FRONT dataset, the object-level rigid motion of the ShapeNet dataset, and the deformable objects in the DeformingThings4D dataset to generate a point cloud registration dataset with diverse and complex motion.

[0045] (2) The present invention provides a 3D point cloud registration method, system, device, and medium based on a probability model. A probability model named HybridReg is proposed, and an uncertainty mask prediction mechanism is designed. Combined with the negative log-likelihood (NLL) loss, the robustness of feature extraction and correspondence matching is improved.

[0046] (3) The present invention is qualitatively and quantitatively evaluated on a wide range of indoor and outdoor datasets. Experimental results show that the present invention outperforms existing methods in 3D point cloud hybrid motion registration scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention is further described in conjunction with the following drawings and embodiments, and all concepts and innovations of the present invention should be regarded as disclosed contents and the protection scope of the present invention.

[0048] Figure 1 A flowchart of a 3D point cloud registration method based on a probability model and a method system, device, and medium provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a 3D point cloud registration method based on a probability model and a data set construction in a system, device, and medium provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of the HybirdReg probability model in a 3D point cloud registration method and system, device, and medium based on a probability model provided in an embodiment of the present application;

[0051] Figure 4 Schematic diagram A of a 3D point cloud registration method and system, device, and medium based on a probability model provided in an embodiment of the present application;

[0052] Figure 5 Schematic diagram B of the effects of a 3D point cloud registration method based on a probability model and the system, equipment, and medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "disposed," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediate medium; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention.

[0055] Example 1:

[0056] The present embodiment provides a 3D point cloud registration method, system, device, and medium based on a probability model. Figure 1 As shown, after constructing the HybridMatch dataset and the HybridReg probability model, the present invention trains the HybridReg probability model in the HybridMatch dataset, uses the loss function to guide the HybridReg probability model, and outputs the trained HybridReg probability model; selects point cloud pairs in the HybridMatch dataset and inputs them into the trained diffusion network model, predicts super point pair matching, calculates the point cloud 3D transformation matrix based on the super point pair matching, and performs 3D transformation on the point cloud to complete 3D point cloud registration.

[0057] Example 2:

[0058] This embodiment is further optimized based on embodiment 1. Figure 2 As shown in Figure 1, the first step is to build the HybridMatch dataset.

[0059] Specifically, it includes constructing a background scene, injecting foreground object motion into the background scene to form a hybrid motion scene, then using a virtual camera to generate a depth map of the hybrid motion scene, and then converting the depth map into a point cloud to generate alignment data. Finally, the point cloud is divided according to the overlap ratio to generate the HybridMatch dataset.

[0060] Building the background scene:

[0061] We use the 3D-FRONT dataset to generate indoor backgrounds, which contain professionally designed room layouts and a rich variety of furniture. Leveraging models from ShapeNet, we increase scene complexity by randomly scaling, rotating, and placing objects (e.g., placing them on furniture or hanging them in the air). We also introduce rigid background motion when switching camera perspectives.

[0062] Foreground object motion includes non-rigid objects and rigid objects:

[0063] (1) Non-rigid objects (deformable objects):

[0064] We select human and animal models and their animation sequences from the DeformingThings4D dataset. We randomly select animation sequences from scenes, where each scene contains animations for a single person or multiple people / animals. We use random offsets to enhance the diversity of motion amplitudes.

[0065] (2) Rigid objects (movable objects):

[0066] Place rigid models from ShapeNet around non-rigid objects (e.g., jm chairs, cups, etc.). Apply random XY plane offsets to simulate the movement of rigid objects, while also introducing noise and subtle furniture movement to enhance the realism of the scene. Ensure that all foreground objects are inserted without collision to maintain physical plausibility.

[0067] Generation of registration data:

[0068] A virtual camera is used to generate a depth map of the constructed mixed motion scene and convert the depth map into a point cloud. The depth range is limited and large planes (such as walls and floors) are randomly removed to highlight the motion of foreground objects. The viewing angle range is set to [0°, 360°] in the horizontal direction and [0°, 45°] in the vertical direction. The source and target views are uniformly sampled within the viewing angle range, with sampling intervals of 30° and 15°. The true matching 3D transformation matrix of the source point cloud pair and the target point cloud pair is generated based on the camera pose of the sampled source and target views.

[0069] Data partitioning:

[0070] Based on the point cloud overlap ratio, it is divided into HybridMatch (>30% overlap) and HybridLoMatch (10-30% overlap). Based on the non-rigid motion ratio (the proportion of foreground object motion in the scene), it is further subdivided into 10-30% motion ratio and 30-50% motion ratio. It also includes rigid benchmark segmentation without animation. The synthesis method of the HybridMatch dataset combines rigid and non-rigid motion and generates realistic indoor point cloud scenes with mixed motion through physical plausibility and diversity design.

[0071] The rest of this embodiment is the same as that of embodiment 1, so it will not be described again.

[0072] Example 3:

[0073] This embodiment is further optimized based on the above embodiment 1 or 2. Figure 3 As shown, the HybridReg method specifically includes:

[0074] 1. Point cloud input module

[0075] Input point cloud pair is the source point cloud, is the target point cloud. A neural network is used to estimate the point pair matching between the source point cloud and the target point cloud.

[0076] exist Figure 2 In the middle, the left side is the source point cloud The right side is the target point cloud At this point, the source point cloud and the target point cloud are in a state to be registered.

[0077] 2. Point cloud downsampling and feature extraction module

[0078] Use downsampling to obtain superpoints, use kernel point convolution and feature pyramid network KPConv-FPN to extract point cloud features, and obtain the learning features of source point cloud and target point cloud.

[0079] Downsample the input source point cloud and target point cloud to obtain the superpoints corresponding to the source point cloud and target point cloud. and The KPConv network is used to extract the features of the super points and obtain the feature values ​​of the super points. and Upsample the downsampled superpoints and extract features to obtain and and the corresponding eigenvalues and

[0080] The eigenvalue of the superpoint is used for matching the superpoint (point cloud rough matching), and the eigenvalue of the up-sampled point cloud is used when the patch composed of the up-sampled point cloud and the superpoint is used for subsequent 3D transformation estimation. The point obtained by down-sampling the point cloud is a superpoint, and the eigenvalue extracted from the superpoint is sent to the geometric transformation module. Then the superpoint obtained by down-sampling the point cloud is up-sampled back, which is a smoothing operation on the original point cloud, and can preliminarily reduce noise and other interference, so it needs to be up-sampled back. The eigenvalue of the point cloud up-sampled back is used for subsequent 3D transformation estimation (RANSAC and LGR) (the patch composed of the up-sampled point cloud and the superpoint), so the two eigenvalues (the eigenvalue of the superpoint and the eigenvalue of the up-sampled point cloud) are different and have different functions.

[0081] The up-sampled point cloud and the superpoint are used to form a point cloud patch according to the nearest neighbor method, and then participate in the operation in the last point cloud 3D transformation prediction (ransac and LGR). That is, the estimation of the 3D transformation is based on the patch composed of the up-sampled points.

[0082] 3、Geometric transformer

[0083] The geometric transformation module is divided into a self-attention module and a cross-attention module. The self-attention module learns the global correlation within the point cloud, and uses geometric structure embedding (such as distance and angle information between points) to enhance the feature representation of the point cloud. The cross-attention module comprehensively learns the correlation between the source point cloud and the target point cloud, and learns the potential geometric consistency between adjacent frames, so as to obtain the relevant features between the source point cloud and the target point cloud.

[0084] The geometric transformation module encodes the input superpoint and and its eigenvalue and in the spatial domain, and outputs the point cloud feature value with position information.

[0085] Self-attention block, self-attention calculation is performed on the superpoint and the eigenvalue after position encoding, and the self-representation of the feature is strengthened.

[0086] Cross-attention block, cross-attention calculation is performed on the superpoint and the eigenvalue after self-attention calculation, and the features of the source point cloud and the target point cloud are combined to find the cross information of the source point cloud and the target point cloud.

[0087] Calculate the Gaussian correlation of the super-point feature values ​​after the cross-attention calculation to obtain a rough match between the source point cloud and the target point cloud:

[0088] 4. Build point cloud patch module

[0089] The upsampled source point cloud and the superpoints obtained by downsampling the source point cloud are matched into source point cloud patches according to the nearest neighbor algorithm:

[0090] The calculation of the target point cloud is similar, and we get

[0091] 5. Uncertainty Mask Estimation Module

[0092] By estimating the conditional probability density is the matching pair of the true source point cloud and the target point cloud, is the input source point cloud and target point cloud, θ is the parameter of the neural network, and the uncertainty of the prediction is obtained. We designed a network to predict the parameters of the probability distribution The parameters include the matching relationship between the source point cloud and the target point cloud (mean μ) and the uncertainty of the relationship (variance σ 2 ).

[0093] The network corresponds to Figure 3 UncertaintyMaskGenerator in

[15] .

[0094] Among them, the uncertainty of the relationship (variance σ 2 ) is used as an uncertainty mask to indicate the confidence of the matching pair. We use the variance σ 2 Constrained to the range [0, 1], values ​​close to 1 represent high-confidence matches corresponding to possible overlapping point pairs, and values ​​close to 0 represent low-confidence matches corresponding to non-overlapping or foreground point pairs.

[0095] 6. Use uncertainty mask to filter point cloud coarse matching module

[0096] After extracting and normalizing the superpoint features, the Gaussian correlation matrix S is calculated. The Gaussian correlation matrix S is filtered using the mask generated by the uncertainty mask generation module to suppress false matches caused by mixed motion. The largest N matching pairs in the filtered Gaussian matrix are selected as the final superpoint correspondences.

[0097] Calculate the source point cloud and the target point cloud patch to get the Gaussian correlation matrix (Correlation Matrix S), and use the uncertainty mask to filter the Gaussian correlation matrix. Suppress the false matching caused by mixed motion. Select the largest N matching (Top-K) pairs in the filtered Gaussian matrix as the final super point correspondence

[0098] 7. Estimate the final transformation module

[0099] The 3D transformation between the source and target point clouds is computed from the filtered superpoint matches using the RANSAC algorithm or the LGR estimation method.

[0100] That is, the RANSAC or LGR algorithm is used to calculate the final 3D transformation matrix {R, t} from the filtered matches. It is applied to the target point cloud to achieve registration of the source and target point clouds.

[0101] The rest of this embodiment is the same as any of the above embodiments 1-2, so it will not be repeated here.

[0102] Example 4:

[0103] This embodiment further optimizes any one of the above embodiments 1-3. In the training of the above model, the loss function used is mainly as follows:

[0104] Uncertaintymaskloss:

[0105]

[0106] This loss is used to train the uncertainty mask generator to capture the uncertainty of point pair matching by maximizing the predicted conditional probability. This loss uses the negative log-likelihood loss (NLL), which is based on the Laplace distribution model. represents the uncertainty of the source point cloud and the target point cloud, c i，j Represents the true source point cloud and target point cloud matching pair. i，j is the matching result predicted by the network. Here we use the predicted matching S extracted from the Gaussian correlation matrix i，j .

[0107] Overlap-aware circle loss:

[0108]

[0109] A represents the set of anchor regions in the source point cloud that have at least one matching region (positive sample) in the target point cloud. For each anchor region in A and Represents the positive and negative samples of the region in the target point cloud, is the distance of the feature, middle represent and The overlap ratio. and Represents positive and negative weights. Δ p and Δ n Set to 0.1 and 0.4. The calculation is similar.

[0110] This loss calculates the overlap between the anchor region in the source point cloud and the positive and negative samples in the target point cloud, and uses a weighting function to adjust the influence of positive and negative samples. This loss improves the matching quality by preferentially matching superpoints with higher overlap areas through overlap perception.

[0111] Point matching loss:

[0112]

[0113] Represents the true source point cloud and target point cloud matching pair. Represents the soft matching matrix u rows and v columns. and Represents unmatched pairs of points.

[0114] This loss calculates the matching error for each point, optimizing the match between the source and target point clouds.

[0115] In summary, the loss function of the model consists of the three losses mentioned above:

[0116]

[0117] represents the uncertainty mask loss mentioned above, represents the above-mentioned overlap-based circular loss, represents the above point matching loss.

[0118] The rest of this embodiment is the same as any of the above embodiments 1-3, so it will not be repeated here.

[0119] Example 5:

[0120] This embodiment is further optimized based on any one of the above embodiments 1-4. Figure 4 As shown in Figure 1, the first column is the input, the second column is the registration result, and the third column is the ground truth match. Dark black and light gray represent the source point cloud and target point cloud respectively.

[0121] In addition, if Figure 5 As shown in the figure, there are also some registration results on the real dataset (3DMatch) to verify the generalization performance of the model. The first column is the input, the second column is the registration result, and the third column is the ground truth (GroundTruth) match. Dark black and light gray represent the source point cloud and target point cloud respectively.

[0122] Finally, there are some registration results on the outdoor dataset (ETH), which can also verify the generalization performance of the model.

[0123] The rest of this embodiment is the same as any of the above embodiments 1-4, so it will not be repeated here.

[0124] Example 6:

[0125] The present invention also provides a 3D point cloud registration system based on a probability model that matches the method.

[0126] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the above-mentioned probability model-based 3D point cloud registration system.

[0127] Example 7:

[0128] The present invention also provides a computer-readable storage medium comprising instructions; when the instructions are executed on the electronic device described in the above embodiment, the electronic device executes the method described in the above embodiment. Optionally, the computer-readable storage medium may be a memory.

[0129] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0130] The memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0131] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0133] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0135] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located on a single device or distributed across multiple devices. Some or all of the modules may be selected to achieve the purpose of this embodiment based on actual needs.

[0136] In addition, the functional modules in the various embodiments of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more media that can be integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0138] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention falls within the scope of protection of the present invention.

Claims

1. A 3D point cloud registration method based on a probability model, characterized in that: The following steps are involved: Step S1, constructing the HybridMatch dataset; Step S11, constructing a background scene; Step S12, introducing foreground moving objects into the background scene to construct a mixed motion scene, wherein the foreground moving objects include rigid objects and non-rigid objects; Step S13, using a virtual camera to generate a depth map of the mixed motion scene; Step S14, converting the depth map into a point cloud to generate registration data; Step S15, dividing the registration data according to the point cloud overlap ratio to construct a HybridMatch dataset; Step S2, constructing a HybirdReg probability model, which includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud coarse matching module using the uncertainty mask to filter the point cloud, and a final transformation estimation module; Step S3, training the HybridReg probability model in the HybridMatch dataset, using a loss function to guide the HybridReg probability model, and outputting the trained HybridReg probability model; In step S4, point cloud pairs are selected from the HybridMatch dataset and input into the trained diffusion network model to predict super point pair matching. The point cloud 3D transformation matrix is ​​calculated based on the super point pair matching, and the point cloud is transformed into 3D to complete the 3D point cloud registration.

2. A 3D point cloud registration method based on a probability model according to claim 1, characterized in that: The step S2 comprises: The point cloud pair input module, point cloud downsampling and feature extraction module, geometric transformation module, point cloud patch construction module, uncertainty mask estimation module, point cloud coarse matching module using uncertainty mask filtering, and final transformation estimation module are connected in sequence from front to back; The geometric transformation module includes a self-attention module and a cross-attention module.

3. The 3D point cloud registration method based on a probability model according to claim 2, characterized in that: include: The point cloud input module inputs a point cloud pair into the point cloud downsampling and feature extraction module, wherein the point cloud pair includes a source point cloud and a target point cloud; The point cloud downsampling and feature extraction module downsamples the source point cloud and the target point cloud to obtain superpoints corresponding to the source point cloud and the target point cloud, performs feature extraction on the superpoints to obtain feature values ​​of the superpoints, upsamples the superpoints and extracts features to obtain features of the upsampled superpoints and their corresponding feature values; the point cloud downsampling and feature extraction module inputs the superpoints and the feature values ​​of the superpoints into the geometric transformation module; The geometric transformation module obtains a coarse match between the source point cloud superpoints and the target point cloud superpoints; The point cloud patch construction module upsamples the source point cloud superpoints and matches them with the source point cloud superpoints according to the nearest neighbor algorithm to form a source point cloud patch; upsamples the target point cloud superpoints and matches them with the target point cloud superpoints according to the nearest neighbor algorithm to form a target point cloud patch; The uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density; The method uses the uncertainty mask to filter the point cloud coarse matching module to obtain the final super-point correspondence; The estimated final transformation module uses the RANSAC or LGR algorithm to calculate the final 3D transformation matrix from the filtered matches and applies it to the target point cloud to achieve alignment between the source point cloud and the target point cloud.

4. The 3D point cloud registration method based on a probability model according to claim 3, characterized in that: The method for obtaining a rough match between the source point cloud and the target point cloud by the geometric transformation module includes: The geometric transformation module encodes the super-points and their eigenvalues ​​in the spatial domain, and outputs point cloud eigenvalues ​​with position information. The self-attention module performs self-attention calculation on the super-points and their eigenvalues ​​after position encoding; the cross-attention module performs cross-attention calculation on the super-points and their eigenvalues ​​calculated by the self-attention module, and finds the cross information of the source point cloud and the target point cloud by combining the features of the source point cloud and the target point cloud; the Gaussian correlation of the super-point eigenvalues ​​after the cross-attention calculation is calculated to obtain a rough match between the source point cloud and the target point cloud.

5. The 3D point cloud registration method based on a probability model according to claim 3, characterized in that: The method for the uncertainty mask estimation module to obtain the uncertainty of the prediction by estimating the conditional probability density includes: Design a network to predict the parameters of a probability distribution , where the parameters include the matching relationship between the source point cloud and the target point cloud and the uncertainty of the relationship ; By estimating the conditional probability density Get the uncertainty of the forecast, where is the matching pair of the true source point cloud and the target point cloud, are the input source point cloud and target point cloud, are the parameters of the neural network; Among them, the uncertainty of the relationship Used as an uncertainty mask to indicate the confidence of the matching pair, the variance Constrain to range , close to 1 represents high confidence matches corresponding to possible overlapping point pairs, and close to 0 represents low confidence matches corresponding to non-overlapping or foreground point pairs.

6. The 3D point cloud registration method based on a probability model according to claim 1, characterized in that: The loss function in step S3 is composed of an uncertainty mask loss function, an overlap-based circle loss function, and a point matching loss function.

7. A 3D point cloud registration system based on a probability model, characterized in that: It includes a dataset construction unit, a model construction unit, a model training unit and a registration transformation unit, wherein: A data set construction unit, configured to construct a HybridMatch data set; to construct a background scene; and to introduce foreground moving objects into the background scene to construct a hybrid motion scene, wherein the foreground moving objects include rigid objects and non-rigid objects. Used to generate a depth map of the hybrid motion scene using a virtual camera; used to convert the depth map into a point cloud to generate registration data; used to divide the registration data according to the point cloud overlap ratio to construct a HybridMatch dataset; A model construction unit is used to construct a HybirdReg probability model, which includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud coarse matching module using the uncertainty mask to filter the point cloud, and a final transformation estimation module; a model training unit, configured to train the HybridReg probability model in the HybridMatch dataset, guide the HybridReg probability model using a loss function, and output a trained HybridReg probability model; The registration transformation unit is used to select point cloud pairs in the HybridMatch dataset and input them into the trained diffusion network model, predict super point pair matching, calculate the point cloud 3D transformation matrix based on the super point pair matching, and perform 3D transformation on the point cloud to complete 3D point cloud registration.

8. An electronic device, characterized in that: It comprises a processor and a memory; the processor comprises the 3D point cloud registration system based on the probability model as described in claim 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes instructions; when the instructions are executed on the electronic device according to claim 8, the electronic device is caused to execute the method according to any one of claims 1 to 6.

Images