3D point cloud registration method and system based on probability model, equipment and medium

Abstract

The invention relates to the technical field of computer vision and computational graphics, and discloses a 3D point cloud registration method and system based on a probability model, and the method comprises the steps: building a Hybrid Match data set and building a Hybrid Reg probability model, training the Hybrid Reg probability model in the Hybrid Match data set, guiding the Hybrid Reg probability model through a loss function, and outputting the trained Hybrid Reg probability model; and selecting a point cloud pair in the Hybrid Match data set, inputting the point cloud pair into the trained diffusion network model, predicting super-point pair matching, calculating according to the super-point pair matching to obtain a point cloud 3D transformation matrix, and performing 3D transformation on the point cloud to complete 3D point cloud registration. The system comprises a data set construction unit, a model construction unit, a model training unit and a registration transformation unit. The invention further discloses electronic equipment and a computer readable storage medium. The method is used for 3D point cloud registration in a robust processing hybrid motion scene.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and computational graphics, and in particular to a 3D point cloud registration method and system, device and medium based on a probability model, which are used for robustly processing 3D point cloud registration in mixed motion scenes. Background Art

[0002] 3D point cloud registration refers to the process of aligning point cloud data collected at different times, perspectives or sensors. It is widely used in fields such as robotics, computer graphics and computer vision. In existing research, many algorithms rely on the rigid body assumption, assuming that only rigid bodies move in the entire scene. However, in actual indoor scenes, point clouds usually contain two types of motion: the first is the rigid background motion caused by the camera; the second is the non-rigid foreground motion caused by object movement or deformation.

[0003] Current point cloud registration algorithms usually ignore the impact of non-rigid foreground motion on the results, especially in partially overlapping scenes, which can easily lead to incorrect correspondences and transformation calculations. In addition, existing large-scale datasets mostly assume rigid scenes and lack large-scale datasets containing mixed motion scenes, which limits the generalization ability and robustness of the model.

[0004] Although some works have attempted to alleviate the problem of non-rigid matching by using simulated data, such as Lepard introducing an object-level non-rigid point cloud registration dataset, the simple shape prior at the object level is still significantly different from the complex indoor structures in real scenes and is difficult to generalize to complex scenes.

[0005] In addition, points with similar local geometric structures in the dynamic foreground may become outliers due to the inconsistency with the background motion, making the matching of correspondences more difficult. Therefore, the performance of existing methods will be significantly degraded when directly applied to mixed motion scenes.

[0006] In summary, the present invention provides a 3D point cloud registration method and system, device, and medium based on a probability model, which are used to process highly robust registration of complex mixed motion scenes in 3D point clouds. Summary of the invention

[0007] The present invention provides a 3D point cloud registration method and system, equipment and medium based on a probability model, which are used for processing high robustness registration of complex mixed motion scenes in 3D point clouds.

[0008] The present invention is implemented by the following technical solution: A 3D point cloud registration method based on a probability model comprises the following steps:

[0009] Step S1, constructing a HybridMatch dataset;

[0010] Step S2, constructing a HybirdReg probability model, the probability model includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud rough matching module using an uncertainty mask to filter, and a final transformation estimation module;

[0011] Step S3, training the HybirdReg probability model in the HybridMatch data set, using a loss function to guide the HybirdReg probability model, and outputting the trained HybirdReg probability model;

[0012] Step S4, selecting point cloud pairs in the HybridMatch dataset and inputting them into the trained diffusion network model, predicting super point pair matching, calculating the point cloud 3D transformation matrix based on the super point pair matching, performing 3D transformation on the point cloud to complete 3D point cloud registration.

[0013] In order to better implement the present invention, further, step S1 includes:

[0014] Step S11, constructing a background scene;

[0015] Step S12, introducing foreground moving objects into the background scene to construct a mixed moving scene, wherein the foreground moving objects include rigid objects and non-rigid objects;

[0016] Step S13, using a virtual camera to generate a depth map of the mixed motion scene;

[0017] Step S14, converting the depth map into a point cloud to generate registration data;

[0018] Step S15, dividing the registration data according to the point cloud overlap ratio to construct a HybridMatch data set.

[0019] In order to better implement the present invention, further, step S2 includes:

[0020] The point cloud pair input module, the point cloud downsampling and feature extraction module, the geometric transformation module, the point cloud patch construction module, the uncertainty mask estimation module, the point cloud rough matching module using the uncertainty mask filtering and the final transformation estimation module are sequentially connected from front to back;

[0021] The geometric transformation module includes a self-attention module and a cross-attention module.

[0022] In order to better implement the present invention, further, the point cloud input module inputs a point cloud pair into the point cloud downsampling and feature extraction module, the point cloud pair including a source point cloud and a target point cloud;

[0023] The point cloud downsampling and feature extraction module downsamples the source point cloud and the target point cloud to obtain super points corresponding to the source point cloud and the target point cloud, extracts features from the super points to obtain feature values ​​of the super points, upsamples the super points and extracts features to obtain features of the upsampled super points and their corresponding feature values; the point cloud downsampling and feature extraction module inputs the super points and the feature values ​​of the super points into the geometric transformation module;

[0024] The geometric transformation module obtains a coarse match between the source point cloud super points and the target point cloud super points;

[0025] The point cloud patch building module upsamples the source point cloud superpoints and matches them with the source point cloud superpoints according to the nearest neighbor algorithm to form a source point cloud patch; upsamples the target point cloud superpoints and matches them with the target point cloud superpoints according to the nearest neighbor algorithm to form a target point cloud patch;

[0026] The uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density;

[0027] The uncertainty mask filtering point cloud coarse matching module obtains the final super-point correspondence;

[0028] The estimated final transformation module uses the RANSAC or LGR algorithm to calculate the final 3D transformation matrix from the filtered matches and applies it to the target point cloud to achieve registration of the source point cloud and the target point cloud.

[0029] In order to better implement the present invention, further, the method for the geometric transformation module to obtain a rough match between the source point cloud and the target point cloud includes:

[0030] The geometric transformation module encodes the superpoints and their feature values ​​in the spatial domain, and outputs point cloud feature values ​​with position information. The self-attention module performs self-attention calculation on the superpoints and their feature values ​​after position encoding; the cross-attention module performs cross-attention calculation on the superpoints and their feature values ​​calculated by the self-attention module, and finds the cross information of the source point cloud and the target point cloud by combining the features of the source point cloud and the target point cloud; the Gaussian correlation of the superpoint feature values ​​after the cross-attention calculation is calculated to obtain a rough match between the source point cloud and the target point cloud.

[0031] In order to better implement the present invention, further, the method in which the uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density includes:

[0032] Design a network to predict the parameters of a probability distribution Among them, the parameters include the matching relationship between the source point cloud and the target point cloud and the uncertainty of the relationship σ 2 ;

[0033] By estimating the conditional probability density Get the uncertainty of the prediction, where is the matching pair of the real source point cloud and the target point cloud, are the input source point cloud and target point cloud, θ is the parameter of the neural network;

[0034] Among them, the uncertainty of the relationship σ 2 Used as an uncertainty mask to indicate the confidence of the matching pair, the variance σ 2 Constrained to the range [0, 1], values ​​close to 1 represent high confidence matches corresponding to possible overlapping point pairs, and values ​​close to 0 represent low confidence matches corresponding to non-overlapping or foreground point pairs.

[0035] In order to better implement the present invention, further, the loss function in step S3 is composed of an uncertainty mask loss function, an overlap-based circle loss function and a point matching loss function.

[0036] The present invention also provides a 3D point cloud registration system based on a probability model, comprising a data set construction unit, a model construction unit, a model training unit and a registration transformation unit, wherein:

[0037] Dataset construction unit, used to construct the HybridMatch dataset;

[0038] A model building unit, used to build a HybirdReg probability model, wherein the probability model includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch building module, an uncertainty mask estimation module, a point cloud coarse matching module using an uncertainty mask to filter the point cloud, and a final transformation estimation module;

[0039] A model training unit, used for training the HybirdReg probability model in the HybridMatch data set, guiding the HybirdReg probability model using a loss function, and outputting a trained HybirdReg probability model;

[0040] The registration transformation unit is used to select point cloud pairs in the HybridMatch data set and input them into the trained diffusion network model, predict super point pair matching, calculate the point cloud 3D transformation matrix according to the super point pair matching, and perform 3D transformation on the point cloud to complete 3D point cloud registration.

[0041] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the 3D point cloud registration system based on the probability model described in the second aspect above.

[0042] The present invention also provides a computer-readable storage medium, the computer-readable storage medium comprising instructions; when the instructions are executed on the electronic device described in the third aspect, the electronic device executes the method described in the first aspect.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] (1) The present invention provides a 3D point cloud registration method and system, device, and medium based on a probability model, and constructs the first large-scale indoor hybrid motion synthetic dataset, named HybridMatch. This dataset combines the rigid background of the 3D-FRONT dataset and the object-level rigid motion of the ShapeNet dataset, as well as the deformable objects in the DeformingThings4D dataset, to generate a point cloud registration dataset with diverse and complex motions;

[0045] (2) The present invention provides a 3D point cloud registration method and system, device, and medium based on a probability model, proposes a probability model named HybridReg, designs an uncertainty mask prediction mechanism, and combines the negative log-likelihood (NLL) loss to improve the robustness of feature extraction and correspondence matching;

[0046] (3) The present invention is qualitatively and quantitatively evaluated on a wide range of indoor and outdoor datasets. Experimental results show that the present invention outperforms existing methods in 3D point cloud hybrid motion registration scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention is further described in conjunction with the following drawings and embodiments, and all the concepts and innovations of the present invention should be regarded as the disclosed contents and the protection scope of the present invention.

[0048] Figure 1 A method flow chart of a 3D point cloud registration method based on a probability model and a method in a system, device, and medium provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the structure of a 3D point cloud registration method based on a probability model and a data set construction in a system, device, and medium provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of the structure of a HybirdReg probability model in a 3D point cloud registration method and system, device, and medium based on a probability model provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the effect of a 3D point cloud registration method and system, device, and medium based on a probability model provided in an embodiment of the present application;

[0052] Figure 5 Schematic diagram B of the effect of a 3D point cloud registration method and system, device, and medium based on a probability model provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0055] Embodiment 1:

[0056] A 3D point cloud registration method, system, device, and medium based on a probability model in this embodiment, such as Figure 1 As shown, after constructing the HybridMatch data set and the HybridReg probability model, the present invention trains the HybridReg probability model in the HybridMatch data set, uses the loss function to guide the HybridReg probability model, and outputs the trained HybridReg probability model; selects point cloud pairs in the HybridMatch data set and inputs them into the trained diffusion network model, predicts super point pair matching, calculates the point cloud 3D transformation matrix according to the super point pair matching, and performs 3D transformation on the point cloud to complete 3D point cloud registration.

[0057] Embodiment 2:

[0058] This embodiment is further optimized on the basis of embodiment 1. Figure 2 As shown, the first step is to build the HybridMatch dataset.

[0059] Specifically, it includes constructing a background scene, injecting foreground object motion into the background scene to form a hybrid motion scene, then using a virtual camera to generate a depth map of the hybrid motion scene, converting the depth map into a point cloud, generating registration data, and finally dividing the point cloud according to the overlap ratio to generate a HybridMatch dataset.

[0060] The background scene is being constructed:

[0061] Use the 3D-FRONT dataset to generate indoor backgrounds, which contain professionally designed room layouts and a rich variety of furniture. Use the model in ShapeNet to increase scene complexity by random scaling, rotation, and placement (such as placing on room furniture or hanging in the air). When switching camera perspectives, introduce rigid background motion.

[0062] Foreground object motion includes non-rigid objects and rigid objects:

[0063] (1) Non-rigid objects (deformable objects):

[0064] Select human and animal models and their animation sequences from the DeformingThings4D dataset. Randomly select animation sequences in scenes, each scene contains animations of a single person or multiple people / animals. Enhance the diversity of motion amplitudes by random offsets.

[0065] (2) Rigid objects (movable objects):

[0066] Place rigid models from ShapeNet around non-rigid objects (such as jm chairs, cups, etc.). Apply random XY plane offsets to simulate the movement of rigid objects, while introducing noise and small furniture movements to improve the realism of the scene. Ensure that all foreground objects avoid collisions when inserted to maintain physical plausibility.

[0067] Generation of registration data:

[0068] A virtual camera is used to generate a depth map of the constructed mixed motion scene and the depth map is converted into a point cloud. The depth range is limited and large planes (such as walls and floors) are randomly removed to highlight the motion of foreground objects. The viewing angle range is set to [0°, 360°] horizontally and [0°, 45°] vertically. The source and target views are uniformly sampled within the viewing angle range, with sampling intervals of 30° and 15°. The true matching 3D transformation matrix of the source point cloud pair and the target point cloud pair is generated based on the camera pose of the sampled source and target views.

[0069] Data partitioning:

[0070] According to the overlap ratio of point clouds, it is divided into HybridMatch (>30% overlap) and HybridLoMatch (10-30% overlap). According to the non-rigid motion ratio (the ratio of foreground object motion to the scene), it is further subdivided into 10-30% motion ratio and 30-50% motion ratio, and also includes rigid benchmark segmentation without animation. The synthesis method of the HybridMatch dataset combines rigid and non-rigid motion, and generates realistic indoor point cloud scenes with mixed motion through physical rationality and diversity design.

[0071] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0072] Embodiment 3:

[0073] This embodiment is further optimized on the basis of the above embodiment 1 or 2. Figure 3 As shown, the HybridReg method specifically includes:

[0074] 1. Point cloud input module

[0075] Input point cloud pair is the source point cloud, is the target point cloud. A neural network is used to estimate the point pair matching between the source point cloud and the target point cloud.

[0076] exist Figure 2 In the middle, the left side is the source point cloud The right side is the target point cloud At this time, the source point cloud and the target point cloud are in a state of waiting for registration.

[0077] 2. Point cloud downsampling and feature extraction module

[0078] Use downsampling to obtain superpoints, use kernel point convolution and feature pyramid network KPConv-FPN to extract point cloud features, and obtain the learning features of source point cloud and target point cloud.

[0079] Downsample the input source point cloud and target point cloud to obtain the superpoints corresponding to the source point cloud and target point cloud. and The KPConv network is used to extract the features of the super points and obtain the feature values ​​of the super points. and Upsample the downsampled superpoints and extract features to obtain and And the corresponding eigenvalues and

[0080] The eigenvalues ​​of superpoints are used for superpoint matching (rough matching of point clouds), and the eigenvalues ​​of upsampled point clouds are used for subsequent 3D transformation estimation after the upsampled point cloud and superpoints form a patch. The points obtained by downsampling the point cloud are superpoints, and the eigenvalues ​​extracted from the superpoints are sent to the geometric transformation module. The superpoints downsampled from the point cloud are then upsampled back. This step is to smooth the original point cloud and initially reduce noise and other interference, so it is upsampled back and the eigenvalues ​​of the upsampled point cloud are extracted again. The eigenvalues ​​of the upsampled point cloud are used later in estimating 3D transformations (RANSAC and LGR) (the upsampled point cloud and superpoints form a patch), so these two eigenvalues ​​(superpoint eigenvalues, upsampled point cloud eigenvalues) are different and have different functions.

[0081] The upsampled point cloud and superpoints form a point cloud patch using the nearest neighbor method, and then participate in the calculation in the final point cloud 3D transformation prediction (ransac and LGR). That is, the estimation of the 3D transformation is based on the patch composed of upsampled points.

[0082] 3. Geometric Transformer

[0083] The geometric transformation module is divided into self-attention and cross-attention modules. The self-attention module learns the global correlation within the point cloud and uses geometric structure embedding (such as distance and angle information between points) to enhance the feature expression of the point cloud. The cross-attention module comprehensively learns the correlation between the source point cloud and the target point cloud, and learns the geometric consistency between potential adjacent frames, thereby obtaining the relevant features between the source point cloud and the target point cloud.

[0084] The geometric transformation module transforms the input superpoint and and its eigenvalues and The super points are encoded in the spatial domain and point cloud feature values ​​with location information are output.

[0085] The self-attention block performs self-attention calculations on the super-points and feature values ​​after position encoding to enhance the self-representation of the features.

[0086] The cross-attention block performs cross-attention calculation on the super points and feature values ​​after self-attention calculation, combines the features of the source point cloud and the target point cloud, and finds the cross information of the source point cloud and the target point cloud.

[0087] Calculate the Gaussian Correlation of the super-point feature values ​​after the cross-attention calculation to obtain a rough match between the source point cloud and the target point cloud:

[0088] 4. Build point cloud patch module

[0089] The upsampled source point cloud and the super points obtained by downsampling the source point cloud are matched into source point cloud patches according to the nearest neighbor algorithm:

[0090] The calculation of the target point cloud is similar, and we get

[0091] 5. Uncertainty Mask Estimation Module

[0092] By estimating the conditional probability density is the matching pair of the real source point cloud and the target point cloud, are the input source point cloud and target point cloud, θ is the parameter of the neural network, and the uncertainty of the prediction is obtained. We designed a network to predict the parameters of the probability distribution The parameters include the matching relationship between the source point cloud and the target point cloud (mean μ) and the uncertainty of the relationship (variance σ 2 ).

[0093] The network corresponds to Figure 3 The uncertainty mask generator network in .

[0094] Among them, the uncertainty of the relationship (variance σ 2 ) is used as an uncertainty mask to indicate the confidence of the matching pair. We denote the variance σ 2 Constrained to the range [0, 1], values ​​close to 1 represent high confidence matches corresponding to possible overlapping point pairs, and values ​​close to 0 represent low confidence matches corresponding to non-overlapping or foreground point pairs.

[0095] 6. Use uncertainty mask to filter the point cloud coarse matching module

[0096] After extracting features from superpoints and normalizing them, the Gaussian correlation matrix S is calculated. The mask generated by the uncertainty mask generation module is used to filter the Gaussian correlation matrix S to suppress false matches caused by mixed motion. The largest N matching pairs in the filtered Gaussian matrix are selected as the final superpoint correspondences.

[0097] Calculate the source point cloud and the target point cloud patch to get the Gaussian correlation matrix (Correlation Matrix S), and use the uncertainty mask to filter the Gaussian correlation matrix. Suppress the false matching caused by mixed motion. Select the largest N matching (Top-K) pairs in the filtered Gaussian matrix as the final super-point correspondence

[0098] 7. Estimate the final transformation module

[0099] The 3D transformation between the source and target point clouds is computed from the filtered superpoint matches using the RANSAC algorithm or the LGR estimation method.

[0100] That is, the RANSAC or LGR algorithm is used to calculate the final 3D transformation matrix {R, t} from the filtered matches and applied to the target point cloud to achieve the registration of the source point cloud and the target point cloud.

[0101] The other parts of this embodiment are the same as any of the above embodiments 1-2, so they are not repeated here.

[0102] Embodiment 4:

[0103] This embodiment is further optimized on the basis of any one of the above embodiments 1-3. In the training of the above model, the loss function used is mainly as follows:

[0104] Uncertaintymaskloss:

[0105]

[0106] This loss is used to train the uncertainty mask generator, capturing the uncertainty of point pair matching by maximizing the predicted conditional probability. This loss uses the negative log-likelihood loss (NLL), which is based on the Laplace distribution model. represents the uncertainty of the source point cloud and the target point cloud, c i，j Represents the true source point cloud and target point cloud matching pair. i，j is the matching result predicted by the network. Here we use the predicted matching S extracted from the Gaussian correlation matrix i，j .

[0107] Overlap-aware circle loss:

[0108]

[0109] A represents the set of anchor regions in the source point cloud that have at least one matching region (positive sample) in the target point cloud. For each anchor region in A and Represents the positive and negative samples of the region in the target point cloud, is the distance of the feature, middle represent and The overlap ratio. and Represents positive and negative weights. Δ p and Δ n Set to 0.1 and 0.4. The calculation of is similar.

[0110] This loss calculates the overlap between the anchor point area in the source point cloud and the positive and negative samples in the target point cloud, and uses a weighting function to adjust the influence of positive and negative samples. This loss improves the matching quality by preferentially matching super points with higher overlap areas through overlap perception.

[0111] Point matching loss:

[0112]

[0113] Represents the true source point cloud and target point cloud matching pair. Represents the soft matching matrix The u rows and v columns of . and Represents unmatched pairs of points.

[0114] This loss calculates the matching error for each point, optimizing the match between the source and target point clouds.

[0115] In summary, the loss function of the model consists of the above three losses:

[0116]

[0117] represents the above uncertainty mask loss, represents the above-mentioned overlap-based circular loss, represents the above point matching loss.

[0118] The other parts of this embodiment are the same as any one of the above embodiments 1-3, so they are not repeated here.

[0119] Embodiment 5:

[0120] This embodiment is further optimized based on any one of the above embodiments 1-4. Figure 4 As shown in Figure 1, the first column is the input, the second column is the registration result, and the third column is the ground truth match. Dark black and light gray represent the source point cloud and target point cloud, respectively.

[0121] In addition, if Figure 5 As shown, there are also some registration results on the real dataset (3DMatch) to verify the generalization performance of the model. The first column is the input, the second column is the registration result, and the third column is the real (GroundTruth) match. Dark black and light gray represent the source point cloud and target point cloud respectively.

[0122] Finally, there are some registration results on the outdoor dataset (ETH), which can also verify the generalization performance of the model.

[0123] The other parts of this embodiment are the same as any one of the above embodiments 1-4, so they are not repeated here.

[0124] Embodiment 6:

[0125] The present invention also provides a 3D point cloud registration system based on a probability model that matches the method.

[0126] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the above-mentioned probability model-based 3D point cloud registration system.

[0127] Embodiment 7:

[0128] The present invention also provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the above embodiment, the electronic device executes the method described in the above embodiment. Optionally, the computer-readable storage medium may be a memory.

[0129] The processor involved in the embodiments of the present application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0130] The memory involved in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0131] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0133] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0135] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one device or distributed on multiple devices. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0136] In addition, each functional module in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0138] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A 3D point cloud registration method based on a probability model, characterized in that: The following steps are involved: Step S1, constructing a HybridMatch dataset; Step S2, constructing a HybirdReg probability model, the probability model includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch construction module, an uncertainty mask estimation module, a point cloud rough matching module using an uncertainty mask to filter, and a final transformation estimation module; Step S3, training the HybirdReg probability model in the HybridMatch data set, using a loss function to guide the HybirdReg probability model, and outputting the trained HybirdReg probability model; Step S4, selecting point cloud pairs in the HybridMatch dataset and inputting them into the trained diffusion network model, predicting super point pair matching, calculating the point cloud 3D transformation matrix based on the super point pair matching, performing 3D transformation on the point cloud to complete 3D point cloud registration.

2. A 3D point cloud registration method based on a probability model according to claim 1, characterized in that: The step S1 comprises: Step S11, constructing a background scene; Step S12, introducing foreground moving objects into the background scene to construct a mixed moving scene, wherein the foreground moving objects include rigid objects and non-rigid objects; Step S13, using a virtual camera to generate a depth map of the mixed motion scene; Step S14, converting the depth map into a point cloud to generate registration data; Step S15, dividing the registration data according to the point cloud overlap ratio to construct a HybridMatch data set.

3. The 3D point cloud registration method based on a probability model according to claim 1, characterized in that: The step S2 comprises: The point cloud pair input module, the point cloud downsampling and feature extraction module, the geometric transformation module, the point cloud patch construction module, the uncertainty mask estimation module, the point cloud rough matching module using the uncertainty mask filtering and the final transformation estimation module are sequentially connected from front to back; The geometric transformation module includes a self-attention module and a cross-attention module.

4. The 3D point cloud registration method based on a probability model according to claim 3, characterized in that: include: The point cloud input module inputs a point cloud pair into the point cloud downsampling and feature extraction module, wherein the point cloud pair includes a source point cloud and a target point cloud; The point cloud downsampling and feature extraction module downsamples the source point cloud and the target point cloud to obtain super points corresponding to the source point cloud and the target point cloud, extracts features from the super points to obtain feature values ​​of the super points, upsamples the super points and extracts features to obtain features of the upsampled super points and their corresponding feature values; the point cloud downsampling and feature extraction module inputs the super points and the feature values ​​of the super points into the geometric transformation module; The geometric transformation module obtains a coarse match between the source point cloud super points and the target point cloud super points; The point cloud patch building module upsamples the source point cloud superpoints and matches them with the source point cloud superpoints according to the nearest neighbor algorithm to form a source point cloud patch; upsamples the target point cloud superpoints and matches them with the target point cloud superpoints according to the nearest neighbor algorithm to form a target point cloud patch; The uncertainty mask estimation module obtains the uncertainty of the prediction by estimating the conditional probability density; The uncertainty mask filtering point cloud coarse matching module obtains the final super-point correspondence; The estimated final transformation module uses the RANSAC or LGR algorithm to calculate the final 3D transformation matrix from the filtered matches and applies it to the target point cloud to achieve registration of the source point cloud and the target point cloud.

5. The 3D point cloud registration method based on a probability model according to claim 4, characterized in that: The method for obtaining a rough match between the source point cloud and the target point cloud by the geometric transformation module includes: The geometric transformation module encodes the superpoints and their feature values ​​in the spatial domain, and outputs point cloud feature values ​​with position information. The self-attention module performs self-attention calculation on the superpoints and their feature values ​​after position encoding; the cross-attention module performs cross-attention calculation on the superpoints and their feature values ​​calculated by the self-attention module, and finds the cross information of the source point cloud and the target point cloud by combining the features of the source point cloud and the target point cloud; the Gaussian correlation of the superpoint feature values ​​after the cross-attention calculation is calculated to obtain a rough match between the source point cloud and the target point cloud.

6. The 3D point cloud registration method based on a probability model according to claim 4, characterized in that: The method for the uncertainty mask estimation module to obtain the uncertainty of the prediction by estimating the conditional probability density includes: Design a network to predict the parameters of a probability distribution Among them, the parameters include the matching relationship between the source point cloud and the target point cloud and the uncertainty of the relationship σ 2 ; By estimating the conditional probability density Get the uncertainty of the prediction, where is the matching pair of the real source point cloud and the target point cloud, are the input source point cloud and target point cloud, θ is the parameter of the neural network; Among them, the uncertainty of the relationship σ 2 Used as an uncertainty mask to indicate the confidence of the matching pair, the variance σ 2 Constrained to the range [0, 1], values ​​close to 1 represent high confidence matches corresponding to possible overlapping point pairs, and values ​​close to 0 represent low confidence matches corresponding to non-overlapping or foreground point pairs.

7. The 3D point cloud registration method based on a probability model according to claim 1, characterized in that: The loss function in step S3 is composed of an uncertainty mask loss function, an overlap-based circle loss function, and a point matching loss function.

8. A 3D point cloud registration system based on a probability model, characterized in that: It includes a data set construction unit, a model construction unit, a model training unit and a registration transformation unit, wherein: Dataset construction unit, used to construct the HybridMatch dataset; A model building unit, used to build a HybirdReg probability model, wherein the probability model includes a point cloud pair input module, a point cloud downsampling and feature extraction module, a geometric transformation module, a point cloud patch building module, an uncertainty mask estimation module, a point cloud coarse matching module using an uncertainty mask to filter the point cloud, and a final transformation estimation module; A model training unit, used for training the HybirdReg probability model in the HybridMatch data set, guiding the HybirdReg probability model using a loss function, and outputting a trained HybirdReg probability model; The registration transformation unit is used to select point cloud pairs in the HybridMatch data set and input them into the trained diffusion network model, predict super point pair matching, calculate the point cloud 3D transformation matrix according to the super point pair matching, and perform 3D transformation on the point cloud to complete 3D point cloud registration.

9. An electronic device, characterized in that: It comprises a processor and a memory; the processor comprises the probability model-based 3D point cloud registration system as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions; when the instructions are executed on the electronic device as claimed in claim 9, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images