A multi-view point cloud registration method and device, a storage medium, an equipment and a product

Abstract

The specification provides a multi-view point cloud registration method, device, storage medium, equipment and product. The method comprises: acquiring visible light images and depth images under multiple collection angles; inputting the visible light images into a preset three-dimensional reconstruction model to obtain initial collection poses corresponding to each collection angle in a global coordinate system and initial three-dimensional point graphs of observable scenes corresponding to each collection angle; respectively correcting the initial collection poses and the initial three-dimensional point graphs according to the measurement scales corresponding to the depth images; establishing a cross-view corresponding relationship between three-dimensional points under different collection angles based on the corrected three-dimensional point graphs; adjusting the corrected collection poses of each collection angle to obtain target collection poses, with the minimum deviation between three-dimensional points with the cross-view corresponding relationship as the target, so as to align the point clouds corresponding to each collection angle in the global coordinate system through the target collection poses and determine a point cloud registration result.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of computer technology, and in particular to a multi-view point cloud registration method, apparatus, storage medium, device and product. Background Technology

[0002] With the development of technologies such as 3D perception, augmented reality, digital twins, and 3D content reconstruction, acquiring scene data from multiple perspectives and performing point cloud registration has become a crucial foundational step in 3D scene understanding and reconstruction. Image acquisition devices can typically acquire visible light and depth images of a scene from different perspectives and reconstruct the scene's spatial structure information based on multi-view data.

[0003] However, in practical applications, existing multi-view point cloud registration methods are prone to insufficient registration accuracy due to factors such as data differences between different acquisition perspectives, occlusion, noise interference, and fluctuations in depth information quality. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a multi-view point cloud registration method is proposed, comprising: Acquire visible light and depth images from multiple acquisition perspectives; The visible light image is input into a preset three-dimensional reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial three-dimensional point map of the observable scene corresponding to each acquisition viewpoint. Based on the metric scale corresponding to the depth image, the initial acquisition pose and the initial 3D point map are scaled and corrected to obtain the corrected acquisition pose and the corrected 3D point map. Based on the corrected 3D point map, establish cross-view correspondence between 3D points under different acquisition perspectives; With the goal of minimizing the deviation between 3D points with cross-view correspondence, the corrected acquisition pose of each acquisition viewpoint is adjusted to obtain the target acquisition pose. The point cloud corresponding to each acquisition viewpoint is then aligned in the global coordinate system using the target acquisition pose to determine the point cloud registration result.

[0005] According to a second aspect of one or more embodiments of this specification, a multi-view point cloud registration apparatus is provided, comprising: The acquisition module is used to acquire visible light images and depth images from multiple acquisition perspectives; The input module is used to input the visible light image into a preset three-dimensional reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial three-dimensional point map of the observable scene corresponding to each acquisition viewpoint. The correction module is used to perform scale correction on the initial acquisition pose and the initial 3D point map according to the metric scale corresponding to the depth image, so as to obtain the corrected acquisition pose and the corrected 3D point map. The matching module is used to establish cross-view correspondence between 3D points under different acquisition perspectives based on the corrected 3D point map; The registration module is used to adjust the corrected acquisition pose of each acquisition viewpoint with the goal of minimizing the deviation between 3D points with cross-view correspondence, so as to obtain the target acquisition pose. Then, the point cloud corresponding to each acquisition viewpoint is aligned in the global coordinate system using the target acquisition pose to determine the point cloud registration result.

[0006] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor implements the steps of the method described above by executing the executable instructions.

[0007] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0008] According to a fifth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0009] As can be seen from the above embodiments, this specification combines the scene structure information contained in the visible light image with the measurement information provided by the depth image to perform scale correction on the initial acquisition pose and initial 3D point map obtained from 3D reconstruction, so that the data from different acquisition perspectives have a unified measurement basis. On this basis, a cross-view correspondence between 3D points under different acquisition perspectives is established based on the corrected 3D point map, and the acquisition pose of each acquisition perspective is adjusted by minimizing the deviation between 3D points with cross-view correspondence, thereby reducing the registration error introduced by factors such as scale inconsistency and initial estimation error, improving the alignment accuracy of point clouds in the global coordinate system under different acquisition perspectives, further improving the multi-view point cloud registration accuracy, and enhancing the consistency and reliability of multi-view scene reconstruction results. Attached Figure Description

[0010] Figure 1 This is a schematic diagram of the architecture of a point cloud registration service system provided in an exemplary embodiment; Figure 2 This is a flowchart illustrating a multi-view point cloud registration method provided in an exemplary embodiment; Figure 3 This is a flowchart illustrating a pose adjustment method as provided in an exemplary embodiment; Figure 4 This is an exemplary embodiment of a point cloud registration flowchart. Figure 5 This is a schematic diagram of the structure of a device provided in an exemplary embodiment; Figure 6 This is a block diagram of a multi-view point cloud registration device provided in an exemplary embodiment. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0012] In multi-view 3D scene reconstruction, it is often necessary to unify point clouds acquired from multiple viewpoints into the same global coordinate system to form registration results that can be used for scene understanding, spatial measurement, virtual-real fusion, or 3D modeling. Existing solutions commonly employ pairwise matching between different viewpoints, followed by global solving based on the pairwise matching results, or gradually accumulating local geometric features to estimate the pose of each viewpoint. These methods are prone to problems such as unstable initial estimation, inconsistent scale, and gradual accumulation of local errors when there are significant viewpoint differences, insufficient local texture, depth noise, or occlusion or repetitive structures in the scene, leading to a decrease in global registration accuracy.

[0013] Furthermore, when data quality fluctuates across different acquisition perspectives, relying solely on local features or single modal information for pose recovery is susceptible to the influence of regions lacking depth, reflective areas, distant regions, and sparsely observed areas. Especially in scenarios where multi-view data participates in global registration simultaneously, if the initial poses of some perspectives are deviated, or if the 3D structures recovered from different perspectives lack a unified metric, subsequent global alignment processes are prone to misalignment, drift, or ghosting of local structures, making it difficult to meet the requirements for high-precision multi-view point cloud registration.

[0014] Furthermore, performing full point-level matching directly across all views typically involves high computational complexity and introduces numerous irrelevant candidate points, easily leading to noisy correspondences that interfere with subsequent pose adjustments. Therefore, effectively utilizing structural information from visible light images and metric information from depth images in multi-view scenarios to obtain initial results at a unified scale, and then establishing more reliable cross-view correspondences based on these results, is a crucial issue for improving the accuracy of multi-view point cloud registration.

[0015] Based on this, this specification provides a multi-view point cloud registration method. The method first acquires visible light and depth images from multiple acquisition viewpoints, and inputs the visible light images into a preset 3D reconstruction model to obtain the initial acquisition pose and initial 3D point map corresponding to each acquisition viewpoint in the global coordinate system. Then, combining the metric scale corresponding to the depth image, scale correction is performed on the initial acquisition pose and initial 3D point map to ensure that the data from different acquisition viewpoints have a unified metric basis. Next, based on the corrected 3D point map, a cross-view correspondence is established between 3D points from different acquisition viewpoints. With the goal of minimizing the deviation between 3D points with cross-view correspondence, the corrected acquisition pose of each acquisition viewpoint is adjusted. Finally, the point clouds corresponding to each acquisition viewpoint are aligned in the global coordinate system. This method can reduce the impact of scale inconsistency and initial estimation bias on the registration results, improving the alignment accuracy and overall consistency of multi-view point clouds in the global coordinate system.

[0016] To facilitate understanding of the technical solutions in this specification, some key concepts involved in this specification are explained below.

[0017] Visible light images: Color or grayscale images of a scene acquired by the visible light imaging module in a camera or depth camera at various acquisition viewpoints. These visible light images are used to characterize the texture, edges, structural contours, and visual relationships between viewpoints of the scene. In this specification, visible light images are primarily used as input for a preset 3D reconstruction model to recover the initial acquisition pose and initial 3D point map corresponding to each acquisition viewpoint.

[0018] Depth images: Depth images are depth observation results acquired by structured light cameras, binocular depth estimation modules, time-of-flight sensors, or other depth sensing devices, used to characterize the distance information corresponding to the position of each pixel in a scene. In this specification, depth images are mainly used to provide metric scale information for scale correction of the initial acquisition pose and initial 3D point map.

[0019] Initial Acquisition Pose: The initial spatial pose of each acquisition viewpoint in the global coordinate system, reconstructed from visible light images from multiple acquisition viewpoints. This initial acquisition pose may include rotation and translation parameters. In some implementations, the acquisition pose typically corresponds to the pose of the image acquisition device; the rotation parameters characterize the orientation of the acquisition device, and the translation parameters characterize the position of the acquisition device in the global coordinate system. Since the result reconstructed solely from visible light images may contain scale ambiguities, this initial acquisition pose usually requires further scale correction.

[0020] Initial 3D point map: A representation of the scene's 3D structure reconstructed from visible light images from multiple acquisition perspectives. This initial 3D point map corresponds to a dense or semi-dense set of 3D points in the observable scene from each acquisition perspective, used to characterize the 3D coordinate distribution at various locations within the scene. In different implementations, this initial 3D point map can be a per-view point map or a global set of 3D points associated with each viewpoint observation. In subsequent spatial matching and registration processing, the 3D point map can be converted into a 3D point cloud for further processing.

[0021] Cross-viewpoint correspondence: A correspondence between 3D points acquired from different viewpoints that satisfies spatial proximity or geometric consistency. In real-world 3D scenes, 3D points at the same spatial location within the scene often exhibit correspondences from different viewpoints.

[0022] Point cloud registration result: After the acquisition pose adjustment, the point clouds corresponding to each acquisition viewpoint are aligned in the global coordinate system. This result can be a set of point clouds in a unified coordinate system, or a scene 3D model, surface reconstruction result, or spatial mapping result further formed from this point cloud set.

[0023] The technical solutions described in the various embodiments of this specification will be explained in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a schematic diagram of the architecture of a point cloud registration service system provided in an exemplary embodiment. Figure 1 As shown, the system may include a server 11, a network 12, and several electronic devices, such as a personal computer (PC) 13, a mobile phone 14, etc.

[0025] Server 11 can be a physical server containing an independent host, or it can be a virtual server hosted by a host cluster. During operation, server 11 can run server-side programs for a certain application to implement the relevant functions of that application. For example, when server 11 runs a point cloud registration service program, it can become a corresponding point cloud registration service platform.

[0026] PC13 and mobile phone14 are just some of the types of electronic devices that users can use. In reality, users can obviously also use electronic devices such as tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments in this specification do not limit this. During operation, the electronic device can run a client-side program of an application to implement the relevant functions of that application. For example, when the electronic device runs a point cloud registration service program, it can implement a client for that point cloud registration service. The aforementioned point cloud registration service client application can be launched and run on the electronic device. This client-side program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. Of course, when using web technologies such as HTML5 or similar, the relevant functions can be implemented through a page displayed by a browser. This browser can be a standalone browser application or a browser module embedded in some applications.

[0027] As for the network 12 that enables interaction between electronic devices such as PC13 and mobile phone 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, PC13 can support both wired and wireless communication, so it can use either wired or wireless networks as needed. Mobile phone 14 typically only supports wireless communication, so it can use a wireless network for communication.

[0028] In this specification, the execution subject used to perform the point cloud registration method can be a designated device such as a server, or it can be a device such as a mobile phone or computer and a client installed on these devices. For ease of description, the following will use a server as the execution subject to explain the multi-view point cloud registration method provided in this specification.

[0029] Figure 2 This is a flowchart illustrating a multi-view point cloud registration method provided in an exemplary embodiment, including the following steps: S200: Acquire visible light and depth images from multiple acquisition perspectives.

[0030] The server can acquire visible light images and depth images from multiple acquisition perspectives from image acquisition devices, mobile terminals, depth camera arrays, 3D scanning devices, or data storage systems. Multiple acquisition perspectives can be a set of perspectives acquired from different positions and orientations around the same scene. Correspondingly, each acquisition perspective typically includes at least one visible light image and a corresponding depth image in time and space.

[0031] In practical applications, servers can directly acquire raw data or pre-processed image data. For example, visible light images can undergo distortion correction, resolution unification, color normalization, or exposure adjustment; depth images can undergo hole filling, outlier filtering, scale unit unification, or effective region selection. These processes improve the stability of subsequent 3D reconstruction and scale correction processes.

[0032] It should be noted that the multiple acquisition perspectives in this step are not required to be arranged in chronological order, nor are they required to have a fixed trajectory relationship. In an unordered scanning scenario, the image data acquired by the server from multiple acquisition perspectives can be an unordered set; in an ordered scanning scenario, the image data acquired by the server from multiple acquisition perspectives can also be arranged in chronological order of acquisition. Regardless of whether they are ordered, as long as these acquisition perspectives correspond to all or part of the same target scene, the method in this specification is applicable.

[0033] For example, in an indoor room scanning scenario, a user can hold a terminal with a depth module and take pictures at multiple locations in the room, and the server can acquire the visible light images and depth images corresponding to these locations; in a workpiece 3D measurement scenario, multiple fixed cameras can simultaneously acquire visible light images and depth images of the workpiece from different directions, and the server can also perform subsequent steps based on this data.

[0034] S202: Input the visible light image into the preset 3D reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial 3D point map of the observable scene corresponding to each acquisition viewpoint.

[0035] After the server acquires visible light images from multiple acquisition perspectives, these images can be input into a preset 3D reconstruction model. This 3D reconstruction model can be a model with multi-view geometric recovery capabilities, such as a Visual Geometry Grounded Transformer (VGGT), Pi3, or a depth estimation model (DepthAnything), used to recover the 3D structure of the scene and the spatial pose of each acquisition perspective in the global coordinate system based on images from multiple acquisition perspectives.

[0036] In one embodiment, the 3D reconstruction model can be a pre-trained model built upon visual geometric priors, used to directly output the initial acquisition poses and initial 3D point maps corresponding to multiple acquisition viewpoints through a single or minimal forward inference. This model can employ a multi-view feature encoding structure, a cross-view feature fusion structure based on an attention mechanism, a 3D recovery structure based on geometric consistency constraints, or other model structures capable of recovering scene geometric relationships from multi-view images. For consistency of terminology, the term "3D reconstruction model" will be used uniformly in the following description.

[0037] During the training phase, sample visible light images with acquisition pose labels and 3D structure labels can be input into the model to be trained to obtain prediction results. The loss value is determined based on the difference between the prediction results and the corresponding labels. The model parameters are iteratively updated based on the loss value until the preset training termination condition is met, and the trained 3D reconstruction model is obtained.

[0038] The initial acquisition pose, located in the global coordinate system, can be understood as the initial estimation result of the position and orientation of each acquisition viewpoint by the 3D reconstruction model within a unified reference coordinate framework. The initial 3D point map can be understood as the 3D structural representation of the scene reconstructed by the model based on observations from multiple acquisition viewpoints. This initial 3D point map can output a set of 3D points for each acquisition viewpoint, or it can be used to create a corresponding point map representation based on the observable area of ​​each acquisition viewpoint.

[0039] In one embodiment provided in this specification, the server can input all visible light images into the three-dimensional reconstruction model at once, and the three-dimensional reconstruction model outputs the initial acquisition pose and initial three-dimensional point map corresponding to all acquisition viewpoints.

[0040] In another embodiment provided in this specification, the server may also divide the visible light images from multiple acquisition perspectives into multiple batches of input 3D reconstruction models, and then map the output results of each batch to the same global coordinate system.

[0041] It should be noted that although the initial acquisition pose and initial 3D point map output by the 3D reconstruction model have global coordinate consistency, their scale is essentially a predicted scale rather than a true measurement scale. In other words, the model can recover the relative distance relationships between viewpoints and the overall structural relationships of the scene relatively well, but the absolute scale may still differ between different scenes or different batches of data.

[0042] S204: Based on the metric scale corresponding to the depth image, scale correction is performed on the initial acquisition pose and the initial 3D point map to obtain the corrected acquisition pose and the corrected 3D point map.

[0043] Since initial pose and 3D point maps obtained solely from visible light images are generally better at recovering relative geometry, they may lack a precise metric scale. Depth images, on the other hand, provide a metric basis corresponding to actual distances. Therefore, the server can use the metric information reflected in the depth images to scale the initial results output by the 3D reconstruction model. By combining the two, subsequent cross-view registration can be established on a more accurate 3D spatial basis.

[0044] The server can determine the scale correction parameters based on the deviation between the predicted scale corresponding to the initial 3D point map and the metric scale reflected by the depth image; then, based on the scale correction parameters, the initial acquisition pose and the initial 3D point map are scaled.

[0045] Specifically, the server can first determine the metric 3D points corresponding to the depth images from each acquisition viewpoint. These metric 3D points can be obtained by performing 3D backprojection of the pixels in the depth image in conjunction with camera intrinsic parameters. For example, for any valid pixel in the depth image, the server can recover the 3D coordinates of the pixel in the camera coordinate system based on the pixel's depth value and the camera intrinsic parameter matrix, thereby obtaining the set of metric 3D points. If the depth image has already undergone camera calibration and distortion correction, the backprojection process can directly use the calibration parameters; if not, calibration correction can be performed before this step.

[0046] After obtaining the 3D measurement points, the server can associate these points with corresponding 3D points in the initial 3D point image. This association can be established based on the same pixel position under the same acquisition viewpoint, or based on known image coordinate mapping relationships. Subsequently, the server can calculate the ratio of the coordinate norm of each 3D measurement point to the coordinate norm of the corresponding 3D point in the initial 3D point image. Since this ratio reflects the proportional relationship between the predicted scale and the true measurement scale, the server can determine the scale correction parameters based on these ratios.

[0047] In practical applications, depth images may contain local noise, outlier depths, edge region errors, or reflection region errors. Using the median can reduce the impact of outliers on the overall scale estimation results and make the scale correction parameters more stable. Therefore, the server can determine the scale correction parameters based on the median of each ratio.

[0048] Of course, the scaling correction parameters can also be determined based on the average of the ratios, weighted statistics, or other statistical results.

[0049] After obtaining the scale correction parameters, the server can use these parameters to perform scale correction on the initial acquired pose and the initial 3D point map.

[0050] The correction of the initial acquisition pose can primarily be applied to its translation parameters, since rotation parameters are usually scale-independent; the correction of the initial 3D point map, on the other hand, can directly apply to the 3D point coordinates. In other words, the server can uniformly scale and adjust the translation parameters in the initial acquisition pose and the 3D point coordinates in the initial 3D point map to obtain the corrected acquisition pose and the corrected 3D point map.

[0051] For example, if the scale correction parameter estimated from the depth image is greater than 1, it means that the predicted scale output by the 3D reconstruction model is relatively small. In this case, the server can increase the translation amount of each acquisition viewpoint and the corresponding 3D point coordinates according to the scale correction parameter. If the scale correction parameter is less than 1, it means that the predicted scale is relatively large. In this case, the server can reduce the translation amount and 3D point coordinates accordingly.

[0052] This step enables the spatial structures reconstructed from different acquisition perspectives to have a unified measurement scale, laying the foundation for establishing more accurate cross-perspective correspondences in the future.

[0053] S206: Establish cross-view correspondence between 3D points under different acquisition perspectives based on the corrected 3D point map.

[0054] After obtaining the corrected 3D point map, the server can establish a cross-viewpoint correspondence at the 3D point level between different acquisition perspectives. Unlike correspondence based solely on 2D image features, this step establishes a spatially comparable 3D point correspondence in a global coordinate system. Since the 3D point maps corresponding to each acquisition perspective have been scale-corrected, 3D points in the same scene area observed from different perspectives are more spatially approachable, thus facilitating the establishment of correspondences.

[0055] In one embodiment, for each acquisition viewpoint, the server can use the 3D points in the corrected 3D point map corresponding to that acquisition viewpoint as query points, and search for spatially neighboring points in the corrected 3D point maps corresponding to other acquisition viewpoints; then, based on the spatial distance between the query point and the spatially neighboring points, the cross-viewpoint correspondence between 3D points under different acquisition viewpoints is determined.

[0056] In this context, spatially neighboring points can be understood as candidate 3D points located near the query point in the global coordinate system. The server can determine whether a candidate 3D point belongs to the spatially neighboring points based on Euclidean distance. For example, when the distance between a candidate 3D point and the query point is less than a preset distance threshold, it can be regarded as a spatially neighboring point; in another implementation, the server can also select one or more points with the smallest distance to the query point from multiple candidate 3D points as spatially neighboring points.

[0057] To improve search efficiency, in one embodiment, the server can discretize the corrected 3D point maps corresponding to each acquisition viewpoint into a voxel grid in a global coordinate system. A voxel grid can be understood as a set of spatial units formed by regularly dividing 3D space, with each voxel region corresponding to a spatial range. After assigning each 3D point to its corresponding voxel region, for each query point, the server can prioritize searching for candidate 3D points in other acquisition views within the voxel region where the query point is located and its neighboring voxel regions, without needing to perform a full traversal of all 3D points.

[0058] The neighboring voxel region can be a voxel region that shares edges, faces, or vertices with the voxel region where the query point is located, or it can be a voxel region within a preset number of layers surrounding the voxel region. For example, the server can search for candidate points in the voxel region where the query point is located and in one layer of voxels around it, or it can dynamically determine the range of the neighboring voxel region based on the scene scale, point cloud density, or a preset search radius.

[0059] The server can then determine the cross-view correspondence based on the spatial distance between the query point and the candidate 3D point. For example, if the distance between the query point and a candidate 3D point is the smallest and the distance is less than a preset distance threshold, the cross-view correspondence between the two can be considered a valid correspondence; if there are no candidate 3D points that meet the conditions in the nearby area, it can be considered that no valid correspondence has been found for the query point in the current other acquisition views.

[0060] In practical applications, this correspondence can be represented by ordered point pairs, that is, for query point A, its correspondence with 3D point B can be represented as (A, B). Considering the efficiency of correspondence search, each query point can have a correspondence with only one nearest neighbor 3D point; of course, if multiple candidate matching is allowed, each query point can also have a correspondence with multiple 3D points that meet the spatial distance threshold, thereby further improving the registration accuracy.

[0061] Optionally, when establishing cross-view correspondence, the server can also introduce additional filtering conditions. For example, it can require that corresponding points come from different acquisition viewpoints, require that the normal consistency of corresponding points meets a threshold condition, require that the difference in observation angle of corresponding points does not exceed a preset range, or require that corresponding points meet visibility constraints in the image space.

[0062] S208: With the goal of minimizing the deviation between 3D points with cross-view correspondence, the corrected acquisition pose of each acquisition view is adjusted to obtain the target acquisition pose. The point cloud corresponding to each acquisition view is aligned in the global coordinate system using the target acquisition pose to determine the point cloud registration result.

[0063] After establishing cross-viewpoint correspondences, the server can further adjust the corrected acquisition poses of each acquisition viewpoint based on these correspondences, and determine the adjusted acquisition poses as the target acquisition poses.

[0064] In practical applications, if two 3D points from different acquisition perspectives correspond to the same scene location, ideally, these two points should be as close as possible in the global coordinate system. Therefore, the server can construct an optimization target around the deviation between 3D points with cross-view correspondence, and by adjusting the acquisition pose of each acquisition perspective, minimize these deviations, thereby achieving more accurate multi-view point cloud alignment.

[0065] The server can calculate the deviation between 3D points with cross-view correspondences based on the corrected poses of each current acquisition viewpoint. This deviation can be the spatial distance between the corresponding 3D points in the global coordinate system, or the norm of their coordinate difference vector. The server can calculate a deviation value for each cross-view correspondence.

[0066] Since cross-viewpoint correspondences with larger deviations are more susceptible to noise, mismatches, local occlusion, or depth anomalies, if such correspondences account for a large proportion in the pose adjustment process, they can easily cause the obtained pose adjustment increments to deviate from the actual adjustment direction, thus affecting the stability and accuracy of the overall pose solution. Conversely, cross-viewpoint correspondences with smaller deviations generally have higher reliability. Therefore, the weights of each correspondence can be determined based on the deviations between corresponding 3D points, and the pose adjustment increments corresponding to each acquisition viewpoint can be determined based on the weights of each correspondence.

[0067] The deviation and weight are negatively correlated; that is, the larger the deviation, the smaller the weight of the correspondence, and vice versa. This weight is used to characterize the influence of the corresponding cross-view correspondence when calculating the pose adjustment increment.

[0068] For example, for two cross-view correspondences involved in pose adjustment under the same acquisition viewpoint, if the deviation between corresponding 3D points in the first cross-view correspondence is 0.01 meters and the deviation between corresponding 3D points in the second cross-view correspondence is 0.10 meters, the server can assign a larger weight to the first cross-view correspondence and a smaller weight to the second cross-view correspondence. In this case, the first cross-view correspondence contributes more to the translation and rotation adjustments of that acquisition viewpoint, while the second cross-view correspondence has a smaller pulling effect on the pose adjustment increment. In other words, the final determined pose adjustment increment will be updated more in the direction of reducing the deviation of the first cross-view correspondence, while the abnormal adjustment trend caused by the second cross-view correspondence will be weakened, thereby improving the stability of the pose adjustment result.

[0069] In practical implementation, the server can map deviations to weights based on a preset mapping function. For example, a function that monotonically decreases as the deviation increases can be used, or a piecewise function, an exponential decay function, or a bounded proportional function can be used to suppress the correspondence of large deviations.

[0070] After obtaining the weights of each correspondence, the server can further determine the pose adjustment increment corresponding to each acquisition viewpoint. This pose adjustment increment can include rotation adjustment increments and translation adjustment increments, used to make small updates to the current acquisition pose. The specific solution method can employ linear approximation, least squares solution, iterative optimization solution, or other pose optimization methods. In one implementation, the server can represent the pose increment as a six-dimensional parameter, where three dimensions correspond to rotation adjustment and three dimensions correspond to translation adjustment; in another implementation, a combination of rotation matrix increments, quaternion increments, and translation vectors can also be used for the solution.

[0071] Figure 3 This is a flowchart of a pose adjustment provided in an exemplary embodiment.

[0072] This specification describes a multi-iterative approach to adjusting the acquisition poses of each acquisition viewpoint. In each iteration, the deviation between 3D points with cross-viewpoint correspondences is calculated based on the current acquisition poses of each viewpoint. The weights of each correspondence and the pose adjustment increment for each viewpoint are then determined based on this deviation. After obtaining the pose adjustment increment, the server can adjust the current acquisition poses of each viewpoint to obtain the target acquisition pose. The target acquisition pose can then be used as the input pose for the next iteration to recalculate the 3D point deviations under the cross-viewpoint correspondences and continue updating the pose.

[0073] In this way, the server can repeatedly execute the deviation calculation, weight determination, pose adjustment increment determination, and pose acquisition adjustment process to perform multiple iterations of the acquired pose until the preset stopping condition is met.

[0074] In this specification, the preset stopping condition can be set in several ways. For example, the iteration can be stopped when the pose adjustment increment of N consecutive iterations is less than a preset increment threshold; the iteration can also be stopped when the decrease in the objective function is less than a preset change threshold; in addition, the iteration can also end when the preset maximum number of iterations is reached.

[0075] In a further implementation, after reaching a preset number of iterations, the server can divide the cross-view correspondence into interior and exterior points based on the current optimization state. Specifically, the server can determine cross-view correspondences where the deviation between corresponding 3D points is greater than a preset first deviation threshold as exterior point correspondences, and determine cross-view correspondences other than exterior point correspondences as interior point correspondences.

[0076] In practical applications, although cross-viewpoint correspondences are formally established, some 3D points in these correspondences may not truly correspond to the same scene spatial location, or they may fail to maintain good spatial consistency due to factors such as noise, occlusion, and mismatches. This results in significant deviations between corresponding 3D points and low reliability. Therefore, these cross-viewpoint correspondences can be defined as exterior point correspondences. Conversely, cross-viewpoint correspondences with smaller deviations between corresponding 3D points and a higher probability of corresponding to the same scene spatial location can be defined as interior point correspondences. The server can then further adjust the current acquisition pose of each acquisition viewpoint based on these interior point correspondences.

[0077] Based on this, the aforementioned preset stopping condition can be: the deviation between corresponding 3D points in the interior point correspondence is less than a second deviation threshold. This second deviation threshold is less than the first deviation threshold. In other words, the server first uses a wider first deviation threshold to eliminate exterior point correspondences with significantly larger deviations, and then uses a stricter second deviation threshold to determine whether the current interior point correspondence has reached a high level of accuracy. If the deviation in the interior point correspondence is less than the second deviation threshold, it indicates that the current multi-view point cloud has reached a relatively accurate alignment state, and iteration can stop.

[0078] During pose adjustment, the server can also employ different deviation constraint parameters at different iteration rounds, with these parameters gradually decreasing as the number of iterations increases. This progressively constrains the deviations between 3D points with cross-view correspondences. These deviation constraint parameters are used to reduce the impact of large deviations in cross-view correspondences on the pose adjustment process.

[0079] This approach can be understood as an optimization strategy that progresses from lenient to stringent: in the early stages of iteration, the deviation constraint parameter is relatively large, and the server retains a certain tolerance for correspondences with moderate deviations in order to avoid prematurely eliminating potential valid correspondences due to the initial poses not being fully aligned; as the iteration progresses, the deviation constraint parameter gradually decreases, and the server's suppression effect on correspondences with large deviations is enhanced, thereby gradually shifting the optimization focus to reliable correspondences with smaller deviations.

[0080] For example, in a large indoor scene, after the server establishes cross-view correspondences based on the scale-corrected 3D point map in the initial stage, some correspondences may still have significant deviations due to occlusion or local repetitive structures. If the server adopts strict constraints from the beginning, these large deviations may lead to unstable optimization. However, by gradually reducing the deviation constraint parameters, coarse alignment can be achieved first using the overall spatial structure, and then the constraints on reliable correspondences can be gradually strengthened in subsequent iterations, thus making the final result more accurate.

[0081] After completing the above pose adjustment, the server can perform a unified coordinate transformation on the point cloud corresponding to each acquisition viewpoint based on the target acquisition pose, and complete the alignment in the global coordinate system, thereby determining the point cloud registration result.

[0082] Since the acquisition poses corresponding to each acquisition viewpoint have been adjusted, the spatial position and orientation of the point clouds acquired from each acquisition viewpoint will also change when mapped to the global coordinate system. This will cause the point clouds from multiple views that were originally inconsistent due to offset, misalignment or local overlap to gradually become consistent, and the point clouds corresponding to the same scene spatial position can be more accurately aligned in the global coordinate system.

[0083] It should be added that, in this specification, 3D point maps and point clouds can represent the same set of 3D spatial point data. A 3D point map can be 3D point data organized according to the pixel arrangement of an image, where each pixel position corresponds to a 3D point coordinate; a point cloud can be a set of 3D spatial points formed by this 3D point data. Therefore, a 3D point map can be used directly as a point cloud, or it can be converted into a point cloud after removing the pixel index relationship and retaining the spatial coordinates of each 3D point. In other words, a 3D point map can be understood as an organization and expression form of a point cloud, and both correspond to the same spatial content when representing the 3D geometric information of a scene. Based on this, in the subsequent global coordinate mapping, cross-viewpoint correspondence construction, and pose optimization processes, the 3D point maps obtained from each acquisition viewpoint can be converted into the corresponding point clouds for processing, or they can remain in point map form for processing, as long as they can represent the 3D spatial point distribution of the scene under each viewpoint.

[0084] Based on this, the server can obtain the registration results of multi-view point clouds in a unified coordinate system. These point cloud registration results can be applied to various 3D scene processing tasks. For example, in immersive AR / VR systems, they can be used for real-time multi-view 3D scene reconstruction and alignment to improve spatial consistency between virtual content and real scenes; in embodied intelligence scenarios, they can be used for environmental perception and spatial understanding during robot navigation to assist robots in localization, obstacle avoidance, and path planning; in digital twin scenarios, they can be used for precise 3D digitization of large scenes such as factories and buildings; in 3D environment reconstruction scenarios, they can be used for 3D reconstruction of indoor scenes, outdoor scenes, and object-level scenes; in autonomous driving scenarios, they can serve as foundational data for multi-sensor fusion and high-precision map construction; in cultural heritage digitization scenarios, they can be used for high-precision 3D scanning and reconstruction of cultural relics or buildings; and in industrial inspection scenarios, they can be used for multi-view 3D measurement and defect detection of manufactured parts.

[0085] The above combination Figure 2 This document describes the multi-view point cloud registration method provided. It should be noted that... Figure 2 The order of steps in this example is only an illustrative illustration. Without affecting the overall technical logic, the execution order of each step can be adjusted, and some steps can be executed in parallel or split into multiple sub-steps.

[0086] For example, in some embodiments, after acquiring visible light and depth images from multiple acquisition perspectives, the server can first perform quality screening on the image data, and then input the screened visible light images into the 3D reconstruction model; in other embodiments, the server can also simultaneously calculate some deviation values ​​when establishing cross-view correspondence, so as to prepare data in advance for subsequent pose adjustment. These changes do not affect the implementation of the technical solution in this specification.

[0087] To facilitate understanding, this manual provides an overall flowchart for point cloud registration, such as... Figure 4 As shown.

[0088] Figure 4 This is an exemplary embodiment of a point cloud registration flowchart.

[0089] In this process, after the visible light image is input into the 3D reconstruction model, an initial acquisition pose and an initial 3D point map can be obtained. Simultaneously, the depth image provides metric scale information and, together with the initial acquisition pose and the initial 3D point map, participates in scale correction to obtain a corrected acquisition pose and a corrected 3D point map. Then, based on the corrected 3D point map, a cross-viewpoint correspondence is established, and the point clouds from multiple acquisition perspectives are aligned using the corrected acquisition pose to ultimately determine the registration result.

[0090] The methods described in this manual will be illustrated below using a more specific scenario.

[0091] In one embodiment of indoor environment scanning, a user uses a mobile terminal with depth sensing capabilities to take multi-view photos around the room, and the server acquires visible light images and depth images from multiple acquisition perspectives. Subsequently, the server inputs the visible light images into a 3D reconstruction model to obtain the initial acquisition pose and initial 3D point map in the global coordinate system. Since the output of the 3D reconstruction model may have prediction scale bias, the server obtains the metric 3D points for each acquisition perspective based on backprojection of the depth images, and obtains the scale correction parameter by calculating the median ratio of the norm of the metric 3D point coordinates to the norm of the corresponding 3D point coordinates in the initial 3D point map. Based on this scale correction parameter, the server uniformly corrects the translation parameters in the initial acquisition pose and the 3D point coordinates in the initial 3D point map to obtain the corrected acquisition pose and corrected 3D point map.

[0092] Next, the server discretizes the corrected 3D point maps corresponding to each acquisition viewpoint into a voxel mesh in the global coordinate system. For each query point, it searches for candidate 3D points in other acquisition views within the voxel region where the query point is located and its neighboring voxel regions, establishing cross-view correspondences based on spatial distance. Then, the server calculates the deviation between corresponding 3D points based on the current acquisition pose, determines the weight of each correspondence based on the deviation, and solves for the pose adjustment increment of each acquisition viewpoint, thereby updating the current acquisition pose. After multiple iterations, when a preset iteration number is reached, the server discards or downweights cross-view correspondences with deviations greater than a preset first deviation threshold as out-of-field correspondences, continuing optimization only using interior correspondences. When the deviation between corresponding 3D points in an interior correspondence is less than a second deviation threshold, the iteration ends, and the final point cloud registration result is output.

[0093] In another embodiment of object-level 3D scanning, multiple fixed acquisition devices simultaneously acquire visible light and depth images of the target object from different orientations. Since the surface of the target object may have localized areas with weak texture, relying solely on visible light images can easily lead to scale instability or structural restoration deviations at certain viewpoints. This specification improves the alignment accuracy of point clouds at different orientations in object edges, recessed areas, and areas with insufficient texture by introducing a metric scale from the depth image to correct the initial results and by performing joint pose adjustment through minimizing cross-viewpoint 3D point deviation.

[0094] Figure 5 This is a schematic structural diagram of a device provided in an exemplary embodiment. For example... Figure 5As shown, device 500 mainly consists of a communication interface 502, a user interface 504, a processor 506, and a data storage 508. These components are interconnected and communicate with each other via a system bus, network, or other connection mechanism 510. The communication interface 502 enables device 500 to communicate with other devices, access networks, and transmission networks via analog or digital modulation. For example, the communication interface 502 may include a chipset and antenna for wireless communication with a radio access network or access point. Furthermore, the communication interface 502 can be a wired interface such as Ethernet, Token Ring, or a USB port, or a wireless interface such as Wi-Fi, Bluetooth, Global Positioning System (GPS), or a wide-area wireless interface (e.g., WiMAX or LTE). Of course, the communication interface 502 can also support other forms of physical layer interfaces and standard or proprietary communication protocols. The communication interface 502 may also include multiple physical communication interfaces, such as Wi-Fi interfaces, Bluetooth interfaces, and wide-area wireless interfaces.

[0095] User interface 504 includes receiving user input and providing output to the user. Therefore, user interface 504 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, trackball, joystick, microphone, still camera, and video camera, and output components such as a display screen (which may be combined with a touch-sensitive panel), CRT, LCD, LED, display using DLP technology, printer, and other similar devices known or developed in the future. User interface 504 may also generate auditory output via speakers, speaker jacks, audio output ports, audio output devices, headphones, and other similar devices known or developed in the future. In some embodiments, user interface 504 may include software, circuitry, or other forms of logic capable of transmitting and receiving data from external user input / output devices. Additionally or alternatively, device 500 may support remote access from other devices via communication interface 502 or another physical interface (not shown). User interface 504 may be configured to receive user input, the position and movement of which may be indicated by indicators or cursors described herein. User interface 504 may also be configured as a display device for rendering or displaying text fragments.

[0096] Processor 506 may contain one or more general-purpose processors and / or special-purpose processors.

[0097] Data storage 508 may include one or more volatile and / or non-volatile storage components and may be integrated wholly or partially with processor 506. Data storage 508 may include removable and non-removable components.

[0098] Processor 506 is capable of executing program instructions 518 (e.g., compiled or uncompiled program logic and / or machine code) stored in data storage 508 to perform the various functions described herein. Data storage 508 may contain a non-transitory computer-readable medium on which program instructions are stored, which, when executed by device 500, enable device 500 to perform any methods, processes, or functions disclosed in this specification and / or the accompanying drawings. Execution of program instructions 518 by processor 506 may result in processor 506 using data 512.

[0099] For example, program instructions 518 may include an operating system 522 (e.g., an operating system kernel, device drivers, and / or other modules) installed on device 500 and one or more applications 520 (e.g., a browser, social application, or game application). Similarly, data 512 may include operating system data 516 and application data 514. Operating system data 516 is primarily accessible to the operating system 522, while application data 514 is primarily accessible to one or more applications 520. Application data 514 may reside in a file system visible or hidden from the user of device 500.

[0100] Application 520 can communicate with operating system 522 through one or more application programming interfaces (APIs). These APIs help application 520 read and / or write application data 514, transmit or receive information via communication interface 502, receive or display information on user interface 504, etc.

[0101] In some terminology, application 520 may be simply referred to as "app". Furthermore, application 520 can be downloaded to device 500 through one or more online app stores or app markets. However, applications can also be installed on device 500 in other ways, such as through a web browser or a physical interface on device 500 (e.g., a USB port).

[0102] Please refer to Figure 6 Multi-view point cloud registration devices can be applied to, for example... Figure 5 The device shown implements the technical solution described in this specification. The multi-view point cloud registration device may include: The acquisition module 600 is used to acquire visible light images and depth images from multiple acquisition perspectives; The input module 602 is used to input the visible light image into a preset three-dimensional reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial three-dimensional point map of the observable scene corresponding to each acquisition viewpoint. The correction module 604 is used to perform scale correction on the initial acquisition pose and the initial three-dimensional point map according to the metric scale corresponding to the depth image, so as to obtain the corrected acquisition pose and the corrected three-dimensional point map. Matching module 606 is used to establish cross-view correspondence between three-dimensional points under different acquisition perspectives based on the corrected three-dimensional point map; The registration module 608 is used to adjust the corrected acquisition pose of each acquisition viewpoint with the goal of minimizing the deviation between three-dimensional points with cross-view correspondence, so as to obtain the target acquisition pose, and to align the point cloud corresponding to each acquisition viewpoint in the global coordinate system using the target acquisition pose, thereby determining the point cloud registration result.

[0103] Optionally, the correction module 604 is specifically used to determine scale correction parameters based on the deviation between the predicted scale corresponding to the initial three-dimensional point map and the metric scale reflected by the depth image; and to perform scale correction on the initial acquisition pose and the initial three-dimensional point map based on the scale correction parameters.

[0104] Optionally, the correction module 604 is specifically used to determine the metric 3D points corresponding to the depth images under each acquisition viewpoint, and calculate the ratio of the coordinate norm of each metric 3D point to the coordinate norm of the corresponding 3D point in the initial 3D point image; determine the scale correction parameter based on the median of each ratio; wherein the scale correction parameter is used to scale and adjust the translation parameters in the initial acquisition pose and the coordinates of the 3D points in the initial 3D point image.

[0105] Optionally, the matching module 606 is specifically used to, for each acquisition viewpoint, take the three-dimensional point in the corrected three-dimensional point map corresponding to that acquisition viewpoint as the query point, and search for spatially neighboring points in the corrected three-dimensional point maps corresponding to other acquisition views. Based on the spatial distance between the query point and the spatially adjacent points, the cross-view correspondence between 3D points under different acquisition perspectives is determined.

[0106] Optionally, the matching module 606 is specifically used to discretize the corrected 3D point map corresponding to each acquisition viewpoint into a voxel grid under the global coordinate system; for each query point, search for candidate 3D points in other acquisition views in the voxel region where the query point is located and in the adjacent voxel regions, so as to determine the cross-view correspondence based on the search results.

[0107] Optionally, the registration module 608 is specifically used to: calculate the deviation between 3D points with cross-view correspondence based on the corrected acquisition pose of each current acquisition viewpoint; determine the weight of each correspondence based on the deviation, and determine the pose adjustment increment corresponding to each acquisition viewpoint; wherein the deviation is negatively correlated with the weight, and the weight is used to characterize the degree of influence of the correspondence on the calculation of the pose adjustment increment; adjust the current acquisition pose of each current acquisition viewpoint based on the pose adjustment increment; repeat the above steps to perform multiple iterations on the acquisition pose until a preset stopping condition is met.

[0108] Optionally, the registration module 608 is specifically used to, after reaching a preset number of iterations, determine the cross-view correspondence where the deviation between corresponding 3D points is greater than a preset first deviation threshold as an external point correspondence, and determine the cross-view correspondence other than the external point correspondence as an internal point correspondence; and continue to adjust the current acquisition pose of each acquisition viewpoint based on the internal point correspondence; wherein, the preset stopping condition includes: the deviation between corresponding 3D points in the internal point correspondence is less than a second deviation threshold, and the second deviation threshold is less than the first deviation threshold.

[0109] Optionally, the registration module 608 is specifically used to employ different deviation constraint parameters in different iteration rounds, and the deviation constraint parameters gradually decrease as the iteration rounds increase, so as to progressively constrain the deviation between three-dimensional points with cross-view correspondence; wherein, the deviation constraint parameters are used to reduce the impact of cross-view correspondence with large deviations on the acquisition pose adjustment.

[0110] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0111] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0112] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0113] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0114] What those skilled in the art will understand is: In this specification, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded.

[0115] In this specification, “a,” “an,” and “the” do not specifically refer to the singular, but may also include the plural.

[0116] In this specification, ordinal numbers such as "first," "second," etc., do not necessarily indicate order; they are often used to distinguish between objects. For example, "first server" and "second server" usually refer to two servers. To differentiate between these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0117] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0118] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0119] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0120] Although one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is only one of many possible execution orders and does not represent the only execution order. Therefore, when the claims involve method steps, any changes or adjustments to the order of such steps, or the parallelism between steps, are also within the scope of protection of the claims.

Claims

1. A multi-view point cloud registration method, comprising: Acquire visible light and depth images from multiple acquisition perspectives; The visible light image is input into a preset three-dimensional reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial three-dimensional point map of the observable scene corresponding to each acquisition viewpoint. Based on the metric scale corresponding to the depth image, the initial acquisition pose and the initial 3D point map are scaled and corrected to obtain the corrected acquisition pose and the corrected 3D point map. Based on the corrected 3D point map, establish cross-view correspondence between 3D points under different acquisition perspectives; With the goal of minimizing the deviation between 3D points with cross-view correspondence, the corrected acquisition pose of each acquisition viewpoint is adjusted to obtain the target acquisition pose. The point cloud corresponding to each acquisition viewpoint is then aligned in the global coordinate system using the target acquisition pose to determine the point cloud registration result.

2. The method as described in claim 1, wherein scaling correction is performed on the initial acquired pose and the initial 3D point map, specifically including: The scale correction parameters are determined based on the deviation between the predicted scale corresponding to the initial 3D point map and the metric scale reflected by the depth image. Based on the scale correction parameters, the initial acquisition pose and the initial 3D point map are scaled.

3. The method as described in claim 2, wherein the scale correction parameters are determined based on the deviation between the predicted scale corresponding to the initial 3D point map and the metric scale reflected by the depth image, specifically including: Determine the corresponding 3D measurement points for the depth images under each acquisition viewpoint, and calculate the ratio of the coordinate norm of each 3D measurement point to the coordinate norm of the corresponding 3D point in the initial 3D point image; The scaling correction parameters are determined based on the median of each ratio; The scale correction parameter is used to scale and adjust the translation parameters in the initial acquisition pose and the coordinates of the three-dimensional points in the initial three-dimensional point map.

4. The multi-view point cloud registration method as described in claim 1, which establishes a cross-view correspondence between 3D points under different acquisition views based on the corrected 3D point map, specifically includes: For each acquisition viewpoint, the 3D point in the corrected 3D point map corresponding to that acquisition viewpoint is used as the query point, and the spatially neighboring point is searched in the corrected 3D point map corresponding to other acquisition views. Based on the spatial distance between the query point and the spatially adjacent points, the cross-view correspondence between 3D points under different acquisition perspectives is determined.

5. The multi-view point cloud registration method as described in claim 4, further comprising establishing cross-view correspondence between 3D points under different acquisition perspectives based on the corrected 3D point map, specifically including: Discretize the corrected 3D point map corresponding to each acquisition viewpoint into a voxel grid in the global coordinate system; For each query point, candidate 3D points in other acquisition perspectives are searched in the voxel region where the query point is located and in the adjacent voxel regions, so as to determine the cross-view correspondence based on the search results.

6. The multi-view point cloud registration method as described in claim 1, with the goal of minimizing the deviation between 3D points with cross-view correspondence, adjusts the corrected acquisition pose of each acquisition viewpoint, specifically including: Based on the corrected acquisition poses of each current acquisition viewpoint, calculate the deviation between 3D points with cross-viewpoint correspondence. The weights of each correspondence are determined based on the deviation, and the pose adjustment increments corresponding to each acquisition viewpoint are determined; wherein, the deviation is negatively correlated with the weights, and the weights are used to characterize the degree of influence of the correspondence on the calculation of the pose adjustment increments; The current acquisition pose of each acquisition viewpoint is adjusted based on the aforementioned pose adjustment increment; Repeat the above steps to iterate the acquired pose multiple times until the preset stopping condition is met.

7. The method as described in claim 6, wherein updating the corrected acquisition pose of each acquisition viewpoint based on the pose adjustment increment specifically includes: After reaching a preset number of iterations, the cross-view correspondences where the deviation between corresponding 3D points is greater than a preset first deviation threshold are determined as external point correspondences, and the cross-view correspondences other than the external point correspondences are determined as internal point correspondences. Based on the aforementioned in-point correspondence, the current acquisition pose of each acquisition viewpoint is further adjusted; The preset stopping condition includes: the deviation between corresponding three-dimensional points in the internal point correspondence relationship is less than a second deviation threshold, and the second deviation threshold is less than the first deviation threshold.

8. The method as described in claim 6, wherein adjusting the current acquisition pose of each acquisition viewpoint based on the pose adjustment increment specifically includes: Different deviation constraint parameters are used in different iteration rounds, and the deviation constraint parameters gradually decrease as the iteration rounds increase, so as to progressively constrain the deviation between three-dimensional points with cross-view correspondence. The deviation constraint parameter is used to reduce the impact of large deviations in cross-view correspondence on the acquisition pose adjustment.

9. A multi-view point cloud registration device, comprising: The acquisition module is used to acquire visible light images and depth images from multiple acquisition perspectives; The input module is used to input the visible light image into a preset three-dimensional reconstruction model to obtain the initial acquisition pose corresponding to each acquisition viewpoint in the global coordinate system and the initial three-dimensional point map of the observable scene corresponding to each acquisition viewpoint. The correction module is used to perform scale correction on the initial acquisition pose and the initial 3D point map according to the metric scale corresponding to the depth image, so as to obtain the corrected acquisition pose and the corrected 3D point map. The matching module is used to establish cross-view correspondence between 3D points under different acquisition perspectives based on the corrected 3D point map; The registration module is used to adjust the corrected acquisition pose of each acquisition viewpoint with the goal of minimizing the deviation between 3D points with cross-view correspondence, so as to obtain the target acquisition pose. Then, the point cloud corresponding to each acquisition viewpoint is aligned in the global coordinate system using the target acquisition pose to determine the point cloud registration result.

10. An electronic device, comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-8 by executing the executable instructions.

11. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-8.

12. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1-8.

Images