Dense point cloud denoising method, system and device based on visible shell

By combining visual shell algorithm, motion structure, multi-view stereoscopic geometry algorithm, using preset transformation matrix and point cloud boundary sphere for denoising dense point clouds, the problem of point cloud denoising difficulty in the existing technology is solved. Improved the accuracy and resolution of the three-dimensional reconstruction model.

CN118628398BActive Publication Date: 2024-11-26ZHEJIANG TUOPUYUN AGRI SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411095691.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-26
Estimated Expiration
2044-08-12

AI Technical Summary

Technical Problem

In the existing three-dimensional reconstruction technology, dense point cloud denoising methods based on motion structure and multi-view stereo geometry are difficult to distinguish point clouds and smear noise on objects, resulting in filtering out real point clouds or being unable to completely remove smear noise, affecting reconstruction. Model accuracy and completeness.

Method used

Using a dense point cloud denoising method based on the visual shell, sparse reconstruction and dense reconstruction are carried out by obtaining images of different angles and positions of objects, and using a preset transformation matrix to register the noisy dense point cloud to the three-dimensional shell point cloud coordinate system. And the point cloud boundary sphere is used to denoise merge to obtain the dense point cloud after denoising.

Benefits of technology

It significantly improves the density and completeness of the three-dimensional reconstruction model, enhances the accuracy and resolution of the reconstruction model, and only depends on image input, making data acquisition and easy to operate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118628398B_ABST
    Figure CN118628398B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system and device for denoising dense point clouds based on a visual shell. The method comprises: acquiring images of multiple objects at different angles and positions; performing sparse reconstruction and dense reconstruction on the images using a structure-from-motion algorithm and a multi-view stereo geometry algorithm to obtain a dense point cloud containing smear noise on the outer surface of the object; reconstructing the images using a visual shell algorithm to obtain a three-dimensional shell point cloud; transforming the dense point cloud on the outer surface of the object to the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud; obtaining a point cloud boundary sphere from the three-dimensional shell point cloud, merging the registered dense point clouds that fall within the point cloud boundary sphere to obtain a denoised dense point cloud. The method greatly improves the accuracy and resolution of the three-dimensional reconstruction model by combining the visual shell algorithm, the structure-from-motion algorithm and the multi-view stereo geometry algorithm.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision three-dimensional reconstruction, and in particular relates to a dense point cloud denoising method, system and device based on a visible shell. Background Art

[0002] The VisualHull algorithm is a geometry-based reconstruction technique that uses images of an object captured by a camera from different angles and positions to construct a minimum volume that can surround the object, that is, to restore the shape of a three-dimensional object from two-dimensional images from multiple perspectives. It does not rely on the texture information of the object and is relatively insensitive to lighting changes and image noise, but requires precise values ​​of multiple angles and positions of the camera that captures the object image. Its reconstruction resolution is also limited by the resolution of the original image and the accuracy of voxelization.

[0003] Three-dimensional reconstruction based on Structure from Motion (SfM) and Multi-View Stereo (MVS) are two commonly used computer vision technologies for recovering the structure of a three-dimensional scene from a set of two-dimensional images. SfM and MVS are used in combination to obtain a dense point cloud on the surface of an object through three-dimensional reconstruction, which will produce blurred or discontinuous disparity maps at the edges of the object, thereby generating point cloud smear noise. Conventional point cloud noise removal methods cannot distinguish whether the point cloud is on the object or the smear, so they will filter out the real point cloud on the object and easily produce holes, or fail to completely filter out the smear noise. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention discloses a dense point cloud denoising method, system and device based on a visible shell.

[0005] In order to achieve the above technical effects, the present invention adopts the following technical solutions:

[0006] The dense point cloud denoising method based on visual shell includes the following steps:

[0007] Acquire several images of the object to be measured at different angles and positions of the object;

[0008] Sparsely reconstructing several images of the object to be measured to obtain several camera poses and three-dimensional coordinates of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses remains consistent, and obtaining a noisy dense point cloud of the outer surface of the object to be measured based on the several images of the object to be measured, the several camera poses and the at least one feature point;

[0009] Reconstructing several images of the object to be measured, several camera poses and at least one feature point to obtain a three-dimensional shell of the object to be measured and then converting it into a three-dimensional shell point cloud;

[0010] Based on a preset transformation matrix, the noisy dense point cloud and the three-dimensional shell point cloud are registered, and then the noisy dense point cloud is transformed into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0011] Based on the preset point cloud bounding sphere that meets the conditions of the 3D shell point cloud, the registered dense point clouds falling within the point cloud bounding sphere are merged to obtain the denoised dense point cloud.

[0012] As an implementable method, the preset transformation matrix is ​​obtained by:

[0013] Extracting first key points of a noisy dense point cloud and second key points of a three-dimensional shell point cloud;

[0014] Calculate a first descriptor based on the first key point, and calculate a second descriptor based on the second key point;

[0015] Performing a first registration on the first descriptor through a point cloud registration function to obtain a first transformation matrix;

[0016] Performing a second registration of the point cloud using the point cloud registration function and the first change matrix to obtain a second transformation matrix after the second registration;

[0017] A preset transformation matrix is ​​obtained based on the first transformation matrix and the second transformation matrix.

[0018] As an implementable method, the preset transformation matrix is ​​expressed as follows:

[0019]

[0020] in, Represents the preset transformation matrix, represents the first transformation matrix, Represents the second transformation matrix.

[0021] As an implementable method, the registered dense point cloud is expressed as follows:

[0022]

[0023] in, represents the dense point cloud after registration, Represents the preset transformation matrix, Represents a noisy dense point cloud.

[0024] As an implementable method, the method presets a point cloud bounding sphere that meets the conditions based on the three-dimensional shell point cloud, merges the registered dense point clouds that fall within the point cloud bounding sphere, and obtains the denoised dense point cloud, including the following steps:

[0025] Constructing a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center, wherein the center of the point cloud boundary sphere is any three-dimensional point in the three-dimensional shell point cloud, and the radius of the point cloud boundary sphere is variable and adjusted according to a preset object surface noise range;

[0026] Construct a KD-tree for the registered dense point cloud, retain and merge the points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere, and obtain the denoised dense point cloud;

[0027] The dense point cloud after denoising is represented as follows:

[0028]

[0029] Among them, among them, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, It represents the center of any point cloud bounding sphere in the three-dimensional shell point cloud, and r represents the radius of the point cloud bounding sphere.

[0030] As an implementable method, based on each spatial point in the registered dense point cloud, a corresponding closest distance point is found in the three-dimensional shell point cloud, and the closest distance between each spatial point and the closest distance point is calculated;

[0031] Calculate the standard deviation of all the closest distances, where the standard deviation is the radius of the point cloud bounding sphere;

[0032] The closest distance is expressed as follows:

[0033]

[0034] The point cloud bounding sphere radius is expressed as follows:

[0035]

[0036] in, represents the i-th spatial point in the dense point cloud after registration, represents the shortest distance between the i-th spatial point and the nearest distance point, represents the dense point cloud after registration, Represents a 3D hull point cloud.

[0037] As an implementable method, the radius of the point cloud bounding sphere is a preset object surface noise range, wherein a larger radius value indicates that noise is allowed to exist farther from the object surface; a smaller radius value indicates that noise is allowed to exist closer to the object surface.

[0038] A dense point cloud denoising system based on a visible shell includes: a data acquisition module, a dense point cloud 3D reconstruction module, a shell point cloud 3D reconstruction module, a registration module and a denoising module;

[0039] The data acquisition module is used to acquire a plurality of images of the object to be measured at different angles and positions of the object;

[0040] The dense point cloud 3D reconstruction module performs sparse reconstruction on a plurality of images of the object to be measured, obtains a plurality of camera poses and a 3D coordinate of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses is consistent, and obtains a noisy dense point cloud of the outer surface of the object to be measured based on the plurality of images of the object to be measured, the plurality of camera poses and the at least one feature point;

[0041] The shell point cloud 3D reconstruction module reconstructs a plurality of images of the object to be measured, a plurality of camera positions and at least one feature point to obtain a 3D shell of the object to be measured and then converts it into a 3D shell point cloud;

[0042] The registration module registers the noisy dense point cloud and the three-dimensional shell point cloud based on a preset transformation matrix and then transforms the noisy dense point cloud into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0043] The denoising module presets a point cloud bounding sphere that meets the conditions based on the three-dimensional shell point cloud, merges the registered dense point clouds that fall within the point cloud bounding sphere, and obtains the denoised dense point cloud.

[0044] As an implementable embodiment, the denoising module is configured as follows:

[0045] Constructing a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center, wherein the center of the point cloud boundary sphere is any three-dimensional point in the three-dimensional shell point cloud, and the radius of the point cloud boundary sphere is adjusted according to a preset object surface noise range;

[0046] Construct a KD-tree for the registered dense point cloud, retain and merge the points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere, and obtain the denoised dense point cloud;

[0047] The denoised dense point cloud is represented as follows:

[0048]

[0049] in, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, It represents the center of any point cloud bounding sphere in the three-dimensional shell point cloud, and r represents the radius of the point cloud bounding sphere.

[0050] As an implementable embodiment, the denoising module is configured as follows:

[0051] Based on each spatial point in the registered dense point cloud, find the corresponding closest distance point in the 3D shell point cloud, and calculate the closest distance between each spatial point and the closest distance point;

[0052] Calculate the standard deviation of all the closest distances, where the standard deviation is the radius of the point cloud bounding sphere;

[0053] The closest distance is expressed as follows:

[0054]

[0055] The point cloud bounding sphere radius is expressed as follows:

[0056]

[0057] in, represents the i-th spatial point in the dense point cloud after registration, represents the shortest distance between the i-th spatial point and the nearest distance point, represents the dense point cloud after registration, Represents a 3D hull point cloud.

[0058] A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the following method:

[0059] Acquire several images of the object to be measured at different angles and positions of the object;

[0060] Sparsely reconstructing several images of the object to be measured to obtain several camera poses and three-dimensional coordinates of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses remains consistent, and obtaining a noisy dense point cloud of the outer surface of the object to be measured based on the several images of the object to be measured, the several camera poses and the at least one feature point;

[0061] Reconstructing several images of the object to be measured, several camera poses and at least one feature point to obtain a three-dimensional shell of the object to be measured and then converting it into a three-dimensional shell point cloud;

[0062] Based on a preset transformation matrix, the noisy dense point cloud and the three-dimensional shell point cloud are registered, and then the noisy dense point cloud is transformed into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0063] Based on the preset point cloud bounding sphere that meets the conditions of the 3D shell point cloud, the registered dense point clouds falling within the point cloud bounding sphere are merged to obtain the denoised dense point cloud.

[0064] A device for a dense point cloud denoising method based on a visual shell includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following method when executing the computer program:

[0065] Acquire several images of the object to be measured at different angles and positions of the object;

[0066] Sparsely reconstructing several images of the object to be measured to obtain several camera poses and three-dimensional coordinates of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses remains consistent, and obtaining a noisy dense point cloud of the outer surface of the object to be measured based on the several images of the object to be measured, the several camera poses and the at least one feature point;

[0067] Reconstructing several images of the object to be measured, several camera poses and at least one feature point to obtain a three-dimensional shell of the object to be measured and then converting it into a three-dimensional shell point cloud;

[0068] Based on a preset transformation matrix, the noisy dense point cloud and the three-dimensional shell point cloud are registered, and then the noisy dense point cloud is transformed into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0069] Based on the preset point cloud bounding sphere that meets the conditions of the 3D shell point cloud, the registered dense point clouds falling within the point cloud bounding sphere are merged to obtain the denoised dense point cloud.

[0070] The present invention has significant technical effects due to the adoption of the above technical solution:

[0071] 1. The present invention combines the visual shell algorithm, motion structure and multi-view stereo geometry algorithm, and registers the dense point cloud of the external surface obtained after 3D reconstruction by the motion structure and multi-view stereo geometry algorithm with the 3D shell point cloud obtained by 3D reconstruction by the visual shell algorithm, which greatly improves the density and completeness of the 3D reconstructed model;

[0072] 2. The present invention uses the point cloud bounding sphere of the 3D shell point cloud obtained by 3D reconstruction based on the visual shell algorithm to denoise the dense point cloud after registration, which greatly improves the accuracy and resolution of the 3D reconstruction model;

[0073] 3. The dense point cloud denoising method based on visible shell described in the present invention only uses images as system input, and data acquisition is convenient and easy to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The following drawings are examples of specific implementation methods of the present application:

[0075] Figure 1 It is a schematic diagram of the process of the dense point cloud denoising method based on the visual shell according to the present invention;

[0076] Figure 2 It is an overall schematic diagram of the dense point cloud denoising system based on the visible shell described in the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are explanations of the present invention and the present invention is not limited to the following embodiments.

[0078] Embodiment 1:

[0079] A dense point cloud denoising method based on visual hull, such as Figure 1 As shown, the following steps are included:

[0080] S100, acquiring a plurality of images of the object to be measured at different angles and positions of the object;

[0081] S200, sparsely reconstructing a plurality of images of the object to be measured to obtain a plurality of camera poses and three-dimensional coordinates of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses remains consistent, and based on the plurality of images of the object to be measured, the plurality of camera poses and at least one feature point, obtaining a noisy dense point cloud of the outer surface of the object to be measured;

[0082] S300, reconstructing a plurality of images of the object to be measured, a plurality of camera positions and at least one feature point to obtain a three-dimensional shell of the object to be measured and then converting it into a three-dimensional shell point cloud;

[0083] S400, registering the noisy dense point cloud and the three-dimensional shell point cloud based on a preset transformation matrix, and then transforming the noisy dense point cloud into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0084] S500, based on the three-dimensional shell point cloud, a point cloud bounding sphere that meets the conditions is preset, and the registered dense point clouds that fall within the point cloud bounding sphere are merged to obtain a denoised dense point cloud.

[0085] The present invention obtains images of an object at multiple different angles and positions; uses a dense point cloud denoising method based on a visual shell to perform sparse reconstruction and dense reconstruction on the above images based on a motion structure algorithm and a multi-view stereo geometry algorithm to obtain a noisy dense point cloud on the outer surface of the object; reconstructs the image using a visual shell-based algorithm to obtain a three-dimensional shell point cloud; transforms the dense point cloud on the outer surface of the object into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud; obtains a point cloud boundary sphere from the three-dimensional shell point cloud, searches for and merges the registered dense point clouds that fall within the point cloud boundary sphere, and then completes the denoising of the dense point cloud.

[0086] In this embodiment, step S200 uses a Structure from Motion (SfM) algorithm to perform sparse reconstruction on the above-mentioned multiple images to obtain multiple camera poses and three-dimensional coordinates of multiple feature points on the surface of the object; and uses a Multi-View Stereo (MVS) algorithm to perform dense reconstruction on the above-mentioned multiple images, multiple camera poses, and three-dimensional coordinates of at least one feature point to obtain a noisy dense point cloud of the outer surface of the object, which specifically includes the following steps:

[0087] S210: sparsely reconstruct the plurality of images using a SfM algorithm, such as colmap or openMVG, to obtain a plurality of camera poses and a three-dimensional coordinate of at least one feature point of the object;

[0088] S220: Using an MVS algorithm, such as openMVS, to obtain a noisy dense point cloud of the outer surface of the object based on the above-mentioned multiple images, multiple camera poses and the three-dimensional coordinates of at least one object feature point .

[0089] The openMVS algorithm used in this embodiment can generate a densified point cloud and a gridded point cloud, thereby completing three-dimensional reconstruction.

[0090] In step S300, the above-mentioned plurality of images and plurality of camera poses are reconstructed by using a visual hull algorithm to obtain a three-dimensional hull, and the three-dimensional hull is converted into a three-dimensional hull point cloud, which specifically includes the following steps:

[0091] Step S310: using a VisualHull algorithm, such as the open source Matlab-VisualHull, to obtain a three-dimensional shell hull of the object on the above-mentioned plurality of images and plurality of camera poses;

[0092] Step S320: Convert the mesh hull into a three-dimensional point cloud The conversion method is the Mesh->Sample Points function in CloudCompare.

[0093] In step S400, the noisy dense point cloud and the three-dimensional shell point cloud are registered to obtain a transformation matrix, and the dense point cloud of the outer surface of the object is transformed into the three-dimensional shell point cloud coordinate system according to the transformation matrix to obtain a registered dense point cloud, which specifically includes the following steps:

[0094] Step S410: and Registration, the transformation matrix is ​​recorded as , the preset transformation matrix , obtained by:

[0095] Extracting first key points of a noisy dense point cloud and second key points of a three-dimensional shell point cloud;

[0096] Calculate a first descriptor based on the first key point, and calculate a second descriptor based on the second key point;

[0097] Performing a first registration on the first descriptor and the second descriptor through a point cloud registration function to obtain a first transformation matrix;

[0098] Performing a second registration of the point cloud using the point cloud registration function and the first change matrix to obtain a second transformation matrix after the second registration;

[0099] Based on the first transformation matrix and the second transformation matrix, a preset transformation matrix is ​​obtained ; The preset transformation matrix is ​​expressed as follows:

[0100]

[0101] in, Represents the preset transformation matrix, represents the first transformation matrix, Represents the second transformation matrix.

[0102] Step S420: Transform to In the coordinate system, we get ,Right now .

[0103] In this embodiment, the first registration method is point cloud feature point matching or the Align function in CloudCompare; the second registration method is ICP (Iterative Closest Point) or the Fine Registration (ICP) function in CloudCompare.

[0104] In step S500, based on the three-dimensional shell point cloud, a point cloud bounding sphere that meets the conditions is preset, and the registered dense point clouds falling within the point cloud bounding sphere are merged to obtain a denoised dense point cloud, including the following steps:

[0105] Step S510: construct a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center. For any 3D point in the 3D shell point cloud, the radius of the point cloud bounding sphere is preset according to the actual application requirements;

[0106] Step S520: construct a KD-tree for the registered dense point cloud. The points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere are retained and merged to obtain the denoised dense point cloud;

[0107] The dense point cloud after denoising is represented as follows:

[0108]

[0109] in, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, express The point in The center of any point cloud bounding sphere in is r, and r represents the radius of the point cloud bounding sphere.

[0110] The calculation formula of the point cloud boundary sphere radius r is:

[0111]

[0112]

[0113] in, Dense point cloud after registration For every spatial point in Point cloud for the shell The closest distance point in is calculated by the nearest distance.

[0114] That is to say, the radius of the point cloud bounding sphere is the preset object surface noise range, wherein the larger the radius value, the farther away from the object surface the noise is allowed to exist; the smaller the radius value, the closer to the object surface the noise is allowed to exist.

[0115] The embodiment of the present invention combines a visual shell algorithm with a motion structure and multi-view stereo geometry algorithm to register a dense point cloud of the external surface obtained after three-dimensional reconstruction based on the motion structure and multi-view stereo geometry algorithm with a three-dimensional shell point cloud obtained by three-dimensional reconstruction based on the visual shell algorithm, thereby improving the density and completeness of the three-dimensional reconstructed model.

[0116] The embodiment of the present invention performs denoising on the dense point cloud after registration based on the point cloud bounding sphere of the three-dimensional shell point cloud obtained by three-dimensional reconstruction using the visual shell algorithm, thereby improving the accuracy and resolution of the three-dimensional reconstructed model.

[0117] Embodiment 2:

[0118] A dense point cloud denoising system based on visual hull, such as Figure 2 As shown, it includes: a data acquisition module 100, a dense point cloud 3D reconstruction module 200, a shell point cloud 3D reconstruction module 300, a registration module 400, and a denoising module 500;

[0119] The data acquisition module 100 is used to acquire a plurality of images of the object to be measured at different angles and positions of the object;

[0120] The dense point cloud 3D reconstruction module 200 performs sparse reconstruction on a plurality of images of the object to be measured, and obtains a plurality of camera poses and a 3D coordinate of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses is consistent, and based on the plurality of images of the object to be measured, the plurality of camera poses and at least one feature point, a noisy dense point cloud of the outer surface of the object to be measured is obtained;

[0121] The shell point cloud 3D reconstruction module 300 reconstructs a plurality of images of the object to be measured, a plurality of camera positions and at least one feature point to obtain a 3D shell of the object to be measured and then converts it into a 3D shell point cloud;

[0122] The registration module 400 registers the noisy dense point cloud and the three-dimensional shell point cloud based on a preset transformation matrix and then transforms the noisy dense point cloud into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud;

[0123] The denoising module 500 presets a point cloud bounding sphere that meets the conditions based on the three-dimensional shell point cloud, merges the registered dense point clouds that fall within the point cloud bounding sphere, and obtains the denoised dense point cloud.

[0124] Various changes and modifications can be made without departing from the spirit and scope of the present invention, and all equivalent technical solutions also belong to the scope of the present invention.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0126] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0127] The present invention is described with reference to the flowcharts and / or block diagrams of the method, terminal device (system), and computer program product according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0128] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0129] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0130] It should be noted that:

[0131] The "one embodiment" or "embodiment" mentioned in the specification means that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment of the present invention. Therefore, the phrases "one embodiment" or "embodiment" appearing in various places throughout the specification do not necessarily refer to the same embodiment.

[0132] In addition, it should be noted that the shapes and names of the parts of the specific embodiments described in this specification may be different. Any equivalent or simple changes made based on the structure, features and principles described in the patent concept of the present invention are included in the protection scope of the patent of the present invention. The technicians in the technical field of the present invention can make various modifications or supplements to the specific embodiments described or replace them in a similar manner, as long as they do not deviate from the structure of the present invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A dense point cloud denoising method based on visual shell, characterized in that: The following steps are involved: Acquire several images of the object to be measured at different angles and positions of the object; Sparsely reconstructing several images of the object to be measured to obtain several camera poses and three-dimensional coordinates of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses remains consistent, and obtaining a noisy dense point cloud of the outer surface of the object to be measured based on the several images of the object to be measured, the several camera poses and the at least one feature point; Reconstructing several images of the object to be measured, several camera poses and at least one feature point to obtain a three-dimensional shell of the object to be measured and then converting it into a three-dimensional shell point cloud; Based on a preset transformation matrix, the noisy dense point cloud and the three-dimensional shell point cloud are registered, and then the noisy dense point cloud is transformed into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud; Based on the preset point cloud bounding sphere that meets the conditions of the 3D shell point cloud, the registered dense point clouds falling within the point cloud bounding sphere are merged to obtain the denoised dense point cloud; The method of presetting a point cloud boundary sphere that meets the conditions based on the three-dimensional shell point cloud, merging the registered dense point clouds that fall within the point cloud boundary sphere, and obtaining a denoised dense point cloud includes the following steps: Constructing a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center, wherein the center of the point cloud boundary sphere is any three-dimensional point in the three-dimensional shell point cloud, and the radius of the point cloud boundary sphere is adjusted according to a preset object surface noise range; Construct a KD-tree for the registered dense point cloud, retain and merge the points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere, and obtain the denoised dense point cloud; The denoised dense point cloud is represented as follows: in, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, express The center of any point cloud boundary sphere in , r represents the radius of the point cloud boundary sphere; The point cloud bounding sphere radius is obtained by the following steps: Based on each spatial point in the registered dense point cloud, find the corresponding closest distance point in the 3D shell point cloud, and calculate the closest distance between each spatial point and the closest distance point; Calculate the standard deviation of all the closest distances, where the standard deviation is the radius of the point cloud bounding sphere; The closest distance is expressed as follows: The point cloud bounding sphere radius is expressed as follows: in, represents the i-th spatial point in the dense point cloud after registration, represents the shortest distance between the i-th spatial point and the nearest distance point, represents the dense point cloud after registration, Represents a 3D hull point cloud.

2. The method for denoising dense point clouds based on visual hull according to claim 1, characterized in that: The preset transformation matrix is ​​obtained in the following way: Extracting first key points of a noisy dense point cloud and second key points of a three-dimensional shell point cloud; Calculate a first descriptor based on the first key point, and calculate a second descriptor based on the second key point; Performing a first registration on the first descriptor through a point cloud registration function to obtain a first transformation matrix; Performing a second registration of the point cloud using the point cloud registration function and the first change matrix to obtain a second transformation matrix after the second registration; A preset transformation matrix is ​​obtained based on the first transformation matrix and the second transformation matrix.

3. A dense point cloud denoising method based on visual shell according to claim 1 or 2, characterized in that: The preset transformation matrix is ​​expressed as follows: in, Represents the preset transformation matrix, represents the first transformation matrix, Represents the second transformation matrix.

4. The method for denoising dense point clouds based on visual hull according to claim 3, characterized in that: The dense point cloud after registration is expressed as follows: in, represents the dense point cloud after registration, Represents the preset transformation matrix, Represents a noisy dense point cloud.

5. The method for denoising dense point clouds based on visual hull according to claim 1, characterized in that: The radius of the point cloud bounding sphere is the preset noise range of the object surface. The larger the radius value, the farther away from the object surface the noise is allowed to exist; the smaller the radius value, the closer to the object surface the noise is allowed to exist.

6. A dense point cloud denoising system based on visual shell, characterized in that: include: Data acquisition module, dense point cloud 3D reconstruction module, shell point cloud 3D reconstruction module, registration module and denoising module; The data acquisition module is used to acquire a plurality of images of the object to be measured at different angles and positions of the object; The dense point cloud 3D reconstruction module performs sparse reconstruction on a plurality of images of the object to be measured, obtains a plurality of camera poses and a 3D coordinate of at least one feature point on the surface of the object to be measured, wherein the number of images and camera poses is consistent, and obtains a noisy dense point cloud of the outer surface of the object to be measured based on the plurality of images of the object to be measured, the plurality of camera poses and the at least one feature point; The shell point cloud 3D reconstruction module reconstructs a plurality of images of the object to be measured, a plurality of camera positions and at least one feature point to obtain a 3D shell of the object to be measured and then converts it into a 3D shell point cloud; The registration module registers the noisy dense point cloud and the three-dimensional shell point cloud based on a preset transformation matrix and then transforms the noisy dense point cloud into the three-dimensional shell point cloud coordinate system to obtain a registered dense point cloud; The denoising module presets a point cloud bounding sphere that meets the conditions based on the three-dimensional shell point cloud, merges the registered dense point clouds that fall within the point cloud bounding sphere, and obtains a denoised dense point cloud; The step of presetting a point cloud boundary sphere that meets the conditions based on the three-dimensional shell point cloud and merging the registered dense point clouds that fall within the point cloud boundary sphere to obtain a denoised dense point cloud includes the following steps: Constructing a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center, wherein the center of the point cloud boundary sphere is any three-dimensional point in the three-dimensional shell point cloud, and the radius of the point cloud boundary sphere is adjusted according to a preset object surface noise range; Construct a KD-tree for the registered dense point cloud, retain and merge the points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere, and obtain the denoised dense point cloud; The denoised dense point cloud is represented as follows: in, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, represents the center of any point cloud boundary sphere in the three-dimensional shell point cloud, and r represents the radius of the point cloud boundary sphere; The point cloud bounding sphere radius is obtained by the following steps: Based on each spatial point in the registered dense point cloud, find the corresponding closest distance point in the 3D shell point cloud, and calculate the closest distance between each spatial point and the closest distance point; Calculate the standard deviation of all the closest distances, where the standard deviation is the radius of the point cloud bounding sphere; The closest distance is expressed as follows: The point cloud bounding sphere radius is expressed as follows: in, represents the i-th spatial point in the dense point cloud after registration, represents the shortest distance between the i-th spatial point and the nearest distance point, represents the dense point cloud after registration, Represents a 3D hull point cloud.

7. A dense point cloud denoising system based on visual shell, characterized in that: The denoising module is set to: Constructing a point cloud boundary sphere with each point in the three-dimensional shell point cloud as the center, wherein the center of the point cloud boundary sphere is any three-dimensional point in the three-dimensional shell point cloud, and the radius of the point cloud boundary sphere is adjusted according to a preset object surface noise range; Construct a KD-tree for the registered dense point cloud, retain and merge the points in the registered dense point cloud that fall into the outer shell of the point cloud bounding sphere, and obtain the denoised dense point cloud; The denoised dense point cloud is represented as follows: in, represents the dense point cloud after denoising, represents the dense point cloud after registration, represents the 3D hull point cloud, It represents the center of any point cloud bounding sphere in the three-dimensional shell point cloud, and r represents the radius of the point cloud bounding sphere.

8. The dense point cloud denoising system based on visual shell according to claim 7, characterized in that: The denoising module is set to: Based on each spatial point in the registered dense point cloud, find the corresponding closest distance point in the 3D shell point cloud, and calculate the closest distance between each spatial point and the closest distance point; Calculate the standard deviation of all the closest distances, where the standard deviation is the radius of the point cloud bounding sphere; The closest distance is expressed as follows: The point cloud bounding sphere radius is expressed as follows: in, represents the i-th spatial point in the dense point cloud after registration, represents the shortest distance between the i-th spatial point and the nearest distance point, represents the dense point cloud after registration, Represents a 3D hull point cloud.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

10. A device for dense point cloud denoising method based on visual shell, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.