Point cloud density control method and device for dense point cloud
By determining visible images in dense point clouds and building Gaussian spheres for point removal operations, the problem of instability of the density process in 3D Gaussian Splatting technology is solved, and the accuracy and stability of the point cloud are achieved, which is suitable for large-scale 3D Gaussian reconstruction. .
Patent Information
- Application Number
- CN202411140044.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-20
AI Technical Summary
3D Gaussian Splatting technology has problems of under-reconstruction and over-reconstruction in the density processing of sparse point clouds, resulting in unstable density process and the accuracy and number of point clouds cannot be guaranteed, limiting the computing and restoration capabilities in large spaces.
By determining all visible images for each point in a dense point cloud, point removal operation is performed, and a Gaussian ball is constructed for secondary point removal after execution. The points to be removed are determined based on the back projection error and transparency parameters to achieve effective point cloud density control.
Ensure the spatial accuracy and stability of point clouds, providing a stable and high-quality point cloud foundation for large-scale and ultra-large-scale 3D Gaussian reconstruction.
Smart Images

Figure CN118657913B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of point cloud data processing, and in particular to a point cloud density control method and device for dense point clouds. Background Art
[0002] 3D Gaussian Splatting technology has made good progress in the field of 3D reconstruction. It can perform Gaussian estimation based on the results of SFM, which not only restores the physical target well, but also greatly improves the rendering speed.
[0003] However, the 3D Gaussian technology is divided into two cases in the process of densification based on sparse point clouds: Under-Reconstruction and Over-Reconstruction. In the case of under-reconstruction, the Gaussian is copied, and in the case of over-reconstruction, the Gaussian is split to control the density of the point cloud. This method relies too much on the radiation characteristics of visualization and does not fully consider the constraints of spatial calculation. As a result, the densification of sparse point clouds is an unstable process. The accuracy and quantity of the points after densification cannot be guaranteed, which directly limits its computational restoration capabilities in large spaces. Summary of the invention
[0004] The main purpose of the present invention is to provide a point cloud density control method and device for dense point clouds to solve the technical problems existing in the related art.
[0005] In order to achieve the above-mentioned purpose, according to a first aspect of the present invention, a point cloud density control method for a dense point cloud is provided, comprising determining, for each point in the dense point cloud, all corresponding visible images, so as to perform a point elimination operation based on all the visible images; after performing the one point elimination operation, constructing a Gaussian sphere for each remaining point, so as to perform a secondary point elimination operation based on the constructed Gaussian sphere.
[0006] Optionally, for each point in the dense point cloud, all corresponding visible images are determined, and a point removal operation is performed based on all visible images, including: for any point in the dense point cloud, all visible images of the any point are determined, and a back projection error mean is determined based on the back projection error of each visible image; based on the back projection error mean corresponding to each point, a point to be removed is determined once, and the point to be removed once is removed.
[0007] Optionally, constructing a Gaussian sphere for each remaining point to perform a secondary point elimination operation based on the constructed Gaussian sphere includes: constructing a Gaussian sphere for each remaining point to estimate specified parameters of each point; based on the specified parameters of each point, determining secondary points to be eliminated, and eliminating the secondary points to be eliminated.
[0008] Optionally, constructing a Gaussian sphere for each remaining point to perform a secondary point elimination operation based on the constructed Gaussian sphere includes: constructing a Gaussian sphere for each remaining point to estimate specified parameters of each point; based on the specified parameters of each point, determining secondary points to be eliminated, and eliminating the secondary points to be eliminated.
[0009] According to a second aspect of the present invention, a point cloud density control device for a dense point cloud is provided, comprising a primary point removal unit, configured to determine all corresponding visible images for each point in the dense point cloud, so as to perform a primary point removal operation based on all the visible images; and a secondary point removal unit, configured to construct a Gaussian sphere for each remaining point after performing the primary point removal operation, so as to perform a secondary point removal operation based on the constructed Gaussian sphere.
[0010] Optionally, the primary point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine a mean back projection error based on the back projection error of each visible image; determine primary points to be eliminated based on the mean back projection error corresponding to each point, and eliminate the primary points to be eliminated; the secondary point elimination unit is configured to construct a Gaussian sphere for each remaining point to estimate the specified parameters of each point; determine secondary points to be eliminated based on the specified parameters of each point, and eliminate the secondary points to be eliminated.
[0011] Optionally, based on the mean back-projection error corresponding to each point, determining the points to be removed once includes:
[0012] It is determined whether the mean value of the back-projection error of each point is less than the set pixel value; and the point indicated by the back-projection error value not less than the set pixel value is determined as a point to be eliminated.
[0013] Optionally, based on the designated parameters of each point, determining the secondary points to be eliminated includes: judging whether the transparency of each point is greater than a set value; and determining the points indicated by the designated parameters that are not greater than the set value as the secondary points to be eliminated.
[0014] According to a third aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to implement the method described in any one of the first aspects.
[0015] According to a fourth aspect of the present invention, there is provided an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method described in any one implementation of the first aspect.
[0016] According to a fifth aspect of the present invention, there is provided a computer program product, comprising a computer program, wherein the computer program implements the method described in any one of the first aspects when executed by a processor.
[0017] This embodiment is a point cloud density control method and device for dense point clouds, wherein the method includes determining all corresponding visible images for each point in the dense point cloud, so as to perform a point culling operation based on all the visible images; after performing a point culling operation, constructing a Gaussian sphere for each remaining point, so as to perform a secondary point culling operation based on the constructed Gaussian sphere. The control of the dense point cloud is achieved by performing Gaussian estimation based on the dense point cloud, which effectively guarantees the spatial accuracy of the point cloud and the stability of the point cloud, and can provide a stable and high-quality point cloud foundation for subsequent large-scale and ultra-large-scale 3D Gaussian reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 is a flow chart of a point cloud density control method for a dense point cloud according to an embodiment of the present invention;
[0020] Figure 2 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0023] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0024] According to an embodiment of the present invention, a point cloud density control method for a dense point cloud is provided, such as Figure 1 As shown, it includes the following steps 101 to 102:
[0025] Step 101: for each point in the dense point cloud, determine all corresponding visible images, and perform a point culling operation based on all the visible images.
[0026] In this step, the existing 3D Gaussian Splatting performs point cloud densification and Gaussian estimation on the basis of the sparse point cloud output by SFM. This embodiment performs Gaussian estimation on the basis of the dense point cloud, wherein the dense point cloud can be obtained through spatial calculation in the MVS process. Compared with the point cloud constructed by 3D Gaussian Splatting, the point cloud has higher accuracy and larger quantity, and on this basis, the dense point cloud is effectively controlled.
[0027] The visible images of the current spatial point are determined, and a point culling can be performed based on the back-projection error of each image.
[0028] As an optional implementation method of this embodiment, for each point in the dense point cloud, all corresponding visible images are determined, and a point removal operation is performed based on all visible images, including: for any point in the dense point cloud, all visible images of the any point are determined, and a back projection error mean is determined based on the back projection error of each visible image; based on the back projection error mean corresponding to each point, a point to be removed is determined once, and the point to be removed once is removed.
[0029] In this optional implementation, if the current spatial point is visible on n images, the back-projection error on each image is ,based on Determine the mean back-projection error for the current point.
[0030] As an optional implementation method of this embodiment, based on the mean back projection error corresponding to each point, determining the points to be eliminated once includes: judging whether the mean back projection error of each point is less than a set pixel value; and determining the point indicated by the back projection error value not less than the set pixel value as the point to be eliminated once.
[0031] In this optional implementation, That is, the mean back-projection error of the current spatial point. If , The threshold range is If the unit is pixel, the current spatial point will be deleted.
[0032] Step 102: After performing a point elimination operation, construct a Gaussian sphere for each remaining point, and perform a secondary point elimination operation based on the constructed Gaussian sphere.
[0033] In this step, Gaussian estimation is performed based on the dense point cloud.
[0034] As an optional implementation method of this embodiment, based on the specified parameters of each point, determining the secondary points to be eliminated includes: judging whether the transparency of each point is greater than a set value; and determining the points indicated by the specified parameters that are not greater than the set value as the secondary points to be eliminated.
[0035] In this optional implementation, a Gaussian sphere with its spatial coordinate as the mean is constructed for each remaining point, and the transparency, spherical harmonic coefficients and other parameters of each spatial point are estimated. The transparency of the current Gaussian sphere, ɑ, satisfies , The threshold range is If the space is between , the current spatial point will be deleted.
[0036] This embodiment can not only effectively ensure the spatial accuracy of the point cloud, but also ensure the stability of the point cloud, and can provide a stable and high-quality point cloud foundation for subsequent large-scale and ultra-large-scale 3D Gaussian reconstruction.
[0037] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0038] The present invention also provides a point cloud density control device for a dense point cloud, comprising a primary point removal unit, configured to determine all corresponding visible images for each point in the dense point cloud, so as to perform a primary point removal operation based on all visible images; and a secondary point removal unit, configured to construct a Gaussian sphere for each remaining point after performing the primary point removal operation, so as to perform a secondary point removal operation based on the constructed Gaussian sphere.
[0039] As an optional implementation method of this embodiment, the primary point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the mean back projection error based on the back projection error of each visible image; determine the primary points to be eliminated based on the mean back projection error corresponding to each point, and eliminate the primary points to be eliminated; the secondary point elimination unit is configured to construct a Gaussian sphere for each remaining point to estimate the specified parameters of each point; determine the secondary points to be eliminated based on the specified parameters of each point, and eliminate the secondary points to be eliminated.
[0040] As an optional implementation method of this embodiment, based on the mean back projection error corresponding to each point, determining the points to be eliminated once includes: judging whether the mean back projection error of each point is less than a set pixel value; and determining the point indicated by the back projection error value not less than the set pixel value as the point to be eliminated once.
[0041] As an optional implementation method of this embodiment, based on the specified parameters of each point, determining the secondary points to be eliminated includes: judging whether the transparency of each point is greater than a set value; and determining the points indicated by the specified parameters that are not greater than the set value as the secondary points to be eliminated.
[0042] According to an embodiment of the present invention, the present invention also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can implement the method described in any of the above embodiments when executing.
[0043] According to an embodiment of the present invention, the present invention further provides a readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to implement the method described in any of the above embodiments when executed.
[0044] According to an embodiment of the present invention, the present invention further provides a computer program product, which can implement the method described in any of the above embodiments when executed by a processor.
[0045] Figure 2A schematic block diagram of an example electronic device 300 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices.
[0046] like Figure 2 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.
[0047] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0048] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the object matching method. For example, in some embodiments, the object matching method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method described above may be performed.
[0049] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0050] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0051] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
Claims
1. A point cloud density control method for dense point clouds, characterized in that: include: For each point in a dense point cloud, all corresponding visible images are determined to perform a point elimination operation based on all the visible images, including: for any point in the dense point cloud, all visible images of the any point are determined, and a back projection error mean is determined based on the back projection error of each visible image; based on the back projection error mean corresponding to each point, a point to be eliminated is determined once, and the point to be eliminated once is eliminated, wherein, based on the back projection error mean corresponding to each point, determining the point to be eliminated once includes: judging whether the back projection error mean of each point is less than a set pixel value; determining a point indicated by a back projection error value not less than the set pixel value as a point to be eliminated once; After performing a point elimination operation, a Gaussian sphere is constructed for each of the remaining points, and a secondary point elimination operation is performed based on the constructed Gaussian sphere, including: constructing a Gaussian sphere for each of the remaining points to estimate the specified parameters of each point; based on the specified parameters of each point, determining the secondary points to be eliminated, and eliminating the secondary points to be eliminated; wherein, based on the specified parameters of each point, determining the secondary points to be eliminated includes: judging whether the transparency of each point is greater than a set value; and determining the point indicated by the specified parameter that is not greater than the set value as the secondary point to be eliminated.
2. A point cloud density control device for dense point clouds, characterized in that: include: A primary point culling unit is configured to determine all corresponding visible images for each point in the dense point cloud, so as to perform a primary point culling operation based on all the visible images; A secondary point elimination unit is configured to construct a Gaussian sphere for each remaining point after performing a primary point elimination operation, so as to perform a secondary point elimination operation based on the constructed Gaussian sphere; The first point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the back projection error mean based on the back projection error of each visible image; determine the first point to be eliminated based on the back projection error mean corresponding to each point, and eliminate the first point to be eliminated, and determine the first point to be eliminated based on the back projection error mean corresponding to each point, including: judging whether the back projection error mean of each point is less than a set pixel value; determining the point indicated by the back projection error value not less than the set pixel value as the first point to be eliminated; the second point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the back projection error mean based on the back projection error of each point, and eliminate the first point to be eliminated; the second point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the back projection error mean based on the back projection error of each point, and eliminate the first point to be eliminated; the second point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the back projection error mean based on the back projection error of each point, and determine ... first point to be eliminated; the second point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the first point to be eliminated; the second point elimination unit is configured to determine all visible images of any point in the dense point cloud, and determine the first point to be eliminated; the second point elimination unit is configured A Gaussian sphere is constructed for each remaining point to estimate the specified parameters of each point; based on the specified parameters of each point, secondary points to be eliminated are determined, and the secondary points to be eliminated are eliminated, including: constructing a Gaussian sphere for each remaining point to estimate the specified parameters of each point; based on the specified parameters of each point, secondary points to be eliminated are determined, and the secondary points to be eliminated are eliminated; wherein, based on the specified parameters of each point, determining the secondary points to be eliminated includes: judging whether the transparency of each point is greater than a set value; and determining the point indicated by the specified parameter that is not greater than the set value as the secondary point to be eliminated.
3. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method of claim 1.
4. An electronic device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the method of claim 1.
5. A computer program product, characterized in that A computer program is included which, when executed by a processor, implements the method according to claim 1.
Citation Information
Patent Citations
Scene orthoimage generation method based on MVS dense point cloud
CN114913297A
Point cloud data processing method and device, equipment and medium
CN117372483A
Three-dimensional point cloud denoising method and system for complex underground roadway scene
CN117893436A
Dense vision SLAM (Simultaneous Localization and Mapping) method and system using three-dimensional Gaussian back-end representation
CN117990088A
