Lidar point cloud denoising method, device and electronic equipment

Abstract

The application provides a laser radar point cloud denoising method and device and electronic equipment, and relates to the field of three-dimensional point cloud data processing. The method comprises the following steps: determining the corresponding observation relationship between each point cloud point in the LiDAR point cloud and the pose of the sensor according to the data of the sensor, constructing the line of sight corresponding to the observation space visual angle between the point cloud point and the sensor according to the observation relationship; constructing a three-dimensional Delaunay tetrahedral grid based on the point cloud point, determining the foreground weight of the foreground space and the background weight of the background space corresponding to each three-dimensional Delaunay tetrahedral grid based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral grid; and converting the three-dimensional Delaunay tetrahedral grid into a foreground and background graph according to the foreground weight and the background weight, performing global optimal image segmentation between the foreground and the background based on the foreground and background graph, and obtaining the surface points corresponding to the foreground and the noise points corresponding to the background.

Description

Technical Field

[0001] This application relates to the field of 3D point cloud data processing technology, and in particular to a method, apparatus, and electronic device for denoising point clouds using lidar. Background Technology

[0002] Currently, LiDAR, as an active 3D measurement method, is widely used in fields such as autonomous driving, high-precision map construction, urban 3D modeling, and surveying engineering. Mobile measurement platforms (such as vehicle-mounted and airborne LiDAR systems) can acquire large-scale 3D point cloud data through continuous scanning, providing dense spatial sampling for the geometric reconstruction of scenes. However, raw LiDAR point clouds inevitably contain a large amount of noise and outliers, and existing LiDAR point cloud denoising methods have low denoising accuracy. Summary of the Invention

[0003] The purpose of this application is to provide a method, apparatus, and electronic device for denoising LiDAR point clouds, in order to solve the technical problem of low accuracy in LiDAR point cloud denoising.

[0004] In a first aspect, this application provides a method for denoising point clouds using lidar, the method comprising: Data from sensors that acquire point clouds of Laser Radar (LiDAR) and observe point cloud points in the LiDAR point cloud; Based on the data from the sensor, determine the corresponding observation relationship between each point cloud point and the pose of the sensor, and construct the line of sight corresponding to the observation spatial perspective between the point cloud point and the sensor based on the observation relationship; A three-dimensional Delaunay tetrahedral mesh is constructed based on the point cloud points, and the foreground weight and background weight of the face of each three-dimensional Delaunay tetrahedral mesh are determined based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh. The three-dimensional Delaunay tetrahedral mesh is transformed into foreground and background images based on the foreground and background weights, and global optimal image segmentation between the foreground and background is performed based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. The LiDAR point cloud is filtered and denoised based on the surface points and the noise points to obtain the denoising result.

[0005] In one possible implementation, after acquiring the LiDAR point cloud, the method further includes: The spatial range corresponding to the LiDAR point cloud is divided into voxel grids to discretize the spatial range into several uniform voxel grids. The voxel mesh is aggregated into an initial spatial block according to a preset block dimension, and when the number of point cloud points in a single initial spatial block exceeds a preset threshold, a recursive binary refinement process is performed to obtain an intermediate spatial block. A spatial boundary buffer is added to each intermediate spatial block to obtain the final spatial block. Neighboring points within the buffer participate in the calculation of the current final spatial block but do not output the result.

[0006] In one possible implementation, the sensor data includes the sensor's pose trajectory data; determining the corresponding observation relationship between each point cloud point and the sensor's pose based on the sensor data includes: For each point cloud point in the final spatial block, the sensor pose of the sensor corresponding to the point cloud point at each time moment is determined based on the pose trajectory data. Based on the sensor pose and the acquisition time of the point cloud points, the target sensor pose index corresponding to the point cloud points is obtained by matching the most recent time. Based on the target sensor pose index and the point cloud points, establish the corresponding observation relationship between the point cloud points and the pose of the sensor.

[0007] In one possible implementation, constructing a 3D Delaunay tetrahedral mesh based on the point cloud points includes: All the point cloud points are spatially sorted using the Computational Geometry Algorithms Library (CGAL) to obtain the insertion order; According to the insertion order, Delaunay triangulation (Delaunay triangulation algorithm) triangulations are inserted point by point, and distance downsampling is performed during the insertion of the Delaunay triangulations to obtain the Delaunay vertices; Based on the Delaunay vertices, the continuous space corresponding to the final space block is discretized into a finite number of three-dimensional Delaunay tetrahedral meshes; wherein, each tetrahedral mesh contains four triangular faces, and adjacent tetrahedrals share common triangular faces.

[0008] In one possible implementation, determining the foreground weight in foreground space and the background weight in background space corresponding to each face of the 3D Delaunay tetrahedron mesh based on the geometric relationship of the line of sight passing through the 3D Delaunay tetrahedron mesh includes: In all the tetrahedrons, the common triangular face shared between the finite tetrahedrons and the infinite tetrahedrons is determined to obtain the convex hull surface; For each observation space perspective, calculate the target tetrahedron where the corresponding line of sight is located. If the line of sight is inside the convex hull surface, then determine the four faces of the target tetrahedron as the set of ray tracing starting faces, and determine the foreground of the face of the tetrahedron containing the camera as the foreground space, and determine the occlusion space of the face of the tetrahedron located behind the object surface as the background space. Based on the viewpoint and the vertices of the tetrahedron, ray traversal calculations are performed on each of the observation space perspectives of all the vertices of the tetrahedron using the set of ray tracing starting faces to obtain the calculation results, and the face weights of the tetrahedron are determined based on the calculation results.

[0009] In one possible implementation, the step of converting the 3D Delaunay tetrahedral mesh into a foreground and background image based on the foreground weights and the background weights includes: Using the tetrahedron as each node in the foreground and background graphs, the foreground weight is used as the weight for connecting the node to the foreground space, and the background weight is used as the weight for connecting the node to the background space. The foreground and background graphs are obtained by constructing the connection relationships between the nodes.

[0010] In one possible implementation, the step of performing globally optimal image segmentation between the foreground and background based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background includes: Based on the foreground and background graphs, the maximum flow between the foreground space and the background space is solved using the Incremental Breadth-First Search (IBFS) algorithm, resulting in the minimum cut that divides all the tetrahedrons into the foreground space and the background space. Based on the minimum cut result, the vertices of each cut triangle of all the tetrahedrons are filtered to obtain surface points and noise points.

[0011] Secondly, this application provides a lidar point cloud denoising device, comprising: The acquisition module is used to acquire data from the LiDAR point cloud and the sensor that observes the point cloud points in the LiDAR point cloud; The construction module is used to determine the corresponding observation relationship between each point cloud point and the pose of the sensor based on the data of the sensor, and to construct the line of sight corresponding to the observation space view between the point cloud point and the sensor based on the observation relationship; The determination module is used to construct a three-dimensional Delaunay tetrahedral mesh based on the point cloud points, and to determine the foreground weight in the foreground space and the background weight in the background space corresponding to the face of each three-dimensional Delaunay tetrahedral mesh based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh. The segmentation module is used to convert the three-dimensional Delaunay tetrahedral mesh into foreground and background images according to the foreground weight and the background weight, and to perform global optimal image segmentation between the foreground and background based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. The denoising module is used to filter and denoise the LiDAR point cloud based on the surface points and the noise points to obtain the denoising result.

[0012] Thirdly, this application also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect above.

[0013] Fourthly, this application also provides a computer-readable storage medium storing computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method described in the first aspect above.

[0014] This application brings the following beneficial effects: This application provides a method, apparatus, and electronic device for denoising LiDAR point clouds, capable of acquiring LiDAR point clouds and data from sensors observing the point clouds; determining the corresponding observation relationship between each point cloud point and the pose of the sensor based on the sensor data, and constructing a line of sight corresponding to the observation spatial viewpoint between the point cloud point and the sensor based on the observation relationship; constructing a three-dimensional Delaunay tetrahedral mesh based on the point cloud points, and determining the foreground weight and background weight of the face of each three-dimensional Delaunay tetrahedral mesh based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh; and denoising the three-dimensional Delaunay tetrahedral mesh based on the foreground weight and the background weight. The image is converted into foreground and background images, and global optimal image segmentation is performed between the foreground and background based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. The LiDAR point cloud is then filtered and denoised based on the surface points and noise points to obtain the denoising result. In this scheme, by observing from the sensor's perspective, the real surface points should satisfy line-of-sight connectivity and consistency between the normal and the incident angle. If a point is not visible from most perspectives, that is, it is occluded by its own point cloud, the angle between the normal and the sensor direction is too large, or there is a visibility conflict with neighboring points. For example, if a sudden change in the normal causes a point that should be occluded to not be occluded, it may be noise or an artifact. This scheme uses point clouds observed from multiple perspectives to remove isolated points, flying points, or spliced ​​error points by checking the visibility consistency of points in each frame, thereby improving the denoising accuracy of the LiDAR point cloud.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 A schematic flowchart illustrating the lidar point cloud denoising method provided in this application embodiment; Figure 2 This is another schematic flowchart of the lidar point cloud denoising method provided in the embodiments of this application; Figure 3 A schematic diagram of the weight construction process in the lidar point cloud denoising method provided in the embodiments of this application; Figure 4 The original point cloud before denoising is performed using the lidar point cloud denoising method provided in this application embodiment; Figure 5 The denoised point cloud is obtained by using the lidar point cloud denoising method provided in the embodiments of this application. Figure 6 This is a schematic diagram of the structure of a lidar point cloud denoising device provided in an embodiment of this application; Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this application, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0020] Currently, raw laser point clouds inevitably contain a large number of noise and outliers. The main sources include: multipath reflection and specular effect: the laser beam undergoes specular reflection or multiple bounces on smooth surfaces (such as glass, water, and metal), generating false echo points that deviate from the actual object surface and are distributed arbitrarily in space. Dynamic object afterimages: during mobile measurements, moving targets such as pedestrians and vehicles are repeatedly scanned at different times, forming trailing noise or floating discrete points in the air. Atmospheric scattering and sensor noise: false echoes are generated by atmospheric suspended particles such as rain, fog, and dust scattering the laser beam; sensors introduce significant ranging errors when measuring at long distances.

[0021] Existing point cloud filtering methods can be mainly classified into the following categories: Methods based on local statistical features (such as statistical outlier removal (SOR) and radius filtering) identify outliers by using neighborhood point density or distance statistics. These methods are computationally simple, but they only utilize local geometric information and lack the ability to make globally consistent judgments. They are also limited in effectiveness in densely noisy regions (such as area noise generated by multipath reflections), and their parameters are sensitive to changes in point cloud density.

[0022] Surface-fitting methods (such as moving least squares) achieve smoothing and downsampling through local surface estimation or spatial quantization. These methods are suitable for homogenization, but they struggle to distinguish between thin layers of noise near the real surface and actual ground features, and they are not well-suited to complex topologies.

[0023] Deep learning-based methods denoise by training a network to learn noise patterns. These methods rely on large amounts of labeled data, their generalization ability is limited by the distribution of the training scene, and they lack interpretability.

[0024] The common limitation of the above methods is that they do not fully utilize the active measurement characteristics of laser sensors—that is, the physical prior that each point is observed along a known line of sight (the line connecting the sensor and the target). The line of sight implies information about the spatial accessibility of the point: a true surface point should exhibit a consistent occlusion relationship in multiple viewpoints, while noisy points usually violate this geometric consistency.

[0025] Multiview Figure 3 While surface extraction methods based on visibility analysis in the field of 3D reconstruction utilize line-of-sight information, these methods are usually designed for sparse reconstruction scenarios of image feature points. When faced with the characteristics of ultra-large scale, high density, and anisotropy of laser point cloud data, they face engineering challenges such as excessive computational complexity and excessive memory consumption, and cannot be directly applied to actual production operations.

[0026] Based on this, the present application provides a method, apparatus, and electronic device for denoising LiDAR point clouds, which can solve the technical problem of low accuracy in LiDAR point cloud denoising.

[0027] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0028] Figure 1 This is a flowchart illustrating a lidar point cloud denoising method provided in an embodiment of this application. Figure 1 As shown, the method includes: Step S110: Acquire the LiDAR point cloud and the data from the sensor that observes the point cloud points in the LiDAR point cloud.

[0029] As a possible implementation, after acquiring the LiDAR point cloud as described above, the method may further include the following steps: dividing the spatial range corresponding to the LiDAR point cloud into voxel grids to discretize the spatial range into several uniform voxel grids; aggregating the voxel grids into initial spatial blocks according to a preset block dimension, and performing recursive binary refinement processing when the number of point cloud points in a single initial spatial block exceeds a preset threshold to obtain intermediate spatial blocks; adding a spatial boundary buffer to each intermediate spatial block to obtain a final spatial block, wherein the neighboring points in the buffer participate in the calculation of the current final spatial block but do not output the result.

[0030] For the process of dividing a large scene into blocks, for example, such as Figure 2 As shown, the input large-scale LiDAR point cloud is divided into several processing blocks according to its spatial location. Specifically, this includes: dividing the spatial range of the point cloud into voxel grids, assuming the voxel side length is... Discretize the space into a uniform voxel grid; divide the voxel grid according to a preset block dimension. Aggregates into spatial blocks; when the number of points within a single block exceeds a threshold... At this time, recursive binary refinement is performed; a spatial boundary buffer is added to each block, and the neighboring points within the buffer participate in the calculation of the current block but do not output the result, thus eliminating the gaps between blocks.

[0031] Step S120: Determine the corresponding observation relationship between each point cloud point and the sensor pose based on the sensor data, and construct the line of sight corresponding to the observation space view between the point cloud point and the sensor based on the observation relationship.

[0032] In one possible implementation, the sensor data includes the sensor's pose trajectory data; determining the corresponding observation relationship between each point cloud point and the sensor's pose based on the sensor data may specifically include the following steps: For each point cloud point in the final spatial block, the sensor pose of the corresponding sensor at each time step is determined based on the pose trajectory data. The target sensor pose index corresponding to the point cloud point is obtained by matching the most recent time based on the sensor pose and the acquisition time of the point cloud point. The corresponding observation relationship between the point cloud point and the sensor pose is established based on the target sensor pose index and the point cloud point.

[0033] For the process of sensor pose association and gaze construction, for example, such as Figure 2 As shown, the correspondence between each point in the point cloud and the sensor position observing that point is established: the sensor trajectory file is loaded to obtain the sensor pose at each time step. ,in For timestamps; if an external parameter calibration file exists, load the rigid body transformation from sensor to carrier. For each point cloud point Based on its collection timestamp Find the corresponding sensor pose index by recent time matching. Establish observation relationship .

[0034] Step S130: Construct a 3D Delaunay tetrahedral mesh based on point cloud points, and determine the foreground weight in the foreground space and the background weight in the background space corresponding to the face of each 3D Delaunay tetrahedral mesh based on the geometric relationship of the line of sight passing through the 3D Delaunay tetrahedral mesh.

[0035] In one optional implementation, the above-mentioned construction of a 3D Delaunay tetrahedral mesh based on point cloud points may specifically include the following steps: sorting all point cloud points in CGAL space to obtain an insertion order; inserting Delaunay triangulations point by point according to the insertion order, and performing distance downsampling during the insertion of Delaunay triangulations to obtain Delaunay vertices; discretizing the continuous space corresponding to the final spatial block into a finite number of 3D Delaunay tetrahedral meshes based on the Delaunay vertices; wherein each 3D Delaunay tetrahedral mesh corresponds to a tetrahedron containing four triangular faces, and adjacent tetrahedrons share common triangular faces.

[0036] For the construction of a three-dimensional Delaunay tetrahedral mesh, an example is as follows: Figure 2 As shown, a 3D Delaunay triangulation is constructed on the point cloud data. The purpose is to discretize the continuous space into a tetrahedral mesh. The specific implementation is as follows: All points are spatially sorted using CGAL to obtain an insertion order with good spatial locality, improving construction efficiency; Delaunay triangulations are inserted point by point, and distance downsampling is performed during the insertion process: for each point to be inserted... First, locate its position within the current Delaunay structure and find the nearest existing vertex. ;like If the distance downsampling threshold is not met, the point will not be inserted; only its line-of-sight observation information will be appended. If the distance is not less than d_{insert}, then insert that point as a new vertex. Then, maintain the information structure for each Delaunay vertex: the associated set of original vertex indices. and a list of observation space perspectives ,in From the perspective The cumulative observation weights are calculated (the weight increases by 1 for each point observed from the same viewpoint). After the Delaunay construction is completed, the space is discretized into a finite number of tetrahedral cells, each tetrahedron having 4 triangular faces, and adjacent tetrahedrons sharing common faces.

[0037] In some embodiments, the foreground weight in foreground space and the background weight in background space corresponding to the face of each three-dimensional Delaunay tetrahedron are determined based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedron mesh. Specifically, this may include the following steps: In all tetrahedrons, the common triangular faces shared between finite and infinite tetrahedrons are determined to obtain the convex hull surface. For each observation space viewpoint, the target tetrahedron containing the corresponding viewpoint is calculated. If the viewpoint is inside the convex hull surface, the four faces of the target tetrahedron are determined as the set of ray tracing starting faces. The foreground of the tetrahedron containing the camera's corresponding face is determined as the foreground space, and the occlusion space behind the object surface in the tetrahedron's face is determined as the background space. Based on the viewpoint and the vertices of the tetrahedron, ray traversal calculations are performed on each observation space viewpoint of all tetrahedron vertices using the set of ray tracing starting faces to obtain the calculation results. The face weights of the tetrahedrons are then determined based on the calculation results.

[0038] For tetrahedral weight calculation based on line-of-sight crossing, an example is... Figure 2 As shown, by utilizing the geometric relationship of the sensor's line of sight passing through the Delaunay mesh, foreground / background weights are assigned to each tetrahedral cell and its face to construct the energy function of the st graph; the specific implementation is as follows: like Figure 3 As shown, for the process of marking the convex hull surface, all tetrahedral elements are traversed, and the triangular faces of the convex hull surface (i.e., the common faces between finite tetrahedrons and infinite tetrahedrons) are collected; then, the cell where the camera is located is located: for each viewpoint k, the line-of-sight point is calculated. The tetrahedral unit where it is located If the viewpoint is inside the convex hull, obtain the four faces of that unit as the set of starting faces for ray tracing; set the source weight of the tetrahedron containing the camera to infinity. That is, force it to be marked as foreground; then, traverse all Delaunay vertices, for each vertex Each observation space perspective Perform ray crossing calculations: like Figure 3 As shown, construct from the viewpoint To the top ray segment ; Calculate the adaptive local scaling parameters: This parameter is determined based on the distance from the point to the observation center and the laser beam divergence angle; the greater the distance, the greater the spatial uncertainty. For forward ray crossing, such as Figure 3 As shown, starting from the tetrahedral element where the viewpoint is located, the sequence of tetrahedral faces through which the ray passes is traced face by face. For each crossing surface Calculate the distance from the face to the target vertex. Accumulated surface weights: ; in, This is a global scale parameter. Accumulated positive weights to the corresponding surface Regarding the physical meaning of the above weight design, for example, the source weight... The tetrahedron containing the camera must be in free space (foreground); sink weight. The occlusion space (background) behind the object's surface represented by a tetrahedron; face weight. The more surfaces a line of sight passes through and the closer it is to the surface, the higher the cost of cutting it, and the less likely it is to be cut.

[0039] Step S140: The 3D Delaunay tetrahedral mesh is transformed into foreground and background images according to the foreground weight and background weight, and the global optimal image segmentation between the foreground and background is performed based on the foreground and background images to obtain the surface points corresponding to the foreground and the noise points corresponding to the background.

[0040] As an optional implementation, the above-described transformation of the 3D Delaunay tetrahedral mesh into a foreground and background image based on foreground and background weights may specifically include the following steps: using the tetrahedron as each node in the foreground and background image, using the foreground weight value as the weight for connecting the node to the foreground space, using the background weight value as the weight for connecting the node to the background space, and obtaining the foreground and background image by constructing the connection relationship between nodes.

[0041] For the specific process of solving the global optimization problem of the ST graph cut, an example is as follows: Figure 2 As shown, the Delaunay tetrahedral mesh is transformed into an ST graph, and the globally optimal foreground-background segmentation is solved using the Incremental Breadth-First Search (IBFS) algorithm. This involves constructing a graph structure: each node in the graph corresponds to a tetrahedral element; the weight connecting each node to the source is equal to the weight of that element. The value, the weight of the connection to the sink is Value; adjacent tetrahedrons are connected by an edge through a common face, and the weight is represented as: ;in, Tetrahedron No. The cumulative cross-surface weight, For surface quality regularization: ; It is the normalized angle between the surface normal vector and the vertex circumscribed sphere normal vector, used to suppress false cut surfaces generated by the elongated tetrahedron.

[0042] In some embodiments, global optimal image segmentation between the foreground and background is performed based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. This may specifically include the following steps: Based on the foreground and background graphs, the maximum flow / minimum cut IBFS algorithm is used to solve the maximum flow in the foreground and background spaces, and the minimum cut result that divides all tetrahedrons into foreground and background spaces is obtained. Based on the minimum cut result, the vertices of each cut triangle of all tetrahedrons are filtered to obtain surface points and noise points.

[0043] For solving the maximum flow problem, for example, such as Figure 2 As shown, the IBFS (Incremental Breadth-First Search) algorithm is used to solve the maximum flow of st, and the minimum cut is obtained to divide all tetrahedral cells into the source side (foreground / free space) and the sink side (background / occupied space).

[0044] For extracting surface points, for example, all common faces of tetrahedrons are traversed to find the secant face (the common face of two adjacent tetrahedrons belonging to different sides). Simultaneously, the vertices of each secant face triangle are filtered: if the lengths of the two triangle sides connected to that vertex both exceed [a certain value], then [the secant face is considered valid]. If the number of original points associated with this vertex is [number], it is considered a noise isolated point; Vertices that are filtered out are considered unreliable vertices; the original vertices associated with each filtered vertex are indexed. Marked as surface points.

[0045] Step S150: Filter and denoise the LiDAR point cloud based on surface points and noise points to obtain the denoising result.

[0046] In practical applications, the original point cloud before denoising is obtained using the lidar point cloud denoising method provided in this application embodiment, such as... Figure 4 As shown, the denoised point cloud after denoising using the lidar point cloud denoising method provided in this application embodiment is as follows: Figure 5 As shown.

[0047] This method can be used as a radar point cloud surface point extraction method that integrates sensor line-of-sight geometric constraints and graph-cut global optimization. It is suitable for noise filtering of LiDAR point clouds and automated extraction of high-confidence ground surface points in large-scale outdoor scenes. It can make full use of the line-of-sight geometric prior of LiDAR sensors, has global optimization capabilities, and is also suitable for efficient point cloud denoising and surface point extraction methods for large-scale point cloud data processing.

[0048] In this embodiment of the application, in the spatial visibility analysis for point cloud denoising, it is assumed that, from the sensor's perspective, real surface points should satisfy line-of-sight connectivity and normal-incident angle consistency. If a point is invisible from most perspectives (occluded by its own point cloud, or the angle between its normal and the sensor direction is too large) or has visibility conflicts with neighboring points (such as a sudden change in normal causing it to be occluded when it should be), it may be noise or an artifact. By using point clouds observed from multiple perspectives, isolated points, flying points, or points with stitching errors are eliminated by checking the visibility consistency of points in each frame.

[0049] Figure 6 A schematic diagram of a lidar point cloud denoising device is provided. Figure 6 As shown, the lidar point cloud denoising device 600 includes: The acquisition module 601 is used to acquire data from the LiDAR point cloud and the sensor that observes the point cloud points in the LiDAR point cloud; The construction module 602 is used to determine the corresponding observation relationship between each point cloud point and the pose of the sensor based on the data of the sensor, and to construct the line of sight corresponding to the observation space view between the point cloud point and the sensor based on the observation relationship; The determination module 603 is used to construct a three-dimensional Delaunay tetrahedral mesh based on the point cloud points, and to determine the foreground weight and background weight of the face of each three-dimensional Delaunay tetrahedral mesh based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh. The segmentation module 604 is used to convert the three-dimensional Delaunay tetrahedron mesh into a foreground and background image according to the foreground weight and the background weight, and to perform global optimal image segmentation between the foreground and background based on the foreground and background image to obtain the surface points corresponding to the foreground and the noise points corresponding to the background. The denoising module 605 is used to filter and denoise the LiDAR point cloud based on the surface points and the noise points to obtain the denoising result.

[0050] The lidar point cloud denoising device provided in this application embodiment has the same technical features as the lidar point cloud denoising method provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0051] An electronic device provided in this application embodiment, such as Figure 7 As shown, the electronic device 700 includes a processor 702 and a memory 701. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps of the method provided in the above embodiments.

[0052] See Figure 7 The electronic device also includes a bus 703 and a communication interface 704. The processor 702, the communication interface 704 and the memory 701 are connected via the bus 703. The processor 702 is used to execute executable modules, such as computer programs, stored in the memory 701.

[0053] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 704 (which can be wired or wireless), such as the Internet, wide area network, local area network, or metropolitan area network.

[0054] Bus 703 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0055] The memory 701 is used to store programs. After receiving an execution instruction, the processor 702 executes the program. The method executed by the apparatus defined by the process disclosed in any of the preceding embodiments of this application can be applied to the processor 702 or implemented by the processor 702.

[0056] The processor 702 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 702 or by instructions in software form. The processor 702 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 701, and processor 702 reads the information from memory 701 and, in conjunction with its hardware, completes the steps of the above method.

[0057] Corresponding to the above-described lidar point cloud denoising method, this application embodiment also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and run by a processor, the computer-executable instructions cause the processor to perform the steps of the above-described lidar point cloud denoising method.

[0058] The lidar point cloud denoising device provided in this application embodiment can be specific hardware on the device or software or firmware installed on the device. The implementation principle and technical effects of the device provided in this application embodiment are the same as those in the foregoing method embodiments. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the foregoing method embodiments. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can all be referred to the corresponding processes in the above method embodiments, and will not be repeated here.

[0059] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0060] For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0061] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0062] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0063] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the lidar point cloud denoising method described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0064] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0065] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for denoising point clouds using lidar, characterized in that, The method includes: Acquire data from LiDAR point clouds and sensors that observe point cloud points in the LiDAR point clouds; Based on the data from the sensor, determine the corresponding observation relationship between each point cloud point and the pose of the sensor, and construct the line of sight corresponding to the observation spatial perspective between the point cloud point and the sensor based on the observation relationship; A three-dimensional Delaunay tetrahedral mesh is constructed based on the point cloud points, and the foreground weight and background weight of the face of each three-dimensional Delaunay tetrahedral mesh are determined based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh. The three-dimensional Delaunay tetrahedral mesh is transformed into foreground and background images based on the foreground and background weights, and global optimal image segmentation between the foreground and background is performed based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. The LiDAR point cloud is filtered and denoised based on the surface points and the noise points to obtain the denoising result.

2. The method according to claim 1, characterized in that, After acquiring the LiDAR point cloud, the method further includes: The spatial range corresponding to the LiDAR point cloud is divided into voxel grids to discretize the spatial range into several uniform voxel grids. The voxel mesh is aggregated into an initial spatial block according to a preset block dimension, and when the number of point cloud points in a single initial spatial block exceeds a preset threshold, a recursive binary refinement process is performed to obtain an intermediate spatial block. A spatial boundary buffer is added to each intermediate spatial block to obtain the final spatial block. Neighboring points within the buffer participate in the calculation of the current final spatial block but do not output the result.

3. The method according to claim 2, characterized in that, The sensor data includes the sensor's pose trajectory data; determining the corresponding observation relationship between each point cloud point and the sensor's pose based on the sensor data includes: For each point cloud point in the final spatial block, the sensor pose of the sensor corresponding to the point cloud point at each time moment is determined based on the pose trajectory data. Based on the sensor pose and the acquisition time of the point cloud points, the target sensor pose index corresponding to the point cloud points is obtained by matching the most recent time. Based on the target sensor pose index and the point cloud points, establish the corresponding observation relationship between the point cloud points and the pose of the sensor.

4. The method according to claim 2, characterized in that, The construction of a 3D Delaunay tetrahedral mesh based on the point cloud points includes: Perform CGAL spatial sorting on all the point cloud points to obtain the insertion order; Delaunay triangulations are inserted point by point according to the insertion order, and distance downsampling is performed during the insertion of the Delaunay triangulations to obtain the Delaunay vertices. Based on the Delaunay vertices, the continuous space corresponding to the final space block is discretized into a finite number of three-dimensional Delaunay tetrahedral meshes; wherein, each tetrahedral mesh contains four triangular faces, and adjacent tetrahedrals share common triangular faces.

5. The method according to claim 4, characterized in that, The determination of the foreground weight in foreground space and the background weight in background space corresponding to each face of the 3D Delaunay tetrahedron mesh based on the geometric relationship of the line of sight passing through the 3D Delaunay tetrahedron mesh includes: In all the tetrahedrons, the common triangular face shared between the finite tetrahedrons and the infinite tetrahedrons is determined to obtain the convex hull surface; For each observation space perspective, calculate the target tetrahedron where the corresponding line of sight is located. If the line of sight is inside the convex hull surface, then determine the four faces of the target tetrahedron as the set of ray tracing starting faces, and determine the foreground of the face of the tetrahedron containing the camera as the foreground space, and determine the occlusion space of the face of the tetrahedron located behind the object surface as the background space. Based on the viewpoint and the vertices of the tetrahedron, ray traversal calculations are performed on each of the observation space perspectives of all the vertices of the tetrahedron using the set of ray tracing starting faces to obtain the calculation results, and the face weights of the tetrahedron are determined based on the calculation results.

6. The method according to claim 5, characterized in that, The step of converting the 3D Delaunay tetrahedral mesh into a foreground and background image based on the foreground weight and the background weight includes: Using the tetrahedron as each node in the foreground and background graphs, the foreground weight is used as the weight for connecting the node to the foreground space, and the background weight is used as the weight for connecting the node to the background space. The foreground and background graphs are obtained by constructing the connection relationships between the nodes.

7. The method according to claim 6, characterized in that, The step of performing globally optimal image segmentation between the foreground and background based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background includes: Based on the foreground and background graphs, the maximum flow in the foreground space and the background space is solved using the maximum flow / minimum cut IBFS algorithm, resulting in the minimum cut that divides all the tetrahedrons into the foreground space and the background space. Based on the minimum cut result, the vertices of each cut triangle of all the tetrahedrons are filtered to obtain surface points and noise points.

8. A lidar point cloud denoising device, characterized in that, include: The acquisition module is used to acquire data from the LiDAR point cloud and the sensor that observes the point cloud points in the LiDAR point cloud; The construction module is used to determine the corresponding observation relationship between each point cloud point and the pose of the sensor based on the data of the sensor, and to construct the line of sight corresponding to the observation space view between the point cloud point and the sensor based on the observation relationship; The determination module is used to construct a three-dimensional Delaunay tetrahedral mesh based on the point cloud points, and to determine the foreground weight in the foreground space and the background weight in the background space corresponding to the face of each three-dimensional Delaunay tetrahedral mesh based on the geometric relationship of the line of sight passing through the three-dimensional Delaunay tetrahedral mesh. The segmentation module is used to convert the three-dimensional Delaunay tetrahedral mesh into foreground and background images according to the foreground weight and the background weight, and to perform global optimal image segmentation between the foreground and background based on the foreground and background images to obtain surface points corresponding to the foreground and noise points corresponding to the background. The denoising module is used to filter and denoise the LiDAR point cloud based on the surface points and the noise points to obtain the denoising result.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 7.

Images