A transient map rendering method for multipath noise in a confocal scanning rangefinding system

Abstract

This invention discloses a transient image rendering method for multipath noise in confocal scanning ranging systems. The method includes the following steps: (1) obtaining an OBJ format file, generating a three-dimensional voxel map based on the three-dimensional scene information described in the OBJ format file, and generating a two-dimensional depth map from the three-dimensional voxel map; (2) performing simulated confocal scanning laser ranging on each point in the two-dimensional depth map, considering multipath noise during the simulation; (3) combining the one-dimensional reflected light signal time histograms obtained from scanning each point into a three-dimensional transient image containing multipath noise. This invention solves the current problem of lacking a three-dimensional transient image rendering method for multipath noise in confocal scanning laser ranging systems. The proposed rendering method is based on actual physical processes, possessing strong scientific validity and rigor. The simulated three-dimensional transient image can effectively simulate the multipath noise effect in actual confocal scanning ranging systems.

Description

Technical Field

[0001] This invention belongs to the fields of Light Detection and Ranging (LiDAR), optical imaging, and 3D reconstruction, and specifically relates to a method for rendering transient 3D scenes with multipath noise in confocal scanning ranging systems. Background Technology

[0002] Traditional RGB cameras can only record planar images containing color and brightness information. This is essentially a projection of the real-world 3D scene onto the plane of the camera film or other photosensitive device, without including depth information of objects within the scene. For computer vision systems, the imaging system and computer processing equipment should have the ability to intelligently and automatically analyze and understand the captured visual information. Two-dimensional images containing only projection information often cannot meet the needs of intelligent and automated computer vision systems. Therefore, recording the depth information of objects in the scene during filming is highly significant. Imaging techniques that can simultaneously record the length, width, and depth coordinates of each point in the scene are called 3D imaging techniques.

[0003] Radar systems are a crucial means of recording depth information of objects in a detected scene. They calculate the distance between the detection system and the target by measuring the flight time of the transmitted signal in space; this three-dimensional imaging method is called Time-of-Flight (ToF) imaging. Besides traditional radars that initially operated in the radio wave band, lidar operating in the visible and infrared bands is currently popular for short-range (within 100m) three-dimensional imaging. A confocal scanning laser ranging system is an optical ranging system based on lidar theory and technology. Its key feature is that it first obtains a narrow, concentrated laser beam through beam shaping technology and lens focusing. Then, its scanning capability is demonstrated by illuminating only one point in a three-dimensional scene at a time, collecting the reflected light signal from that point through a photoelectric conversion device. Subsequently, a voltage-controlled galvanometer group changes the deflection angle of the laser beam, allowing it to illuminate the next point in the scene, and repeating the process of collecting reflected light signals to achieve scanning and ranging of the entire scene. In addition, its confocality is reflected in the fact that by reasonably setting the spatial positions of various optical devices (including lasers, photoelectric conversion devices, lenses, beam splitters, etc.), the convergence points formed by the emitted laser beam and the reflected light signal after passing through the lens can almost coincide, so that the position of the photoelectric converter for light signal detection can be equivalent to the position illuminated by the laser at this time.

[0004] In Time-of-Flight (ToF) laser ranging systems, multipath noise is a common and unavoidable noise phenomenon. It refers to the fact that during the detection of a single scanning point in a scene, the photoelectric converter receives not only the direct reflected light from the illuminated point but also multiple indirect reflected lights from other objects in the scene due to the diffuse reflection effect of the laser at the illuminated point. When multipath noise is present, the actual light signal received by the photoelectric converter is equivalent to a linear combination of the direct reflected light and several multi-level reflected lights. Because multipath noise violates the fundamental assumption in ToF imaging systems that "the reflected light signal only undergoes one direct reflection from the illuminated point," it can introduce significant errors into the ranging results.

[0005] To investigate the impact and characteristics of multipath noise on the accuracy of 3D scene reconstruction based on Time-of-Flight (ToF) laser ranging systems, many researchers have developed simulation tools to render 3D transient maps that take into account multipath noise effects from input 3D scene information. These tools have been used to generate and publish a number of 3D transient map datasets containing multipath noise effects, such as the ToF-FlyingThings3D (TFT3D) and FLAT datasets. However, the illumination and detection methods used by these current tools and datasets designed to simulate multipath noise effects differ significantly from those of confocal scanning laser ranging systems, thus rendering them unsuitable as tools or data foundations for studying multipath noise effects in confocal scanning laser ranging systems. Summary of the Invention

[0006] To address the current lack of a method for rendering 3D transient images of multipath noise in confocal scanning laser ranging systems, this invention proposes a novel method for rendering 3D scene transient images. This method is based on specific actual physical processes, possessing strong scientific rigor and precision. The simulated 3D transient images can effectively simulate the multipath noise effects in actual confocal scanning ranging systems.

[0007] The technical solution adopted in this invention is as follows:

[0008] A transient graph rendering method for multipath noise in a confocal scanning ranging system includes the following steps:

[0009] Step 1: Obtain an OBJ format file, generate a 3D voxel map based on the 3D scene information described in the OBJ format file, and generate a 2D depth map from the 3D voxel map.

[0010] Step 2: Perform simulated confocal scanning laser ranging on each point in the two-dimensional depth map, taking multipath noise into account during the simulation.

[0011] Step 3: Combine the time histograms of the one-dimensional reflected light signals obtained from scanning each point into a three-dimensional transient graph containing multipath noise.

[0012] Further, in step 1, the steps for generating the 3D voxel map are as follows: set the face type of the geometry in the obj format file to "triangle"; according to the 3D spatial coordinates and normal vector information of the three vertex indices of a certain surface recorded in the obj format file, mark the voxel units located inside the closed triangle formed by these three vertices in the 3D voxel map; perform the above marking operation on each surface in sequence to obtain the complete 3D voxel map corresponding to the 3D scene.

[0013] Further, in step 2, the specific steps for simulation are as follows: For the current scanning illumination point, firstly, the direct reflection intensity of the light signal is calculated and used as the initial value of the one-dimensional reflected light signal intensity time histogram of the current scanning illumination point; then, combined with the three-dimensional voxel map obtained in step 1, all possible secondary reflection light paths are searched in the three-dimensional scene using a depth-first search algorithm; next, for each secondary reflection light path, the intensity of the secondary reflected light signal after multi-level reflection through multiple voxel points and the distance traveled in this process are calculated, and the time delay caused by traveling this distance is further derived; finally, the secondary reflected light signal formed by the current secondary reflection light path is delayed for a period of time on the time axis according to the calculated time delay, and then superimposed on the one-dimensional reflected light signal intensity time histogram of the current scanning illumination point as a simulation of multipath noise.

[0014] Furthermore, in step 2, during the search process, the search operation is stopped when the secondary reflected light signal attenuates below a certain threshold or the number of reflection levels of the secondary reflected light path reaches a certain upper limit.

[0015] Compared to existing methods for rendering transient images of 3D scenes containing multipath noise, the innovation of this invention lies in the following: the proposed method solves the problem of the lack of a 3D transient image rendering method for multipath noise in confocal scanning ranging systems; it analyzes and studies the actual physical process of completing the ranging task in a confocal scanning ranging system, and establishes a reasonable mathematical model based on this, and finally implements the mathematical model by designing software programs; the 3D transient image rendering results containing multipath noise can basically and reasonably reflect the influence of multipath noise in the confocal scanning ranging system. Attached Figure Description

[0016] The embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0017] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0018] Figure 2Considering multipath noise, (a) the one-dimensional reflected light signal intensity time histogram obtained when illuminating a voxel point in the scene with relatively obvious multipath noise; (b) the one-dimensional reflected light signal intensity time histogram obtained when illuminating a voxel point in the scene with insignificant multipath noise.

[0019] Figure 3 (a) is the true depth map value of a synthetic 3D scene; (b) is the 2D depth map reconstructed from the 3D transient map of the scene based on the rendering when there is no multipath noise; (c) is the 2D depth map reconstructed from the 3D transient map of the scene based on the rendering when there is multipath noise. Detailed Implementation

[0020] The embodiments of the present invention will be described in detail below. These embodiments are implemented based on the technical solution of the present invention, and detailed implementation methods and specific operation processes are given. However, the scope of protection of the present invention is not limited to the following embodiments.

[0021] Figure 1 This is a schematic diagram of the overall process of this embodiment, which includes the following steps:

[0022] 1. Obtain the 3D scene information described by the .obj file to generate a 3D voxel map and a 2D depth map. The .obj file can be obtained through visual, graphical, interactive 3D modeling tools, or by converting 3D scene files of other formats. The 2D depth map is generated from the 3D voxel map.

[0023] 2. Perform simulated confocal scanning laser ranging on each pixel in the 2D depth map sequentially, and render a one-dimensional reflected light signal intensity time histogram received by the laser receiver when the emitted laser illuminates the point, based on the simulation method considering multipath noise proposed in this invention. The specific implementation steps of the one-dimensional reflected light signal intensity time histogram simulation method considering multipath noise are as follows:

[0024] For the current scanning illumination point, the direct reflection intensity of the light signal is first calculated and used as the initial value of the one-dimensional reflected light signal intensity time histogram of the current illumination point. Then, combined with the three-dimensional voxel map obtained in step 1, a depth-first search algorithm is used to search for all possible secondary reflection light paths in three-dimensional space. For each secondary reflection light path, the intensity of the secondary reflected light signal after multiple reflections through multiple voxel points and the distance traveled during this process are calculated, and the time delay caused by this distance is further derived. Finally, the secondary reflected light signal formed by the secondary reflection light path is delayed by a certain period on the time axis according to the calculated time delay and superimposed on the one-dimensional reflected light signal intensity time histogram of the current illumination point as a simulation of multipath noise. Specifically, for the k-th order (referring to the emitted laser passing through k voxel points and undergoing k reflections), the voxel point p reached by this order of reflected light... k The intensity of the reflected light signal A k The recursive formula is:

[0025]

[0026] In the formula, p k For point p k 3D coordinates; For point p k The normal vector; Given p k and its corresponding The optical signal energy coefficient is calculated based on the Bidirectional Reflectance Distribution Function (BRDF); D(p1,p2) is the Euclidean distance between points p1 and p2; Atten[D(p1,p2)] is the optical signal energy attenuation coefficient calculated given D(p1,p2); Albedo[p k [Point p] k The reflectivity.

[0027] Additionally, point p k Multipath noise (MPI) introduced into the reflected light signal received by the system k The calculation formula is as follows:

[0028]

[0029] In the formula, p0 refers to the three-dimensional coordinates of the laser receiver's spatial location. In a confocal laser ranging system, p0 is also equal to the three-dimensional coordinates of the laser transmitter's spatial location.

[0030] 3. After the simulation of scanning all pixels in the depth map is completed, the time histograms of the one-dimensional reflected light signal intensity simulated when the laser is emitted to illuminate each point are combined to obtain the three-dimensional transient map corresponding to the synthesized three-dimensional scene, which includes the simulation of multipath noise.

[0031] The method of this embodiment will now be described in detail:

[0032] Step 1: Synthesize virtual 3D scenes using 3D modeling software and save them as .obj format files. In this embodiment, several virtual 3D scenes were synthesized using 3ds Max 2018 3D modeling software. Each 3D scene consists of a 3D object from the Stanford 3D Scanning Repository dataset and several simple geometric shapes (including cuboids, spheres, cylinders, cones, tubular bodies, etc.). The 3D coordinates of all vertices of each geometric shape range from [0, 64].

[0033] The .obj file exported by 3D modeling software is actually a plain text file describing 3D scene information. This file specifically describes the object name, vertex coordinates, vertex normals, and surface vertex indices of each object in the 3D scene. When exporting the .obj file, the face type of the exported geometry must be set to "triangle". Therefore, each surface recorded in the .obj file contains three vertex indices. Based on these vertex indices, the 3D spatial coordinates and normals of these three vertices can be found. Subsequently, the voxel units located inside the closed triangles formed by these three vertices are marked in the 3D voxel map. Performing the above marking operation on each surface recorded in the .obj file sequentially forms a complete 3D voxel map corresponding to the 3D scene. Finally, performing an argmax operation on the depth axis of the obtained 3D voxel map yields the corresponding 2D depth map. The spatial resolution of the 3D voxel map is 64×64×1000, and the spatial resolution of the depth map is 64×64. This invention can implement the reading of the above .obj file and the conversion of it into a 3D voxel map and a 2D depth map using the Python programming language.

[0034] Step 2: First, generate the BRDF characteristic curve. The BRDF characteristic curve describes the energy distribution of the reflected light signal in different directions centered on the surface's normal vector when a light signal of a certain energy is incident on the surface. Experiments show that the larger the angle between the direction vector of the reflected light and the surface's normal vector, the smaller the fraction of the reflected light signal's energy into the total reflected light signal energy. This embodiment uses the GGX model as the BRDF characteristic curve.

[0035] Subsequently, an attenuation coefficient curve is generated, showing the energy decrease of the optical signal as the propagation distance increases. In this embodiment, the attenuation coefficient curve is a quadratic function curve, meaning the energy of the optical signal is negatively correlated with the square of the propagation distance. Based on the attenuation coefficient curve, the energy attenuation coefficient of the optical signal at a given propagation distance can be determined.

[0036] Next, simulated confocal scanning laser ranging is performed on each pixel in the two-dimensional depth map in sequence. Based on the simulation method proposed in this invention that takes into account multipath noise, a time histogram of the intensity of the one-dimensional reflected light signal received by the laser receiver when the emitted laser illuminates the point is rendered. Figure 2 This demonstrates the one-dimensional time histograms of reflected light signal intensity obtained when illuminating a voxel point in a scene with relatively obvious multipath noise and one without obvious multipath noise. For example... Figure 2 As shown in (a), when the multipath noise effect is not significant, the obtained one-dimensional reflected light signal intensity time histogram has an ideal shape, that is, the entire one-dimensional histogram has an ideal single-pulse reflected signal only at the time delay corresponding to the depth of the illuminated point. However, as... Figure 2 As shown in (b), when the multipath noise effect is more obvious, in addition to the single pulse at the time delay corresponding to the depth of the illuminated point, there are other signals in the one-dimensional histogram. These signals are caused by the multi-level reflection of light reflected by other voxel points in the three-dimensional scene to the illuminated point, which causes multipath noise.

[0037] Step 3 involves performing simulated scanning and ranging on 64×64 points in the 2D depth map sequentially, resulting in 64×64 one-dimensional reflected light signal energy time histograms, each with a time axis length of 1000. Combining these one-dimensional histograms according to the order of the emission points yields a complete 3D transient image rendering result with a spatial resolution of 64×64×1000.

[0038] Figure 3 This paper presents the ground truth depth map value of a synthetic 3D scene, and the 2D depth map reconstructed from the rendered 3D transient image of the same scene, with and without considering multipath noise. The 2D depth map is obtained by performing an argmax operation on the rendered 3D transient image along the time axis. Figure 3 As shown in (b), when multipath noise is not considered, the 2D depth map reconstructed from the rendered 3D transient map is exactly the same as the true 2D depth map of the 3D scene. However, as... Figure 3 As shown in (c), when multipath noise is considered, the 2D depth map reconstructed from the rendered 3D transient map shows that in areas with strong multipath noise (such as corners), the manually added rounded rectangles are in... Figure 3(c) indicates that the depth value error is relatively large. This also verifies that multipath noise can bring significant errors to the depth map reconstruction results.

Claims

1. A method of transient map rendering against multi-path noise in a confocal scanning rangefinding system, characterized by, The method comprises the following steps: Step 1, obtaining an obj format file, generating a three-dimensional voxel map according to the three-dimensional scene information described in the obj format file, and generating a two-dimensional depth map from the three-dimensional voxel map; Step 2, simulating confocal scanning laser ranging for each point in the two-dimensional depth map, and considering multipath noise in the simulation process; the specific steps of the simulation process are as follows: For the current scanning irradiation point, firstly, the direct reflection intensity of the light signal is calculated and taken as the initial value of the one-dimensional reflection light signal intensity time histogram of the current scanning irradiation point; then, all possible secondary reflection light paths in the three-dimensional scene are searched through a depth-first search algorithm combined with the three-dimensional voxel map obtained in step 1; then, for each secondary reflection light path, the secondary reflection light signal intensity after multi-level reflection of the light path through multiple voxel points and the distance passed in the process are calculated, and the time delay caused by the distance is further derived; finally, the secondary reflection light signal formed by the current secondary reflection light path is superimposed on the one-dimensional reflection light signal intensity time histogram of the current scanning irradiation point after being delayed for a period of time on the time axis according to the calculated time delay, as a simulation of multipath noise; Step 3, combining the one-dimensional reflection light signal time histograms obtained by scanning each point into a three-dimensional transient map containing multipath noise.

2. The method of rendering a transient map for multipath noise in a confocal scanning rangefinder system of claim 1, wherein, In step 1, the generation of the three-dimensional voxel map is as follows: The face type of the geometry of the obj format file is set to "triangle"; according to the three-dimensional space coordinates and normal vector information of the three vertex indexes recorded in the obj format file, the voxel units located inside the closed triangle formed by the three vertices are marked in the three-dimensional voxel map; the above marking operation is performed on each surface in turn to obtain the complete three-dimensional voxel map corresponding to the three-dimensional scene.

3. The method of claim 1, wherein: In step 2, in the search process, when the secondary reflection light signal decays below a certain threshold or the reflection level of the secondary reflection light path reaches a certain upper limit, the search operation is stopped.

Images