Underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization

Abstract

The application provides an underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization, comprising the following steps: reconstructing the original echo data of the underwater single-photon lidar into a superimposed photon number sequence corresponding to X*Y pixel points, wherein the superimposed photon number sequence corresponding to each pixel point is generated by performing photon number superposition based on a time bin grid on the multiple laser pulse echo signals received at its position, and the number M of laser pulses for performing photon number superposition on each pixel point is the same; performing depth-reflectivity estimation based on modal fusion on the superimposed photon number sequence of each pixel point to obtain a depth map and a reflectivity map of the underwater target; and optimizing the depth map and the reflectivity map of the underwater target based on a collaborative variational imaging optimization strategy. Using the method provided by the application, a high-quality reconstructed image of the underwater target can be quickly obtained.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of single-photon lidar imaging, and relates to an underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization. BACKGROUND

[0002] With the deepening of ocean exploration, long-distance and high-resolution imaging of underwater slow-moving targets has become a basic technology to support many other ocean exploration technologies, and has attracted great attention. In the traditional underwater target detection technology, the optical detection method is affected by the problems of light attenuation and low contrast in the underwater imaging environment, resulting in a decline in image quality; the current mainstream acoustic detection is limited by the speed of sound wave propagation and resolution, and the signal attenuation and distortion caused by the non-uniformity of seawater medium, which greatly affects the reliability and practicability.

[0003] The emergence of single-photon lidar provides a new way for underwater detection, and becomes an effective way to solve the problem of detecting targets in weak light and high light attenuation intensity environment. At present, the single-photon lidar system using scanning imaging can perform single-photon level detection on underwater slow-moving targets in the hardware aspect, but with the improvement of detection ability brought by the photon level detection sensitivity, the noise points caused by the noise inevitably increase, and the randomness of detection is also brought. In addition, the information loss caused by serious light attenuation greatly interferes with the selection and reconstruction of target information, and finally affects the detection result. Therefore, a robust imaging algorithm is indispensable to improve the detection accuracy of the single-photon detection system.

[0004] According to the differences in imaging mechanism, data processing means and technical implementation, the existing single-photon lidar imaging algorithms are mainly divided into two categories: imaging based on neural network and imaging based on traditional modeling method. The imaging algorithm based on traditional modeling method has low computing demand and flexible parameter adjustment, and has been widely used. Among the commonly used imaging algorithms based on traditional modeling method, the non-fixed pixel scanning time reconstruction method is represented by the first-photon algorithm and the first-photon group algorithm based on the "first" photon information to reconstruct the target, for example, Peng Xiao et al. proposed a first-photon algorithm (Peng X, Zhao X Y, Li L J, et al. First-photon imaging via a hybrid penalty [J]. Photonics Research, 2020, 8(3): 325-330.) based on the arrival time and echo intensity of the first photon to reconstruct the target image. However, this algorithm is easy to identify strong noise as target reflection signal, and is not suitable for complex environment, especially for low signal-to-noise ratio environment; for example, Chinese invention patent CN112305560A proposes a single-photon lidar imaging algorithm based on first-photon group, but the criterion for screening the first-photon group is greatly affected by the statistical characteristics of the echo signal, so it is not suitable for complex environment, especially for underwater environment, and needs to be manually adjusted for different detection environments and detection processes. And in order to realize the reconstruction of the same underwater target under different conditions, the detection time required is not fixed. Although the fixed pixel acquisition time reconstruction method sacrifices some flexibility and resource efficiency, it is suitable for standardized fast imaging due to its consistency and simplicity. However, the existing fixed pixel acquisition time reconstruction method uses median filtering, which ignores the inherent spatio-temporal relationship and signal distribution characteristics of single-photon data, and also loses a lot of edge information and high-frequency components. In the underwater environment with serious backscattering, the non-local spatial pixel filtering with excellent filtering performance also loses its robustness. SUMMARY

[0005] The purpose of the present application is to provide an underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization, which can fully utilize the different statistical characteristics of single-photon accumulation corresponding to noise and scene information in echo signals, and can efficiently complete the target reconstruction task under various challenging imaging conditions without manual parameter adjustment. The method includes the following steps:

[0006] The original echo data of the underwater single-photon lidar is reconstructed into a superimposed photon number sequence corresponding to X*Y pixel points, wherein the superimposed photon number sequence corresponding to each pixel point is generated by superimposing the photon numbers of the echo signals of multiple laser pulses received at the position of the pixel point based on a time bin grid, and the number M of laser pulses for superimposing the photon numbers of each pixel point is the same;

[0007] The superimposed photon number sequence of each pixel point is subjected to depth-reflectivity estimation based on modal fusion to obtain a depth map and a reflectivity map of the underwater target;

[0008] The depth map and the reflectivity map of the underwater target are optimized based on a collaborative variational imaging optimization strategy.

[0009] Further, the depth-reflectivity estimation based on modal fusion for the superimposed photon number sequence of any pixel point comprises the following steps:

[0010] A sliding window of a preset length is used to stepwise slide over the superimposed photon number sequence of the pixel point with a time bin grid as a step to intercept multiple candidate groups;

[0011] The candidate group with the most photon numbers in the group is selected as the group where the target signal is located;

[0012] The depth of the underwater target corresponding to the pixel point is determined based on the time bin grid number corresponding to the maximum value element in the group where the target signal is located, and

[0013] The reflectivity of the underwater target corresponding to the pixel point is determined based on the sum of the photon numbers in the group where the target signal is located.

[0014] Preferably, the underwater target is a slow-moving target, and the lower limit of M is 5 times and / or the length of the sliding window is between 5 time bins and 9 time bins.

[0015] Preferably, the underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization further comprises a signal-noise separation operation based on the statistical characteristics of the photon number distribution for the superimposed photon number sequence of each pixel point, and the signal-noise separation operation comprises at least one of a mask operation and a matched filtering.

[0016] Further, the mask operation specifically is:

[0017]

[0018] wherein x and y are the horizontal and vertical coordinates of the pixel point, B and B masked are the superimposed photon number sequences before and after the mask operation, j is the index in the time axis direction, and BlindBin is the length of the mask.

[0019] Preferably, before performing the mask operation, the following steps are further performed:

[0020] performing a pre-estimation of depth on the original echo data based on the number of photons of N laser pulses, where N is less than or equal to M;

[0021] determining the length of the BlindBin based on the result of the pre-estimation of depth.

[0022] Further, the matched filtering is specifically:

[0023] B MF (x, y, j) = B(x, y, j) * h(j),

[0024] where h(j) is a pulse response determined based on the laser pulse width and waveform, B MF is a sequence of the number of superimposed photons after matched filtering.

[0025] Further, the collaborative variational imaging optimization strategy includes Wiener-Hopf adaptive filtering operation, total variation deconvolution operation, and morphological reconstruction operation of global feature and local feature fusion on the depth map and / or reflectivity map.

[0026] Further, the Wiener-Hopf adaptive filtering operation is performed by the following formula:

[0027]

[0028] where D noisy , R noisy are the depth map and reflectivity map before adaptive filtering, respectively, D Wiener-Hopf , R Wiener-Hopf are the depth map and reflectivity map after adaptive filtering, respectively, H D , H R are the impulse response functions of the Wiener-Hopf filter corresponding to the depth map and reflectivity map, respectively, k, l are the neighborhood pixel points of pixel point x, y, are the noise powers of the depth map and reflectivity map, respectively, are the local variances of the depth map and reflectivity map at pixel point x, y.

[0029] Further, the total variation deconvolution operation is performed by the following formula:

[0030]

[0031] where I Wiener-Hopf is the depth map D Wiener-Hopfor reflectance map R Wiener-Hopf ,

[0032] I deconvTV is the depth map or reflectance map after total variation deconvolution operation, I is the image variable to be optimized in the iterative optimization process, λ is the regularization parameter of the data fidelity term, τ is the regularization parameter of the total variation term, is the image gradient operator, ||·||2 is the L2 norm, ||·||1 is the L1 norm, PSF is the point spread function, and σ is the standard deviation of the Gaussian beam.

[0033] Further, the morphological reconstruction operation includes sequentially executed edge detection and morphological closing operation, distance transformation and morphological reconstruction, and connected region processing and image enhancement.

[0034] Further, the edge detection and morphological closing operation is performed by the following formula:

[0035] E closed = close(C(I deconvTV ), S(N)),

[0036] wherein C(I deconvTV ) is to extract edges of the image I deconvTV , close represents morphological closing operation, S(N) is to define the neighborhood of pixels in the image, E closed is the edge map obtained by edge detection and morphological closing operation.

[0037] Further, the distance transformation and morphological reconstruction is performed by the following formula:

[0038]

[0039] wherein W(~Eclosed) is the distance transformation of the complement set of E closed , max(·) is the normalization operation, Reconstruct() is the morphological reconstruction function, I Reconstructed is the image after morphological reconstruction.

[0040] Further, the connected region processing and image enhancement is performed by the following formula:

[0041]

[0042] wherein LabelImg==p represents the connected region labeled as p, mode(·) is to count all pixel values in the region, ∪ is all connected regions, num is the number of connected regions, is the image sharpening processing, I final is the depth map or reflectance map generated by final optimization.

[0043] Preferably, the image sharpening processing is performed using a trained Real-ESRGAN model.

[0044] The underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization provided by the embodiments of the present application fully utilizes the difference between the time distribution statistical characteristics of noise and scene information photons in the echo signal of the single-photon lidar, separates the target signal and estimates the time of flight and reflection intensity from the single-photon sequence accumulated by multiple laser pulses, further improves the total variation smoothing constraint, adds adaptive filtering and global morphological constraint structure to deconvTV, better maintains the image edge and texture while suppressing abnormal pixels by dynamically adjusting the filtering parameters, protects the naturalness and visual details of the image, and thus high-quality reconstructed images of underwater targets can be quickly obtained. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 It is a schematic diagram of a system architecture of an underwater single-photon lidar;

[0046] Figure 2 It is a flowchart of the underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization provided by the embodiments of the present application;

[0047] Figure 3A It is a schematic diagram of the data structure of the original echo signal of the underwater single-photon lidar provided by the embodiments of the present application;

[0048] Figure 3B It is a schematic diagram of the superposition photon number sequence provided by some specific embodiments;

[0049] Figure 4 It is a schematic diagram of the superposition photon number sequence of each pixel point of the three-dimensional reconstruction provided by the embodiments of the present application;

[0050] Figure 5 It is a flowchart of the depth-reflectivity estimation operation based on modal fusion provided by the embodiments of the present application;

[0051] Figure 6 It is a depth map and a reflectivity map obtained after depth-reflectivity estimation provided by the embodiments of the present application;

[0052] Figure 7 It is a flowchart of the underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization provided by the embodiments of the present application;

[0053] Figure 8 It is a schematic diagram of the experimental pool layout provided by the specific embodiment 1 of the present application;

[0054] Figure 9 Front view of an underwater target according to specific embodiment 1 of the present application;

[0055] Figure 10 Perspective view of an underwater target according to specific embodiment 1 of the present application;

[0056] Figure 11 Schematic view of the propagation of a laser pulse in water according to specific embodiment 1 of the present application;

[0057] Figure 12A Underwater target reconstruction result using 50 laser pulses according to specific embodiment 1 of the present application;

[0058] Figure 12B Underwater target reconstruction result using 500 laser pulses according to specific embodiment 1 of the present application;

[0059] Figure 13A Underwater target reconstruction result using the peak algorithm;

[0060] Figure 13B Underwater target reconstruction result using the cross-correlation algorithm;

[0061] Figure 13C Underwater target reconstruction result using the first photon algorithm;

[0062] Figure 13D Underwater target reconstruction result using the first photon group algorithm;

[0063] Figure 14 Schematic view of an underwater target according to specific embodiment 2 of the present application;

[0064] Figure 15A Underwater target reconstruction result using 32 pixels / 10 laser pulses according to specific embodiment 2 of the present application;

[0065] Figure 15B Underwater target reconstruction result using 64 pixels / 10 laser pulses according to specific embodiment 2 of the present application;

[0066] Figure 15C Underwater target reconstruction result using 128 pixels / 20 laser pulses according to specific embodiment 2 of the present application. DETAILED DESCRIPTION

[0067] The present application is further explained in the following based on preferred embodiments and with reference to the accompanying drawings.

[0068] In the description in the embodiments of the present application, it should be noted that if the terms "upper", "lower", "inner", "outer" and the like are used to indicate the orientation or position relationship, or the orientation or position relationship of the product of the embodiments of the present application in use, it is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, in the description of the present application, in order to distinguish different units, the first, second and the like are used in the specification, but these are not limited by the order of manufacture, and cannot be understood as indicating or implying relative importance, and the name may be different in the detailed description and the claims of the present application.

[0069] The words in the specification are used to illustrate the embodiments of the present application, but are not intended to limit the present application. It should be noted that, unless otherwise specified and limited, if the terms "provided", "connected", "connected" appear, they should be understood in a broad sense, for example, they can be fixedly connected, or can be detachably connected, or integrally connected; can be mechanically connected, can be directly connected, or indirectly connected through an intermediate medium, can be the communication between two elements inside. For those skilled in the art, the specific meaning of the above terms in the present application can be specifically understood.

[0070] In order to clearly describe the technical solutions provided by the present application, first, the implementation mode of target reconstruction using underwater single-photon lidar is introduced.

[0071] <1. Introduction of scanning imaging single-photon lidar technology>

[0072] Figure 1 A system architecture diagram of an underwater single-photon lidar, the system is a scanning imaging underwater single-photon lidar, not a staring imaging mode, because for the complex and variable underwater environment, the scanning system can adjust the scanning mode and speed to adapt to different detection requirements, which is particularly important for the variable underwater environment.

[0073] As Figure 1As shown, the optical component of the system mainly consists of an FPGA (Field-Programmable Gate Array), a laser, a galvanometric scanner, a single-photon avalanche diode (SPAD), a time-correlated single-photon counter (TCSPC), and optical devices such as a beam expander, an open hole mirror, an optical filter, and a lens, all housed within a sealed cabin. The FPGA is connected to the host controller via a sealed port and provides a trigger signal to the laser. The laser pulse generated by the trigger is expanded to a suitable diameter by the beam expander and, under the control of the galvanometric scanner, emitted through a sapphire glass window into a propagation medium such as water, thereby achieving point-by-point scanning of the target.

[0074] The photon signal reflected from the target is reflected by the aperture mirror, and after being focused by the lens and filtered by the stray light of the optical filter, it enters the single-photon avalanche diode, which converts the optical signal into a detection signal in the form of an electrical signal and sends it to the time-correlated single-photon counter.

[0075] The time-correlated single-photon counter simultaneously receives the detection signal from the single-photon avalanche diode and the synchronization signal from the laser to obtain the flight time of each single photon. The flight time information is transmitted to the FPGA. The FPGA counts the number of photons received in each time period (usually expressed in picoseconds or nanoseconds) to obtain a time histogram corresponding to each scan point. The time histogram corresponding to each scan point is sent to the host as raw echo data, and then underwater target images can be reconstructed using various target reconstruction algorithms.

[0076] In addition, such as Figure 1 As shown, the system also includes modules such as a power supply and a heat exchanger to provide power and heat dissipation for the system equipment.

[0077] After obtaining the original echo data, various target reconstruction algorithms can be used to process the original echo data to reconstruct the depth map and reflectivity map of the underwater target, wherein the depth map can reflect whether there is a target in the scanning area and the distance between each position of the target and the system, and the reflectivity map reflects the reflection intensity information of each position of the target.

[0078] It should be emphasized that the number of laser pulses emitted at the same position is often closely related to the technical route of the target reconstruction algorithm, because the measurement principle of single-photon lidar determines that the depth information of a position of the target is represented by the statistical characteristics of the photons returned by multiple laser pulses, and therefore the number of laser pulses emitted at the same position required to meet the accuracy requirement may vary from several times to several hundred times or even several thousand times for different algorithms, which will significantly affect the amount of data to be processed and the speed of target reconstruction.

[0079] In addition, when using some algorithms for scanning and target reconstruction, the number of laser pulses emitted at different positions may not be the same. For example, Chinese invention patent CN112305560A discloses a single-photon lidar fast imaging method based on a first photon group, the technical route of which is to continuously emit laser pulses at the same position, and find a region that can satisfy the existence of N photon counts within an epsilon time neighborhood from the collected photon count results after each emission. If it cannot be found, continue to emit laser pulses at the position until the first time the region is found (referred to as the first photon group), then record the corresponding emission pulse number Num and the counting time {t1, t2,..., t N} of the N photons; according to the characteristic that the Gaussian function can approximate the time waveform of the laser pulse echo, take the mean value of {t1, t2,..., t N} as the estimated value of the flight time of the pulse signal emitted at the position, and then obtain the estimated value of the depth corresponding to the position; according to the properties of the Poisson random process, take -ln(1-N / Num) as the estimated value of the reflection intensity information of the underwater target at the position.

[0080] The above method has the following problems in the process of detecting and processing data of the underwater target:

[0081] 1) The method needs to determine whether the first photon group is received according to the N-ε condition (also known as the μ-ε condition) of the first photon group screening, so it completely determines whether the target echo appears according to the statistical characteristics of the collected photon count, and in order to screen the first photon group at a certain position, the number of laser pulses emitted at the position is not fixed, which not only leads to inconsistent time for single-photon lidar to scan different positions of underwater targets, but also in some harsh underwater environments, the number of pulses required may exceed the allowable time for actual measurement, undoubtedly greatly prolonging the data processing time, resulting in the inability to meet the need for fast imaging of underwater moving targets;

[0082] 2) The N-ε condition (also known as the μ-ε condition) for screening the first photon group is reasonably set, which will have a fundamental impact on the estimation result: the larger N is obtained and the smaller ε is obtained, which indicates that the definition of the first signal photon group is very tight, and a large number of very concentrated photon counts must appear to be considered as the first signal photon group. Such a result has higher robustness to noise (because the probability of noise satisfying the N-ε condition is smaller), but it obviously requires more accumulation time to obtain the first signal photon group satisfying the N-ε condition, which undoubtedly will lead to an increase in the number of laser pulses Num for screening the first photon group, resulting in a decrease in the efficiency of the entire method. On the contrary, if N is increased and ε is reduced, the probability of noise satisfying N-ε condition and being identified as the first photon group will undoubtedly increase.

[0083] Obviously, each time the underwater target is measured, the values of N and ε need to be reset based on prior knowledge according to the actual underwater environment, laser characteristics and underwater target reflection characteristics, and then the measurement results are repeatedly optimized, otherwise the problems of excessive measurement time or high false recognition rate are prone to occur; at the same time, the method completely estimates the echo flight time and reflection intensity of the underwater target based on the probability model, and the physical image is not clear, especially when the first photon group is screened incorrectly, the estimation of the echo flight time and reflection intensity of the target will completely lose its meaning.

[0084] It can be seen that the optimization of the target reconstruction algorithm and the improvement of the hardware system complement each other and jointly determine the speed and accuracy of target reconstruction. Therefore, the existing target reconstruction algorithm based on underwater single-photon lidar is improved, and an underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization is proposed. The method can effectively utilize the basic characteristics of the different distributions of noise and scene information photons in the time histogram to decompose the pulse cumulative single-photon original signal, and strengthen the target depth and reflectivity image with the help of the optimization strategy, and at the same time, manual parameter adjustment is not required during each measurement to ensure that effective photon flight time information can be captured, so as to meet the needs of fast and high-precision target reconstruction of moving targets in various underwater environments.

[0085] <II. Method framework of the present application>

[0086] Figure 2 A flowchart of the underwater single-photon lidar target reconstruction method based on modal fusion and collaborative variational imaging optimization provided by the present application is shown, as shown in the figure, the method comprises the following steps:

[0087] Step 100, reconstructing the original echo data of the underwater single-photon lidar into a superimposed photon number sequence corresponding to X×Y pixel points;

[0088] Step 200, performing depth-reflectivity estimation based on modal fusion on the superimposed photon number sequence of each pixel point to obtain a depth map and a reflectivity map of the underwater target;

[0089] Step 300, optimizing the depth map and the reflectivity map of the underwater target based on a collaborative variational imaging optimization strategy.

[0090] Among them, step 100 is used to generate a superimposed photon number sequence corresponding to each pixel point on a two-dimensional plane; step 200 estimates the depth and reflectivity by comprehensively considering the statistical characteristics and peak characteristics of the superimposed photon number sequence of each pixel point to preliminarily generate a two-dimensional depth map and reflectivity map, and step 300 improves the accuracy and stability of the underwater target image reconstruction by collaboratively optimizing the total variation deconvolution algorithm and morphological reconstruction of the present image.

[0091] The above steps are described in detail below in combination with the drawings and preferred embodiments.

[0092] <III. Reconstruction of original echo data>

[0093] Figure 3A The data structure of the original echo obtained by scanning the underwater target by the underwater single-photon lidar system in some embodiments is shown as follows Figure 3A When scanning an area with X pixels in the horizontal direction and Y pixels in the vertical direction, X×Y rows of echo data can be obtained, where the element value of any row corresponds to the number of photons received at each time bin when a laser pulse is emitted multiple times at the position of a certain pixel.

[0094] Without loss of generality, for a pixel point x,y,(x∈[1,X]&y∈[1,Y]), the laser pulse emitted each time is intercepted for the first Z time bins, and the time bin sequence corresponding to each laser pulse can be obtained Correspondingly, the number of photons received by the point when each laser pulse is irradiated can be represented by the following M photon number sequences with a length of Z:

[0095]

[0096] …,

[0097]

[0098] The M sequences are sequentially arranged head to tail to form echo data of a pixel point. The echo data of each pixel point is arranged in rows to obtain the raw echo data as shown in FIG. 2. Figure 3A

[0099] Due to the imaging principle of the single-photon lidar, the number sequence of received photons obtained by a single laser pulse cannot reflect the characteristics of the target. In particular, due to optical attenuation and other reasons, some laser pulses may have no photons returned at all. Therefore, only through multiple laser pulse measurements can a statistical distribution of photon arrival time with actual physical meaning be generated.

[0100] In the embodiments of the present application, after the echo data is obtained, the echo data of each pixel point is subjected to photon number superposition based on a time bin grid to obtain a superimposed photon number sequence corresponding to each pixel point.

[0101] Specifically, for any pixel x, y, the elements corresponding to the same time bin grid in the M photon number sequences are sequentially superimposed, for example:

[0102] For the first time bin grid Let

[0103] For the second time bin grid Let

[0104] By analogy, the superimposed photon number sequence corresponding to each pixel point (x, y) can be obtained:

[0105]

[0106] Figure 3B The superimposed photon number sequences at two pixel points are exemplarily shown. After obtaining the superimposed photon number sequence corresponding to each pixel point (x, y), it can be further reconstructed according to a data structure of X horizontal pixels and Y vertical pixels to finally obtain an X × Y × Z-dimensional array, as shown in FIG. 3. Figure 4

[0107] ​​It should be noted that in the embodiments of the present application, the number M of laser pulses used for photon number stacking of each pixel point is the same, and using the same number of laser pulses for photon number stacking can effectively avoid the problem of uncertain data processing time caused by the way of using N-ε criterion to screen in random pulse measurement results in the existing first photon group method, and is more suitable for the requirement of fast imaging of underwater slow-moving targets. In addition, as analyzed in the foregoing, since the single-photon lidar collects and counts the energy of as low as a single photon, the photon counting result of a single pulse generally cannot present a clear difference between the distribution of the echo signal and the distribution of the noise signal, and even the phenomenon that the noise signal in some time bin grid may have a higher count result than the real echo signal. However, with the accumulation of multiple pulse measurement results, the probability distribution characteristics (such as Gaussian distribution) of the echo signal will be continuously strengthened through stacking, and the noise signals randomly distributed in each measurement will show a probability distribution characteristic that is more consistent with random distribution as the number of pulses is stacked, and finally the phenomenon that the signal-noise ratio of the photon number sequence obtained after stacking is improved, that is, the photon counting is concentrated in a certain region. At this time, only the photon counting concentrated region needs to be searched and further processed to obtain the reflection intensity and time information of the echo signal, thereby effectively avoiding the problems of reduced efficiency or increased misidentification rate when N and ε are not properly set in the first photon method.

[0108] <Four, depth-reflection rate estimation based on modal fusion>

[0109] After obtaining the stacked photon number sequence of each pixel point through step 100, the depth and reflectivity of the position of each pixel point can be estimated through step 200. Figure 5 The flow of depth and reflectivity estimation for any pixel point x, y in one specific embodiment is shown.

[0110] As shown in Figure 5 , first, the corresponding stacked photon number sequence of the pixel point is read in step 221 Then, through steps 222 to 224, a window with a length of W bin grids is set, and the window is continuously slid on the stacked photon number sequence from the starting position with a step of 1 bin grid, and W elements are intercepted at each sliding position to obtain multiple candidate groups Where i represents the position of each sliding.

[0111] Obviously, when i = 1,

[0112] Then, the photon numbers in each of the alternative groups are summed in step 226, and the alternative group with the largest sum is determined as the group where the target signal is located And the time bin grid information corresponding to each element in the group is extracted through step 227

[0113] As analyzed above, after the multiple laser pulse measurement results are superimposed, the signal-to-noise ratio of the superimposed photon number sequence has been significantly improved. Obviously, the region with the most concentrated photon count distribution corresponding to the sliding window retrieval result has a great probability of being the region corresponding to the real echo signal of the underwater target at that position. At the same time, the sum of the photon numbers in the group also has a great probability of reflecting the reflection intensity of the underwater target at the position corresponding to the pixel point to the laser pulse. Therefore, the depth and reflectivity of the position can be quickly estimated using the retrieval result.

[0114] Specifically, in step 228, the time of flight of the echo signal can be obtained by retrieving the peak of the photon number in the group where the target signal is located, that is, selecting the τth element with the largest photon number from the group where the target signal is located, and determining the time bin grid determined as the time of flight T x,y of the photons reflected by the target. By performing peak value judgment in the group where the target signal is located obtained through sliding statistics, the influence of the existence of part of the noise extreme value on the time of flight judgment when the peak value method is directly used can be effectively avoided, and the accuracy of the depth estimation is effectively improved.

[0115] At the same time, the sum of the photon numbers in the group where the target signal is located also has a great probability of reflecting the reflection intensity of the underwater target at the position corresponding to the pixel point to the laser pulse. Therefore, in step 229, the values of all elements in the group where the target signal is located are superimposed to obtain N x,y This way of superimposing multiple pulse measurement results and then superimposing the photon count values in the region with the most concentrated photon count distribution to obtain the reflectivity has a more clear physical meaning compared to the way of determining the reflection intensity according to the ratio of a specified photon number N to a random experimental number Num based on the first photon group.

[0116] The above T x,y , N x,y respectively represent the depth and reflectivity of the position where the pixel points x, y are located. For example, in step 230, the time of flight can be further converted into the distance D x,y of the underwater target at the position where x, y is located from the single-photon lidar through the following formula:

[0117] D x,y = c x T x,y / 2,

[0118] where c represents the speed of light in water.

[0119] For each pixel, step 200 is repeated, and the depth and reflectance corresponding to each pixel are obtained, as shown in formula (1). Figure 6 As shown in formula (2), after arranging the pixels according to their positions, the depth map D noisy (x,y) and the reflectance map R noisy (x,y) of the target scene with X×Y pixels are preliminarily obtained, where x∈[1,X], y∈[1,Y].

[0120] <5. Collaborative variational imaging optimization strategy>

[0121] Due to the complexity of light attenuation in the underwater environment, there are still some missing information and noise points in the preliminarily reconstructed depth map and reflectance map. Therefore, in the embodiments of the present application, step 300 is used to further optimize the image of the depth map and the reflectance map of the target scene by using a collaborative variational imaging optimization strategy. The core idea is to use the idea of total variation smoothing constraint, to better maintain the image edge and texture while suppressing abnormal pixels, and to protect the naturalness and visual details of the image through adaptive filtering operation and total variation deconvolution operation (deconvTV) under global morphological constraint.

[0122] 5-1) Wiener-Hopf adaptive filtering operation:

[0123] The depth map D noisy and the reflectance map R noisy of the target scene are obtained through step 200, and Wiener-Hopf adaptive filtering is used to adaptively adjust the filtering parameters by using the local statistical characteristics of the image. In different regions of the image, the behavior of the filter will be different, so as to remove abnormal pixel values while preserving edges and details, and to provide a clean pixel environment for the subsequent smoothing constraint operation.

[0124] Specifically, the filtering of the preliminarily obtained depth map D noisy and the reflectance map R noisy can be performed by the following formula:

[0125]

[0126] Where D Wiener-Hopf and R Wiener-Hopf are the depth map and the reflectance map after adaptive filtering, H D and H R represent the convolution kernel of the Wiener-Hopf filter, and k, l represent the neighborhood pixel points of the pixel points x, y. noise power of the depth map and the reflectivity map, respectively, local variance of the depth map and the reflectivity map at pixel point x, y, respectively.

[0127] Further, the estimated signal and the expected response can be calculated between the depth map D noisy and the reflectivity map R noisy , the cross-power spectral density between the estimated signal and the expected response, the power spectral density of the signal, and the power spectral density of the noise, so as to obtain H D , H R .

[0128] Although the Wiener-Hopf adaptive filter can effectively suppress noise, due to the fact that the single-photon lidar obtains fewer signal photon counts for underwater target detection, in some cases where light attenuation is more serious, the average number of photons per pixel in the reconstructed image of the underwater target scene is even ≯1, so that the variance of the inferred filter value is too large, which leads to a large uncertainty in image reconstruction, and this uncertainty can lead to significant reconstruction errors.

[0129] In order to reduce this uncertainty, in the embodiments of the present application, after the Wiener-Hopf filtering operation, an improved deconvTV processing is further required, the idea of which is to first constrain the depth map and the reflectivity map through deconvTV (Total Variation deconvolution) processing, to maintain edge information by minimizing the sum of absolute values of image gradients, while suppressing noise, and then to adopt a morphological reconstruction operation that fuses global features and local features, to highlight edges and textures in the image to enhance details, so as to improve the accuracy of image reconstruction and enhance the visual effect.

[0130] 5-2) Total Variation deconvolution processing:

[0131] deconvTV (Total Variation deconvolution) is an image deconvolution algorithm based on total variation regularization, mainly used in the fields of image processing and restoration, which combines deconvolution and total variation regularization processing, uses a TV regularization term to maintain image edges while suppressing noise, and solves the optimal solution by minimizing the objective function.

[0132] Specifically, in the embodiments of the present application, the total variation deconvolution processing on the depth map or the reflectivity map after the Wiener-Hopf adaptive filtering can be represented as follows:

[0133] deconvTV(I Wiener-Hopf , PSF, Opts) (4),

[0134] where I Wiener-Hopf represents the D Wiener-Hopf or R Wiener-Hopf , PSF denotes the point spread function, and Opts denotes the parameters needed to be set for performing the deconvTV processing, including the regularization parameter μ, the initial penalty parameter rho_r, the maximum iteration number max_itr, the regularization parameter beta of the weight TV norm, the update constant gamma, the tolerance tol, etc.

[0135] When the underwater single-photon lidar uses a Gaussian beam, the value of the point spread function at x, y can be expressed as follows:

[0136]

[0137] where σ is the standard deviation of the Gaussian beam.

[0138] The specific calculation of the deconvTV processing can be performed by the following formula:

[0139]

[0140] where I deconvTV is the depth map or reflectivity map after the total variation deconvolution processing, I is the image variable to be optimized in the iterative optimization process, λ is the regularization parameter of the data fidelity term, τ is the regularization parameter of the total variation term, is the image gradient operator, ||·||2 is the L2 norm, and ||·||1 is the L1 norm.

[0141] 5-3) Morphological reconstruction operation of global feature and local feature fusion:

[0142] The purpose of the morphological reconstruction operation is to further improve the image quality by using corrosion, expansion, open-close operation and other operations to assist the deconvTV processing. Specifically, in some preferred embodiments, the morphological reconstruction operation includes sequentially performing edge detection and morphological close operation, distance transformation and morphological reconstruction, connected region processing and image enhancement.

[0143] Specifically, the edge detection and morphological close operation are performed by the following formula:

[0144] E closed = close(C(I deconvTV ), S(N)) (7),

[0145] where C(I deconvTV ) represents extracting edges from the image I deconvTV , close represents the morphological close operation, S(N) is a neighborhood that defines pixels in the image, and Eclosed the edge map obtained by edge detection and morphological closing operation.

[0146] Specifically, the distance transform and morphological reconstruction are performed by the following formula:

[0147]

[0148] wherein W(~Eclosed) is the distance transform of the complement set of E closed , max(·) is a normalization operation, Reconstruct() is a morphological reconstruction function, I Reconstructed is the image after morphological reconstruction.

[0149] Connected region processing and image enhancement are used to process the connected regions in the image to enhance the visual contrast effect, specifically, by the following formula:

[0150]

[0151] wherein LabelImg==p represents the connected region labeled as p, mode(·) represents the statistics of all pixel values in the region, ∪ represents all connected regions, num represents the number of connected regions, represents image sharpening processing, I final is the final optimized depth map or reflectivity map.

[0152] In some preferred embodiments, The operation can be performed using a pre-trained Real-ESRGAN model to further improve the resolution and visual effect of the image.

[0153] <Six, signal-to-noise separation operation based on statistical characteristics of photon number distribution>

[0154] With the underwater single-photon lidar being applied in more and more fields such as ocean exploration, underwater archaeology, environmental monitoring, underwater navigation and the like, various harsh imaging conditions and imaging indexes faced by the underwater single-photon lidar put forward higher requirements for the technical scheme of target reconstruction. For example, although the statistical characteristics of echo signal photons can be strengthened by increasing the number of laser pulses emitted at the same position, thereby shortening the length of the sliding window and improving the estimation accuracy, for a target moving slowly underwater (e.g., a target moving underwater at a speed of no more than 2 m / s), too many laser pulse numbers will increase the measurement and data processing time, resulting in that the target cannot be tracked in time, and too few laser pulse numbers will inevitably suppress the noise photons in the system. For another example, in the environment with high light attenuation or high ambient light intensity, the statistical characteristics of noise photons and signal photons in the echo signal will be difficult to separate, and only increasing the length of the sliding window or the number of laser pulses cannot meet the requirements of processing speed and accuracy at the same time.

[0155] It can be seen that, in order to improve the reconstruction accuracy and speed of the underwater moving target at the same time, the time and space distribution characteristics of noise photons and signal photons need to be further utilized for signal-noise separation operation on the basis of comprehensively considering the number of pulses and the length of the window, so as to meet the increasingly challenging underwater detection and imaging requirements.

[0156] 6-1) Optimization of the number of laser pulses and the length of the sliding window

[0157] When high-resolution imaging of the underwater target is required, the number of pixels will increase in a square relationship with the image size. If further high-resolution reconstruction of the moving target is required, the number of laser pulse scans M at the position of each pixel point needs to be reduced as much as possible on the basis of ensuring the imaging effect. It can be known through testing by using the method provided in the present application that, for each pixel point, only 5 groups of laser pulse measurement data are required to make the statistical characteristics of the echo signal and the noise signal in the superimposed photon number sequence appear obvious differences, that is, the lower limit of the number M of laser pulses for superimposing the photon number at the position of each pixel is 5 times.

[0158] At the same time, with the decrease of the number of laser pulses M, the selection of the length of the sliding window W needs to take into account the estimation accuracy and the processing speed. If W is too small, it may lead to misidentification of some noise maximum value positions under the condition of low pulse number, and if W is too large, it will introduce too much noise in the vicinity of the effective signal in the reflectivity calculation. In some preferred embodiments, in order to take into account the speed and imaging accuracy of target reconstruction at the same time, when performing depth-reflectivity estimation based on modal fusion on the superimposed photon number sequence of each pixel point, the preset length of the sliding window is in the range of 5 time bin grids to 9 time bin grids, that is, 5≤W≤9.

[0159] 6-2) Filtering of system noise:

[0160] When the underwater target is in a high-attenuation underwater environment, the signal in the echo data will be extremely weak, and the noise existing in the system will significantly interfere with the extraction of target information. Therefore, in some preferred embodiments, a mask operation is used to filter the system noise photons accumulated in the front part of the echo signal.

[0161] In some specific embodiments, for the superimposed photon number sequence of each pixel point x, y reconstructed into an XxYxZ three-dimensional data structure, a mask operation can be performed using the following formula to filter the system noise:

[0162]

[0163] where B and B masked are the superimposed photon number sequences of each pixel point before and after the mask operation, j is the index in the time axis direction, BlindBin is the length of the mask, and the above formula indicates that the first BlindBin elements in the superimposed photon number sequence of each pixel point are set to zero to filter out the system noise before the signal arrives.

[0164] The length of BlindBin will have a significant impact on the extraction and identification of the signal. For example, if it is set arbitrarily and too large, it will cause the system to have a large blind area, and if it is set too small, the mask operation will lose its significance, and the noise generated by the system will still affect the determination of the time index and the number of photons.

[0165] In order to reasonably set the length of BlindBin, in some preferred embodiments, as shown in Figure 7 a pre-estimation step 101' and a step 102' can be further added before step 100, wherein the original echo data is pre-processed by the photon number superposition operation based on N laser pulses (where N is not greater than M) through step 101', and then the depth of each pixel point is pre-estimated using the aforementioned depth-reflectivity estimation process based on modal fusion (i.e., the same steps as step 200, but only estimating the depth); after obtaining the above pre-estimation result, the depth of each pixel can be roughly obtained in step 102', and the length of BlindBin is set based on this, and then step 100 is re-executed to perform the photon number superposition operation on M laser pulses, and the mask operation is performed on the superimposed photon number sequence through step 103'.

[0166] In addition, it should be noted that the mask operation shown in formula (10) can be used to filter the system noise of the superimposed photon number sequence, or the photon number sequence corresponding to each laser pulse before the photon number superposition, the difference is only the order of the photon number superposition operation, and the noise filtering effect will not be affected.

[0167] 6-3) Filtering of various noise photons mixed with signal photons:

[0168] Although the mask operation can filter out the system noise photons in the front part of the time axis, it cannot be applied to filter out the various noise photons mixed with the echo signal photons after the BlindBin, which includes both system noise photons and noise photons generated by ambient light. In some preferred embodiments of the present application, the noise photons are filtered by matched filtering.

[0169] Suppose the input signal is the superposition of the desired signal (target signal) and noise, where s expect (t) is the desired signal, s noise (t) is noise, then the received echo signal s input (t) can be represented by the following formula.

[0170] s input (t) = s expect (t) + s noise (t) (11),

[0171] Obviously, for the echo signal reflected by the target, the waveform has a strong correlation with the waveform of the laser pulse emitted by the laser radar, so the form of the impulse response h(t) can be determined based on the pulse width and waveform of the laser pulse, and then the output signal y(t) is obtained by convolving the input signal as follows:

[0172] y(t) = s input (t) * h(t) = (s input (t) + s noise (t)) * h(t) (12)

[0173] Since the desired signal s expect (t) part in the input signal is completely matched with the impulse response of the filter, the amplitude of the output signal after convolution is maximum, and the convolution of the noise s noise (t) part and h(t) will not produce the same correlation peak as the desired signal s expect (t), so the influence of the noise in the output signal is suppressed. It can be seen that using the above matched filtering can effectively filter out various noise generated from the original signal, and is more conducive to the detection of the target signal.

[0174] Considering that the superimposed photon number sequence of each pixel point is a discrete signal, it can be expressed as follows:

[0175] B MF (x,y,j)=B(x,y,j)*h(j) (13),

[0176] where h(j) is a discrete form of pulse response determined based on the pulse width and waveform of the laser pulse emitted by the single-photon lidar, B MF is the superimposed photon number sequence after matched filtering.

[0177] Optionally, the mask operation of formula (10) and the matched filtering of formula (13) can be implemented separately in real time, or jointly implemented to better achieve signal-to-noise separation.

[0178] In addition, in some preferred embodiments, considering that in the depth-reflectivity estimation based on modal fusion, a sliding window with a width of W is used for retrieval of the target signal, therefore, a window with a length of W can also be used to truncate h(j) to improve processing speed while ensuring the effect of matched filtering.

[0179] <Specific embodiment 1>

[0180] According to the scanning imaging single-photon lidar shown in Figure 1 , a target arranged in an experimental pool is reconstructed using the method provided in the present application, wherein the laser is a 532nm wavelength solid-state laser, the laser pulse emission frequency is 1-5kHz, the average power is 175mW, the single pulse energy is 35μJ, and the pulse width is 600ps. The scanning imaging module uses a two-axis galvanometer for two-dimensional beam scanning, the optical deflection angle is ±22.5°, and the optical resolution is 12μrad.

[0181] Figure 8 The experimental pool layout is shown in , which can have a size of about 12m*13m*6m, and the single-photon lidar and underwater target are respectively placed at both ends of the pool.

[0182] Figure 9 and Figure 10 , the underwater target information includes 0 / 25 / 50 / 75 / 100 / 150 / 200cm depth in total 12 planes and three different reflectivities.

[0183] By increasing the light attenuation of the experimental pool to simulate a harsh underwater environment, Figure 11 shows the propagation of the laser pulse under water, as Figure 11As shown, the visibility in the experimental pool is very poor, and the underwater visible light camera cannot capture the underwater target at a close distance.

[0184] Figure 12A For the method provided in the present application, 50 pulses are used for underwater target reconstruction for each pixel point, and the obtained depth map (left) and reflectivity map (right) are shown below. Figure 12B The reconstruction results using 500 pulses are shown below. As a comparison, Figure 13A to Figure 13D The depth map (left) and reflectivity map (right) obtained using the peak value, cross-correlation, first photon, and first photon group algorithms with 50 pulses are shown respectively.

[0185] As can be seen from the above figures, the target reconstruction method provided in the present application has good ability to repair abnormal points on the basis of strong information extraction capability compared with various existing algorithms, and the depth resolution reaches 2.5 cm.

[0186] <Embodiment 2>

[0187] In this embodiment, the same experimental environment and experimental equipment as in Embodiment 1 are used, and the underwater target with a hollow structure as shown in Figure 14 is used to test the lateral resolution of the method provided in the present application. The minimum lateral resolution of the underwater target is 5 mm.

[0188] Figure 15A , Figure 15B and Figure 15C are the reconstruction depth map (left) and reflectivity map (right) corresponding to 32 pixels / 10 laser pulses, 64 pixels / 10 laser pulses, and 128 pixels / 20 laser pulses respectively. As can be seen from the figures, in the imaging result of 32 pixels and 10 pulses, the overall shape of the underwater target can be clearly seen and distinguished, and in the imaging result of 128 pixels and 20 pulses, the detailed information of the hollow underwater target is completely revealed, and the minimum resolution is about 5 mm. Thus, it is effectively proved that the method used in the present application can realize fast and accurate reconstruction of underwater targets in a relatively poor underwater environment.

[0189] The specific embodiments of the present application are described in detail above, and those skilled in the art can make some improvements and modifications to the present application without departing from the principles of the present application, and these improvements and modifications also belong to the protection scope of the claims of the present application.

Claims

1. A target reconstruction method for underwater single-photon lidar based on modal fusion and cooperative variational imaging optimization, characterized in that, Includes the following steps: The raw echo data of the underwater single-photon lidar is reconstructed into a superimposed photon number sequence corresponding to X×Y pixels. The superimposed photon number sequence corresponding to each pixel is generated by superimposing the photon number of multiple laser pulse echo signals received at its location based on time bins. The number M of laser pulses used for photon number superposition of each pixel is the same. Depth-reflectivity estimation based on modal fusion is performed on the superimposed photon number sequence of each pixel to obtain the depth map and reflectivity map of the underwater target; The depth map and reflectance map of underwater targets are optimized based on a collaborative variational imaging optimization strategy; Depth-reflectivity estimation based on modal fusion for the superimposed photon number sequence of any pixel includes the following steps: A sliding window of preset length is used to slide in steps over the superimposed photon number sequence of the pixel, with a time bin increment, to capture multiple candidate groups. Select the candidate group with the highest number of photons within the group as the target signal group; The underwater target depth corresponding to the pixel is determined based on the time bin number corresponding to the maximum value element in the group containing the target signal, and... The underwater target reflectivity corresponding to the pixel is determined based on the sum of the number of photons in the group to which the target signal belongs. The collaborative variational imaging optimization strategy includes performing Wiener-Hopf adaptive filtering, total variational deconvolution, and morphological reconstruction by fusing global and local features on the depth map and / or reflectance map.

2. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 1, characterized in that, The underwater target is a slowly moving target, and, The lower limit of M is 5 times and / or the length of the sliding window is between 5 and 9 time bins.

3. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 1, characterized in that, It also includes performing a signal-to-noise separation operation on the superimposed photon number sequence of each pixel based on the statistical characteristics of photon number distribution. The signal-to-noise separation operation includes at least one of masking operation and matched filtering.

4. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 3, characterized in that, The mask operation specifically involves: Where x, y are the x-coordinate and y-coordinate of the pixel, and B, B masked These are the superimposed photon number sequences before and after the masking operation, respectively, where j is the index along the time axis and BlindBin is the length of the mask.

5. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 4, characterized in that, Before performing the masking operation, the following steps are also performed: The original echo data is subjected to a photon number superposition operation based on N laser pulses and a depth pre-estimation, where N is less than or equal to M; The length of the BlindBin is determined based on the pre-estimated depth.

6. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 4, characterized in that, The matched filtering specifically refers to: B MF (x,y,j)=B(x,y,j)*h(j), Where h(j) is the pulse response determined based on the laser pulse width and waveform, B MF This is the superimposed photon number sequence after matched filtering.

7. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 1, characterized in that, The Wiener-Hopf adaptive filtering operation is performed using the following formula: Among them, D noisy R noisy These are the depth map and reflectance map before adaptive filtering, respectively. Wiener-Hopf R Wiener-Hopf These are the adaptively filtered depth map and reflectance map, H. D H R , where k and l are the impulse response functions of the Wiener-Hopf filters corresponding to the depth map and reflectance map, respectively, and k and l are the neighboring pixels of pixel x and y. These are the noise powers of the depth map and reflectance map, respectively. These are the local variances of the depth map and the reflection intensity map at pixel x and y, respectively.

8. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 1, characterized in that, The total variational deconvolution operation is performed using the following formula: Among them, I Wiener-Hopf The depth map D before performing the total variation deconvolution operation Wiener-Hopf or reflectance map R Wiener-Hopf I deconvTV Let I be the depth map or reflectance map after total variation deconvolution, λ be the image variable to be optimized in the iterative optimization process, τ be the regularization parameter of the data fidelity term, and τ be the regularization parameter of the total variation term. Let ||·||2 be the image gradient operator, ||·||1 be the L2 norm, PSF be the point spread function, and σ be the standard deviation of the Gaussian beam.

9. The underwater single-photon lidar target reconstruction method based on modal fusion and cooperative variational imaging optimization according to claim 1, characterized in that, The morphological reconstruction operations include edge detection and morphological closing operations, distance transformation and morphological reconstruction, and connected component processing and image enhancement, performed sequentially. The edge detection and morphological closing operation are performed using the following formula: E closed =close(C(I deconvTV ),S(N)), Among them, C(I) deconvTV For image I deconvTV Edge extraction, `close` represents the morphological closing operation, S(N) defines the neighborhood of a pixel in the image, and E closed The edge map is obtained by edge detection and morphological closing operation; The distance transformation and morphological reconstruction are performed using the following formula: Where W(~Eclosed) is E closed The distance transformation of the complement, max(·) is the normalization operation, Reconstruct() is the morphological reconstruction function, I Reconstructed The image is a morphologically reconstructed image; The connected component processing and image enhancement are performed using the following formula: Where LabelImg==p represents the connected region labeled p, mode(·) represents all pixel values ​​within the statistical region, ∪ represents all connected regions, and num represents the number of connected regions. For image sharpening processing, I final This is used to generate the final optimized depth map or reflectance map.

Images