A single-photon lidar method for three-dimensional imaging through fog in cooperation with photon processing

Abstract

The application discloses a single-photon laser radar fog-penetration three-dimensional imaging method based on synergistic photon processing, and relates to the field of single-photon laser radar three-dimensional imaging. The method solves the problem of how to reconstruct a high-quality target three-dimensional image by using a small amount of statistical frames under the interference of smoke and strong noise. The method comprises the following steps: through pixel-by-pixel signal accumulation compensation and a global noise suppression method, data preprocessing is completed, and a multi-scale superpixel three-dimensional imaging algorithm is used to generate a guide image; the denoised data is subjected to block processing, a depth image guided photon processing algorithm and a pure noise band noise isolation method of multi-dimensional Fourier transform are used, and accurate signal extraction is realized through synergistic photon processing in time, space and frequency; the denoised data is subjected to multi-scale block processing, reconstructed depth and intensity images under different scales are obtained, the obtained images are fused, and a three-dimensional image reconstruction result of smoke interference suppression is output. The method is suitable for the field of single-photon laser radar three-dimensional imaging under environmental conditions such as cloud and fog.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of single-photon laser radar three-dimensional imaging, and particularly relates to a single-photon laser radar fog-penetrating three-dimensional imaging method based on cooperative photon processing. BACKGROUND

[0002] A single-photon laser fog-penetrating method based on a double-quantity estimation method is disclosed in Chinese patent document No. CN113406594A, which estimates echo signals at the signal level, mainly estimates two parameters of a smoke distribution model (Gamma distribution), completes smoke signal estimation, and finally realizes smoke-penetrating three-dimensional imaging of a single-photon laser radar. This technology only estimates echo data from the signal level, and requires a large number of data statistics, close to 20,000 frames of data to complete signal estimation.

[0003] A single-photon laser radar fog-penetrating imaging method based on double Gamma estimation is disclosed in Chinese patent document No. CN115097484A, which first performs a first Gamma estimation for the accumulation effect caused by strong noise interference, completes accumulation compensation and signal correction, and then uses a Gamma estimation method to estimate smoke noise, and finally completes noise suppression and three-dimensional imaging. This technology first estimates echo data from the signal level, and then combines image space information to suppress noise, compensate and optimize the image. However, this technology still requires a large number of data statistics, and requires 20,000 frames of data to complete three-dimensional imaging of a 1.4 km target at a 1.7 km visibility.

[0004] In summary, although multiple frame statistics data can effectively suppress the interference of increased attenuation length on single-photon laser radar three-dimensional reconstruction, multiple frame statistics data have the problems of large memory requirement, long data acquisition time, and unsuitability for dynamic target perception, while few frame statistics data and strong backscattering interference of smoke result in extreme scarcity of signal photons.

[0005] Therefore, under the interference of smoke and strong noise, how to use a small number of statistical frames to reconstruct high-quality target three-dimensional images is still an important scientific problem faced by current single-photon laser radars. SUMMARY

[0006] The present application solves the problem in the prior art that under the interference of smoke and strong noise, how to use a small number of statistical frames to reconstruct high-quality target three-dimensional images is still an important scientific problem faced by current single-photon laser radars.

[0007] To solve the above technical problems, the present application proposes a smoke noise suppression three-dimensional imaging method capable of accurately estimating the position of a target and significantly improving the integrity of the target when there are few statistical frames and extreme scarcity of signal photons, and is realized through the following technical solutions:

[0008] Scheme one, the present application proposes a kind of single-photon laser radar fog penetration three-dimensional imaging method of cooperative photon processing, the single-photon laser radar fog penetration three-dimensional imaging method includes the following steps:

[0009] Step 1, by the signal accumulation compensation of pixel by pixel and global noise suppression method, the pre-processing of array GM-APD laser radar data is completed, and multi-scale superpixel three-dimensional imaging algorithm is used to generate guide image;

[0010] Step 2, the array GM-APD laser radar data pre-processed in step 1 is carried out block processing, utilizes depth image guided photon processing algorithm and pure noise band noise isolation method based on multi-dimensional Fourier transform, accurate extraction of signal is realized through space-time frequency cooperative photon processing;

[0011] Step 3, the data output in step 2 is carried out multi-scale block processing, and the reconstructed depth image and intensity image under different scales are obtained, the obtained image is fused, and the three-dimensional image reconstruction result of smoke interference suppression is output.

[0012] Further, a preferred embodiment is provided, and the signal accumulation compensation method of pixel by pixel used in step 1 is as follows:

[0013]

[0014] In the formula, S is the echo photon rate function under single laser pulse condition, H is the histogram distribution of observation, N is the number of laser pulse periods, and i represents the i th time interval.

[0015] Further, a preferred embodiment is provided, and the accurate extraction method of signal of step 2 space-time frequency cooperative photon processing includes the steps of noise isolation based on 1D denoising data, noise isolation based on 3D local data, search of non-local related data and noise isolation based on 4D non-local data.

[0016] Further, a preferred embodiment is provided, and the noise isolation method based on 1D data is as follows: the pixel by pixel Fourier transform is carried out on the echo signal of global gating noise suppression, the pure noise band B n Threshold realizes noise suppression in frequency domain, and 1D denoising data is obtained by inverse Fourier transform,

[0017] Wherein,

[0018]

[0019] In the formula, τ p Is the laser pulse width.

[0020] Further, a preferred embodiment is provided, the method of noise isolation based on 3D local data is: performing local spatial block processing on the 1D denoised data, performing high-dimensional Fourier transform on the block data, adopting pure noise frequency band B n The threshold realizes the separation of noise and signal in the frequency domain.

[0021] Further, a preferred embodiment is provided, the searching of non-local related data is realized by determining the correlation between local and non-local data, that is, the similarity S d_i is calculated, and the calculation expression is as follows:

[0022] S d_i = αSSIM (P dR , P d ) + βSSIM (P iR , P i ) (6)

[0023] In the formula, α and β are weight coefficients, and satisfy the formula α + β = 1.

[0024] The calculated similarity set is sorted from large to small, and the first N sim The data cube corresponding to the first N sim block in the similarity sorting is defined as non-local related data.

[0025] Further, a preferred embodiment is provided, the method of noise isolation of 4D non-local data is: performing local spatial block processing on the 3D denoised data, and simultaneously performing high-dimensional Fourier transform in combination with the non-local related characteristics, adopting pure noise frequency band B n The threshold realizes the separation of noise and signal in the frequency domain.

[0026] Further, a preferred embodiment is provided, the method for outputting the three-dimensional image reconstruction result of smoke interference suppression by fusing the reconstructed depth image and intensity image obtained in step 3 is:

[0027] The fusion output based on the depth image and the intensity image includes three steps of intensity image fusion, depth image fusion and depth fusion image denoising;

[0028] The intensity image fusion is to superimpose the generated multiple scale intensity images at the pixel level;

[0029] The depth image fusion is to fill the generated multiple scale depth images at the pixel level with the mode to obtain the fused depth image;

[0030] The fused depth image denoising is to carry out threshold segmentation on the superimposed intensity image, and the fused depth image is denoised according to the pixel distribution of the segmentation, and through the processing of the above steps, the image fusion output result of multi-scale collaborative photon processing is obtained.

[0031] Scheme three, a computer device, comprising a memory and a processor, the memory has stored a computer program, when the processor runs the computer program stored in the memory, the processor executes the method in any one of the scheme one.

[0032] Scheme four, a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize the steps of the method in any one of the scheme one.

[0033] The beneficial effects of the present application are:

[0034] The single-photon laser radar fog-penetrating three-dimensional imaging method of the present application solves the problems of large ranging error, low recovery degree and large statistical frame number requirement of single-photon laser radar three-dimensional imaging technology in a smoke environment due to the strong scattering and high attenuation characteristics of smoke, achieves the purpose of improving the performance of single-photon laser radar three-dimensional imaging in a smoke environment, and realizes accurate estimation of the target position in a small number of statistical frames and extreme scarcity of signal photons, and significantly improves the smoke noise suppression three-dimensional imaging method of target integrity.

[0035] The method of the present application is based on the time-space-frequency multi-dimensional features of echo signals, and studies a multi-scale collaborative photon processing three-dimensional reconstruction method, which further reduces the requirement for data statistical frame number in single-image reconstruction by using echo signal time domain features, intensity image features, distance image features, data local features, data non-local features and data frequency domain features.

[0036] The present application is also applicable to the field of single-photon laser radar three-dimensional imaging in cloud, rain, fog, haze and other environmental conditions. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flowchart of the single-photon laser radar fog-penetrating three-dimensional imaging method of the present application is described in the first embodiment.

[0038] Figure 2 The data preprocessing and guide image generation flowchart of the present application is described in the first embodiment.

[0039] Figure 3 The signal extraction algorithm flowchart of the time-space-frequency collaborative photon processing of the present application is described in the first embodiment.

[0040] Figure 4A deep image guided flowchart as claimed in embodiment one.

[0041] Figure 5 A spatio-temporal-frequency collaborative photon processing algorithm flowchart as claimed in embodiment one.

[0042] Figure 6 A multi-scale collaborative photon processing image fusion output algorithm flowchart as claimed in embodiment one.

[0043] Figure 7 A different target deep image reconstruction result comparison schematic diagram as claimed in embodiment eleven. DETAILED DESCRIPTION

[0044] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application.

[0045] Embodiment one, the embodiment provides a collaborative photon processing single-photon lidar fog-penetrating three-dimensional imaging method, the single-photon lidar fog-penetrating three-dimensional imaging method comprises the following steps:

[0046] Step 1, complete the preprocessing of array GM-APD lidar data by means of pixel-by-pixel signal accumulation compensation and global noise suppression method, and generate a guide image by using a multi-scale superpixel three-dimensional imaging algorithm;

[0047] Step 2, block processing is performed on the preprocessed array GM-APD lidar data output in step 1, and precise signal extraction is realized through spatio-temporal-frequency collaborative photon processing by using a deep image guided photon processing algorithm and a pure noise band noise isolation method based on multi-dimensional Fourier transform;

[0048] Step 3, multi-scale block processing is performed on the block-processed data output in step 2 to obtain reconstructed deep images and intensity images under different scales, and the obtained images are fused to output a smoke interference suppressed three-dimensional image reconstruction result.

[0049] Embodiment two, the embodiment is a further limitation of the collaborative photon processing single-photon lidar fog-penetrating three-dimensional imaging method as claimed in embodiment one, and the pixel-by-pixel signal accumulation compensation method used in step 1 is:

[0050]

[0051] In the formula, S is the echo photon rate function under single laser pulse condition, H is the observed histogram distribution, N is the number of laser pulse periods, and i represents the i th time interval.

[0052] Embodiment three, the embodiment is further limited to the single-photon lidar fog-penetrating three-dimensional imaging method of the cooperative photon processing of embodiment one, the precise extraction method of the signal in step 2 includes the steps of noise isolation based on 1D denoising data, noise isolation based on 3D local data, searching of non-local related data and noise isolation based on 4D non-local data.

[0053] Embodiment four, the embodiment is further limited to the single-photon lidar fog-penetrating three-dimensional imaging method of the cooperative photon processing of embodiment three, the method of noise isolation based on 1D data is: performing pixel-by-pixel Fourier transform on the echo signal of global gating noise suppression, adopting pure noise frequency band B n Threshold value realizes noise suppression in frequency domain, and 1D denoising data is obtained through inverse Fourier transform,

[0054] Wherein,

[0055]

[0056] In the formula, τ p is the laser pulse width.

[0057] Embodiment five, the embodiment is further limited to the single-photon lidar fog-penetrating three-dimensional imaging method of the cooperative photon processing of embodiment three, the method of noise isolation based on 3D local data is: performing local spatial block processing on 1D denoising data, performing high-dimensional Fourier transform on the block data, and adopting pure noise frequency band B n Threshold value realizes noise and signal separation in frequency domain.

[0058] Embodiment six, the embodiment is further limited to the single-photon lidar fog-penetrating three-dimensional imaging method of the cooperative photon processing of embodiment three, the searching of non-local related data is realized by determining the correlation between local and non-local data, that is, the similarity S d_i Is calculated by using the structural similarity SSIM index, and the calculation expression is as follows:

[0059] S d_i = αSSIM (P dR , P d ) + βSSIM (P iR , P i ) (6)

[0060] In the formula, α and β are weight coefficients, and satisfy the formula α+β=1.

[0061] The calculated similarity set is sorted from large to small, and the first N simThe number of similar numbers is the number of similarities, and the similarity is ranked in the top N sim The data cube corresponding to each block is defined as non-local related data.

[0062] Embodiment seven, this embodiment is a further limitation of the single-photon lidar fog-penetrating three-dimensional imaging method of collaborative photon processing according to embodiment three, the noise isolation method of 4D non-local data is: the 3D denoising data is processed by local spatial block, and high-dimensional Fourier transform is carried out combined with non-local correlation characteristics, pure noise band B n The threshold value realizes the separation of noise and signal in the frequency domain.

[0063] Embodiment eight, the embodiment is a further limitation of the single-photon lidar fog-penetrating three-dimensional imaging method of collaborative photon processing according to embodiment seven, the method for obtaining the reconstructed depth image and intensity image under different scales in step 3, and fusing the obtained image to output the three-dimensional image reconstruction result of smoke interference suppression is:

[0064] The fusion output based on the depth image and the intensity image includes three steps of intensity image fusion, depth image fusion and depth fusion image denoising;

[0065] The intensity image fusion is to superimpose the generated multiple scale intensity images at the pixel level;

[0066] The depth image fusion is to fill the generated multiple scale depth images at the pixel level with the mode to obtain the fused depth image;

[0067] The fused depth image denoising is to perform threshold segmentation on the superimposed intensity image, and to denoise the fused depth image according to the pixel distribution of the segmentation. Through the above steps, the multi-scale collaborative photon processing image fusion output result is obtained.

[0068] Embodiment nine, the embodiment proposes a computer device, including a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes the method in any one of embodiments one to eight.

[0069] Embodiment ten, the embodiment proposes a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize the steps of the method in any one of embodiments one to eight.

[0070] Embodiment eleven, the embodiment proposes an embodiment for explaining the above embodiments one to eight, and the embodiment is specifically:

[0071] Referring toFigures 1 to 7 The method includes data preprocessing and guide image generation, signal extraction by spatiotemporal frequency collaborative photon processing, and image fusion output by multi-scale collaborative photon processing. The algorithm input end is array GM-APD laser radar data and laser radar system parameters, and the output end is a three-dimensional image reconstruction result of smoke interference suppression.

[0072] Firstly, data preprocessing is completed by pixel-by-pixel signal accumulation compensation and global noise suppression, and a guide image is generated by using a multi-scale superpixel three-dimensional imaging algorithm.

[0073] Secondly, the denoised data is processed by block processing, and the accurate extraction of signals is realized by spatiotemporal frequency collaborative photon processing through a depth image guided photon processing algorithm and a pure noise and noise isolation method based on multi-dimensional Fourier transform.

[0074] Finally, the denoised data is processed by multi-scale block processing by using the processing procedure of the second step, the reconstructed depth images and intensity images under different scales are obtained, the obtained images are fused, and the three-dimensional image reconstruction result of smoke interference suppression is output.

[0075] Step 1: Data preprocessing and guide image generation

[0076] The algorithm input data is an array single-photon laser radar data cube S, and the laser radar system parameters include the FWHM obtained by Gaussian fitting. In order to further reduce the demand for data statistical frame number in single image reconstruction, and to avoid the estimation failure of the model estimation algorithm in smoke signal estimation caused by sparse distributed echo photons, a global gating noise suppression method is used to realize the preliminary separation of signal photons and noise photons. The specific steps of data preprocessing and guide image generation are as follows:

[0077] (1) Use formula (1) for pixel-by-pixel accumulation compensation, superimpose the echo photons of all pixels in the time domain after accumulation compensation, and perform smooth fitting on the superimposed echo signal yall to obtain the curve ys, and remove the data points with burrs and mutations.

[0078]

[0079] In the formula, S is the echo photon rate function under single laser pulse condition, H is the observed histogram distribution, N is the number of laser pulse periods, and i represents the ith time interval.

[0080] (2) According to the smoke distribution scene, an applicable estimation algorithm (such as double quantity estimation method or e exponential fitting estimation) is used to estimate the noise signal and distribution interval to obtain the curve yf.

[0081] (3) Calculate the difference between the smooth curve ys and the estimation result yf to obtain the signal photon distribution curve yt, yt = max(ys-yf, 0), and perform a sliding window operation on yt with the window size of GL to search for the distribution interval of the target signal.

[0082] (4) According to the determined signal distribution interval, perform signal interception on the curve ys and calculate the mean value according to the number of pixels, intercept each pixel signal after the accumulation compensation according to the distribution interval, and calculate the difference value with the mean value result to obtain the global gating noise suppressed echo signal, which is mainly used for subsequent data processing.

[0083] (5) Use the sliding window operation shown in (2) to superimpose the denoised data in the window W to obtain the signal photon enhanced histogram set Yl, and use the log-matched filter algorithm to estimate the target distance for each histogram in the set Yl to obtain the distance image Zl at scale l.

[0084]

[0085] where O is the window size.

[0086] (6) Numerically count the distance images Zl of different scales according to the pixels, and output the fused depth image in the form of mode, which will be used for image guidance in the subsequent processing steps.

[0087] Step two: signal extraction of spatiotemporal frequency collaborative photon processing

[0088] The signal extraction algorithm of spatiotemporal frequency collaborative photon processing has four steps: noise isolation based on 1D data, noise isolation based on 3D local data, search for non-local related data, and noise isolation based on 4D non-local data.

[0089] The core idea of noise isolation processing based on 1D data is to perform pixel-by-pixel Fourier transform on the global gating noise suppressed echo signal, use the pure noise band Bn threshold to realize noise suppression in the frequency domain, and obtain 1D denoised data through inverse Fourier transform.

[0090] Since the GM-APD laser radar uses Gaussian pulse laser, the target echo signal is Gaussian-like distribution. In the case of Gaussian-shaped pulse, the pure noise band Bn is defined as the frequency band greater than three standard deviations of the Gaussian spectrum, and the expression of Bn is as follows:

[0091]

[0092] where τ p is the laser pulse width.

[0093] The core idea of the noise isolation processing based on 3D data is to perform local spatial block processing on the 1D denoised data, perform high-dimensional Fourier transform on the block data, and separate the noise and the signal in the frequency domain by using a pure noise band Bn threshold.

[0094] In order to obtain 3D data, the 1D denoised data is block processed by using windows of different sizes. According to formula (1), the local pixel data in the window W is separated by using a sliding window operation. The separated local pixel data is subjected to high-dimensional Fourier transform, and the noise component and the signal component are separated in the frequency domain by using a pure noise band Bn threshold. The Fourier coefficient energy mean E(|Bin|2) of the signal component within the pure noise band Bn and the Fourier coefficient energy mean E(|Bout|2) of the noise component outside the pure noise band Bn are calculated. At the same time, the normalized depth image used for guidance is block processed by using a sliding window of the same size, and the same Fourier amplitude spatial distribution as the 3D block data is obtained. The spatial distribution is multiplied with the separated signal component, and the product result is subjected to inverse Fourier transform to obtain the estimated echo signal.

[0095] Due to the block processing and the sliding window processing, when the echo signal of all pixels is estimated, there is a pixel overlap, that is, there is a multiple superposition problem when the echo signal of each pixel is estimated. Therefore, the multiple superimposed echo signals can be weighted and averaged to solve, and the echo signal intensity φ of a single pixel is calculated. The calculation expression is as follows:

[0096]

[0097] In the formula, n is the number of all overlapping blocks of the pixel, φi is the echo signal estimation result of the i-th overlapping block at the pixel, and ωi corresponds to the weight of the i-th overlapping block. ωi is inversely proportional to the noise component of the i-th overlapping block, and the calculation expression is as follows:

[0098]

[0099] In the formula, E(|Biout|2) is the Fourier coefficient energy mean of the noise component of the i-th overlapping block. Through the above processing, the 3D denoised data and the estimated intensity image and depth image can be obtained.

[0100] To realize the noise isolation based on 4D non-local data, firstly, the correlation between the local and non-local data cubes needs to be found. In order to find similar data cubes, the search space can be determined according to the intensity image and the depth image estimated by the 3D denoising data, and does not need to rely on the overall data cube. Taking a pixel as an example, first, a depth reference patch PdR(Lx x Ly) and an intensity reference patch PiR(Lx x Ly) are defined in the same area of the depth image and the intensity image, respectively, so that the size of the reference patch is consistent with the size of the sliding window of the 3D data acquisition, and the pixel is located at the top left corner of the reference patch (synchronized with the data cube). Then, a search window with a scale of Ssearch(Ssearch>Lx) is determined around the reference patch, and a depth image patch Pd(Lx x Ly) and an intensity image patch Pi(Lx x Ly) similar to the reference patch are searched in the search window range of the depth image and the intensity image, respectively. The structural similarity SSIM index is used to calculate the similarity Sd_i, and the calculation expression is as follows:

[0101] S d_i =αSSIM(P dR ,P d )+βSSIM(P iR ,P i ) (6)

[0102] In the formula, α and β are weight coefficients, and satisfy the formula α+β=1.

[0103] By sorting the calculated similarity set from large to small, taking the first Nsim number as the similarity number, then the data cube corresponding to the first Nsim patch in the similarity sorting can be defined as the non-local related data.

[0104] The core idea of the noise isolation processing based on 4D data is to perform local spatial patch processing on the 3D denoising data, and at the same time, to perform high-dimensional Fourier transform combined with the non-local correlation characteristics, and to realize the separation of noise and signal in the frequency domain by using the pure noise band Bn threshold.

[0105] In the guided photon processing process, the depth image estimated by the 3D denoising data is used as a guide image, and the echo signal is estimated based on the spatial frequency correlation of the guide image and the 3D denoising data. Finally, the echo signal of a single pixel is solved by using weighted average, and the estimated intensity image and depth image are output.

[0106] Step three: image fusion output of multi-scale collaborative photon processing

[0107] The 1D denoising data is processed by using different block scales to realize local and non-local correlation, and the image fusion output of multi-scale cooperative photon processing is output to realize high-quality three-dimensional imaging of the target in the smoke environment.

[0108] The fusion output based on the depth image and the intensity image includes three steps of intensity image fusion, depth image fusion and depth fusion image denoising. The intensity image fusion is to superimpose the generated multiple scale intensity images at the pixel level; the depth image fusion is to fill the generated multiple scale depth images at the pixel level with the mode to obtain a depth fusion image; and the depth fusion image denoising is to perform threshold segmentation on the superimposed intensity image, and to denoise the depth fusion image according to the pixel distribution of the segmentation. Through the above processing, the image fusion output of multi-scale cooperative photon processing is obtained.

[0109] The present application uses signal-to-background ratio (SBR) and photons per pixel (PPP) two indicators to evaluate the three-dimensional imaging ability of the current single-photon laser radar through the smoke and other atmospheric shielding objects, and the results are shown in Table 1. Compared with the research results in Table 1, the algorithm proposed in the present application reduces the PPP by 18.5% and the SBR by 98.0%.

[0110] Table 1 Comparison of experimental results of single-photon imaging through atmospheric shielding objects

[0111]

[0112] Figure 1Any processes or methods described in the flowcharts or otherwise described herein can be understood as representing code modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions (or steps) of the application, and / or that the various embodiments of the application can represent alternative process or method steps that can be implemented by the functions of the disclosed functions, and that such functions can be carried out in substantially the same way as described in the illustrative embodiments, although not necessarily implemented in the same order as illustrated. The processes, methods, or algorithms described in the flowcharts or otherwise described herein represent example architectures, functions, and operations for possible implementations of the various embodiments of the application. In this regard, each block in the flowcharts or described herein can represent a module, segment, or portion of code which comprises one or more executable instructions for implementing the specified logical functions ("application tasks"). It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It will also be noted that each block of the flowchart illustrations and / or illustrated in the block diagrams and / or described herein, and combinations of blocks in the flowcharts and / or block diagrams and / or described herein, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0113] Those skilled in the art will understand that the above description is merely illustrative of the preferred embodiments of the application and the various features and / or claims thereof can be combined or integrated in various ways, even if such combinations or integrations are not expressly disclosed in the above description. The scope of the application is therefore not limited to the specific embodiments described above, but only to the scope of the appended claims, even if further modifications or changes can be suggested by persons skilled in the art. Modifications and changes can be made in the above-described embodiments of the present application, and in the claims, without departing from the spirit and scope of the application.

[0114] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the foregoing description without departing from the spirit and scope of the application. Accordingly, it is intended that the appended claims be interpreted as including all such variations and modifications as fall within the spirit and scope of the application. It is apparent that those skilled in the art can modify and adapt the application without departing from the spirit and scope of the application. It is therefore intended that the application not be limited to the disclosed embodiments, but that it can also cover modifications and variations within the scope of the present application.

Claims

1. A single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing, characterized in that, The single-photon lidar fog-penetrating three-dimensional imaging method includes the following steps: Step 1: Preprocess the array GM-APD lidar data by using pixel-by-pixel signal stacking compensation and global noise suppression methods, and generate a guide image using a multi-scale superpixel 3D imaging algorithm. Step 2: The preprocessed array GM-APD lidar data output in Step 1 is divided into blocks. The depth image-guided photonics processing algorithm and the pure noise band noise isolation method based on multidimensional Fourier transform are used to achieve accurate signal extraction through spatiotemporal frequency coordinated photonics processing. Step 3: Perform multi-scale block processing on the block-processed data output in Step 2 to obtain reconstructed depth and intensity images at different scales. Fuse the obtained images to output the 3D image reconstruction result with smoke interference suppression.

2. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 1, characterized in that, The pixel-by-pixel signal stacking compensation method used in step 1 is as follows: （1） In the formula, S It is the echo photon rate function under a single laser pulse condition. H For the observed histogram distribution, N The number of laser pulse cycles. i Indicates the first i Each time slot interval.

3. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 1, characterized in that, The method for accurate signal extraction in step 2, which involves spatiotemporal frequency coordinated photonic processing, includes noise isolation based on 1D denoised data, noise isolation based on 3D local data, searching for non-local correlation data, and noise isolation based on 4D non-local data.

4. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 3, characterized in that, The noise isolation method based on 1D data is as follows: perform a pixel-by-pixel Fourier transform on the echo signal of the globally gated noise suppression, and use the pure noise frequency band. B n The threshold achieves noise suppression in the frequency domain, and 1D denoised data is obtained through inverse Fourier transform. in, （3） In the formula τ p This represents the laser pulse width.

5. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 3, characterized in that, The noise isolation method based on 3D local data is as follows: The 1D denoised data is divided into local spatial blocks, and the block data is subjected to a high-dimensional Fourier transform, using the pure noise frequency band. B n Thresholds achieve the separation of noise and signal in the frequency domain.

6. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 3, characterized in that, The search for nonlocally relevant data is achieved by determining the correlation between local and nonlocal data, specifically by using the Structural Similarity Index (SSIM) to assess similarity. S d_i The calculation is performed, and the expression is as follows: （6） In the formula α and β The weighting coefficients satisfy the formula α + β= 1, of which For deep reference blocks, Divide the depth image into blocks. For strength reference blocks, Divide the intensity image into blocks; Sort the calculated similarity sets from largest to smallest, and take the top ones. N sim The number of similarities is used as the number of similarities, then the similarity ranking is as follows: N sim The data cubes corresponding to each block are defined as non-locally correlated data.

7. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 3, characterized in that, The method for noise isolation of 4D nonlocal data is as follows: Local spatial block processing is performed on the 3D denoised data, and a high-dimensional Fourier transform is performed in conjunction with nonlocal correlation characteristics, using the pure noise frequency band. B n Thresholds achieve the separation of noise and signal in the frequency domain.

8. The single-photon lidar fog-penetrating three-dimensional imaging method with synergistic photon processing according to claim 3, characterized in that, The method for fusing the reconstructed depth and intensity images at different scales obtained in step 3 to output the 3D image reconstruction result with smoke interference suppression is as follows: The fusion output based on depth and intensity images includes three steps: intensity image fusion, depth image fusion, and depth fusion image denoising. The intensity image fusion involves superimposing the generated intensity images at multiple scales at the pixel level. The depth image fusion involves filling the generated depth images at multiple scales with the mode at the pixel level to obtain a fused depth image. The denoising of the fused depth image involves thresholding the superimposed intensity image, denoising the fused depth image based on the pixel distribution of the segmentation, and obtaining the image fusion output result of multi-scale collaborative photon processing through the above steps.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor performs the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method described in any one of claims 1 to 8.

Images