Super-resolution imaging method and system using intensity layering to guide photon counting time-slice slicing

Abstract

The present application relates to the field of optical imaging, and particularly relates to a kind of intensity layered guided photon counting time slot slice super-resolution imaging method and system.Mainly solve the phenomenon that the existing imaging method is prone to the weakening of the part edge of depth image or false edge during imaging, leading to the problem of reduced spatial resolution.The method comprises: obtaining the intensity image and single-photon arrival time distribution of the same target scene;According to the single-photon arrival time distribution, obtain low spatial resolution photon counting cubic data;According to time slot order, the low spatial resolution photon counting cubic data is sequentially sliced to obtain t photon counting spatial distribution;According to the intensity image and t photon counting spatial distribution, obtain t high spatial resolution photon counting time slot slice;According to the obtained all high spatial resolution photon counting time slot slice, obtain high spatial resolution photon counting cubic data;Estimate the arrival time of each pixel photon from the high spatial resolution photon counting cubic data.

Description

Technical Field

[0001] This invention relates to an imaging method and system, specifically to a super-resolution imaging method for intensity-layered guided photon counting time-slot slices and a system for implementing the method. Background Technology

[0002] Single-photon lidar (SPD) 3D imaging systems emit a series of high-repetition-rate, narrow-pulse-width laser pulses to illuminate the target of interest. A single-photon sensitive detector records the difference between the laser pulse emission time and the arrival time of the reflected photon, thus obtaining the target distance distribution. SPD systems offer advantages such as strong low-light sensing capability, high-precision detection, and fast imaging speed. However, due to the large pixel size of the single-photon sensitive detector, the spatial resolution of SPD 3D images is often relatively low. Therefore, improving the spatial resolution of 3D images has become a current research hotspot.

[0003] Currently, using the assumption of consistent edge changes between intensity and depth images, guiding super-resolution imaging of depth images with intensity images is a fast and effective solution to the problem of low spatial resolution of depth images. Examples include "Guided Image Filtering" proposed by Kaiming He et al. and "Image Guided Depth Upsampling using Anisotropic Total Generalized Variation" proposed by Festal et al.

[0004] However, existing intensity image-guided super-resolution imaging methods for depth images are generally based on the assumption that the edge changes of the intensity image are consistent with the edges of the depth image. Since the actual texture changes of the intensity image are related not only to distance but also to reflectivity information, this method is prone to weakening some edges of the depth image or producing false edges. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problem that existing intensity image-guided depth image super-resolution imaging methods are prone to weakening of some edges of the depth image or the appearance of false edges during imaging, resulting in reduced spatial resolution. The invention provides an intensity-layered guided photon counting time-slot slicing super-resolution imaging method and system.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A super-resolution imaging method for intensity-layered guided photon counting time-slot slices, characterized by the following steps:

[0008] Step 1: Acquire high spatial resolution intensity images and single photon arrival time distributions of the same target scene;

[0009] Step 2: Obtain low spatial resolution photon counting cube data based on the single photon arrival time distribution;

[0010] Step 3: Slice the low spatial resolution photon counting cube data sequentially in the time dimension according to the time slot order to obtain the spatial distribution of photon counting in t different time slots, where t is greater than 1;

[0011] Step 4: Based on the high spatial resolution intensity image and the spatial distribution of photon counts in t different time slots, obtain high spatial resolution photon count time slot slices guided by t high spatial resolution intensity images;

[0012] Step 5: Obtain high spatial resolution photon counting cube data based on all the acquired high spatial resolution photon counting time slot slices;

[0013] Step 6: Estimate the arrival time of each pixel photon from the high spatial resolution photon counting cube data to obtain a high spatial resolution depth image, thereby achieving super-resolution imaging of the target scene.

[0014] Furthermore, step 4 specifically includes:

[0015] The spatial distribution of photon counting in t different time slots is sequentially sampled onto a high spatial resolution intensity image to obtain t initial high spatial resolution photon counting time slot slices; then the correspondence parameters between the t initial high spatial resolution photon counting time slot slices and the high spatial resolution intensity image are calculated sequentially to obtain the final high spatial resolution photon counting time slot slice guided by the t high spatial resolution intensity images.

[0016] Furthermore, in step 4:

[0017] The spatial distribution of photon counts in t different time slots was sequentially sampled onto a high spatial resolution intensity image using a linear interpolation method.

[0018] Furthermore, step 6 specifically includes:

[0019] The arrival time of each pixel photon in high spatial resolution photon counting cube data is estimated using maximum likelihood estimation, sparse regularization estimation, or first photon reconstruction methods.

[0020] Meanwhile, the present invention also provides a super-resolution imaging system for intensity-layer guided photon counting time slot slices, used to realize the aforementioned super-resolution imaging method for intensity-layer guided photon counting time slot slices;

[0021] It includes a single-photon 3D imaging module, an intensity imaging module, a slice-guided upsampling module, and a 3D reconstruction module;

[0022] The single-photon 3D imaging module is used to acquire the single-photon arrival time distribution of the target scene, and to obtain low spatial resolution photon counting cube data based on the single-photon arrival time distribution.

[0023] The intensity imaging module is used to acquire high spatial resolution intensity images of the target scene;

[0024] The output terminals of the single-photon three-dimensional imaging module and the intensity imaging module are respectively connected to the two input terminals of the slice layering guided upsampling module. The slice layering guided upsampling module is used to obtain high spatial resolution photon count cube data based on the high spatial resolution intensity image and the low spatial resolution photon count cube data.

[0025] The input of the 3D reconstruction module is connected to the output of the slice layering guided upsampling module, and is used to obtain a high spatial resolution depth image based on the high spatial resolution photon counting cube data.

[0026] Furthermore, the single-photon three-dimensional imaging module includes a single-photon detector and a time-correlated photon counting module;

[0027] The single-photon detector is used to acquire the single-photon arrival time distribution of the target scene, and its output is connected to the input of the time-correlated photon counting module; the time-correlated photon counting module is used to obtain low spatial resolution photon counting cube data based on the single-photon arrival time distribution, and its output is connected to one input of the slice layering guided upsampling module.

[0028] Furthermore, the slice-layer guided upsampling module includes a time-slot slicing module and a guide filter;

[0029] The input end of the time slot slicing module is connected to the output end of the time-correlated photon counting module, and is used to obtain the initial high spatial resolution photon counting time slot slice based on the low spatial resolution photon counting cube data;

[0030] One input of the guide filter is connected to the output of the time slot slicing module, the other input is connected to the output of the intensity imaging module, and its output is connected to the input of the three-dimensional reconstruction module. The guide filter is used to obtain the final high spatial resolution photon counting time slot slice guided by the high spatial resolution intensity image based on the initial high spatial resolution photon counting time slot slice and the high spatial resolution intensity image.

[0031] Furthermore, the intensity imaging module is an intensity detector.

[0032] Compared with the prior art, the beneficial effects of the present invention are:

[0033] 1. This invention considers the temporal distribution variation of photon number in single-photon three-dimensional imaging data. Therefore, each time slot is divided. Based on the assumption of consistency between the overall intensity information and the intensity information at different times, it only acts on the two-dimensional space and does not affect the temporal distribution of photons. This can greatly reduce the inconsistency between intensity image and depth image, and increase the realism of three-dimensional reconstruction results while achieving super-resolution imaging.

[0034] 2. This invention can solve the problems of weakened depth image edges and false textures caused by the assumption of consistency between intensity and depth changes when the existing guided upsampling algorithm is applied to single-photon imaging systems. It ensures that the spatial resolution of single-photon three-dimensional images is improved while preserving the depth changes of the original image to the greatest extent, so that the reconstruction results have smaller absolute deviations.

[0035] 3. This invention can overcome the limitation of single-photon detector pixel size on the spatial resolution of three-dimensional distance images, and realize super-resolution three-dimensional imaging based on single-photon imaging system. Attached Figure Description

[0036] Figure 1 This is a schematic diagram illustrating the implementation principle of the super-resolution imaging method for intensity-layered guided photon counting time-slot slicing according to the present invention.

[0037] Figure 2 This is a schematic diagram of the structure of an embodiment of the super-resolution imaging system for intensity-layered guided photon counting time-slot slicing according to the present invention.

[0038] In the diagram: 01 - Target scene;

[0039] 1-Single-photon 3D imaging module; 11-Single-photon detector; 12-Time-correlated photon counting module;

[0040] 2-Intensity imaging module;

[0041] 3-Slice layered guided upsampling module; 31-Time slot slicing module; 32-Guided filter;

[0042] 4-3D Reconstruction Module. Detailed Implementation

[0043] To make the objectives, advantages, and features of this invention clearer, the following detailed description of the intensity-layered guided photon counting time-slot super-resolution imaging method and system proposed in this invention, in conjunction with the accompanying drawings and specific embodiments, will further illustrate these features. The advantages and features of this invention will become clearer according to the following specific embodiments. It should be noted that the accompanying drawings are all in a very simplified form and use non-precise proportions, used only to facilitate and clarify the explanation of the embodiments of this invention; furthermore, the structures shown in the drawings are often part of the actual structures.

[0044] This invention provides a super-resolution imaging system for intensity-layered guided photon counting time-slot slicing, with reference to... Figure 2 It includes four modules: single-photon 3D imaging module 1, intensity imaging module 2, slice layer-guided upsampling module 3, and 3D reconstruction module 4.

[0045] The single-photon 3D imaging module 1 is used to acquire the single-photon arrival time distribution of the target scene 01 and obtain low spatial resolution photon counting cube data based on the single-photon arrival time distribution. Specifically, the single-photon 3D imaging module 1 includes a single-photon detector 11 and a time-correlated photon counting module 12. The single-photon detector 11 is used to acquire the single-photon arrival time distribution of the target scene 01, and its output is connected to the input of the time-correlated photon counting module 12. The time-correlated photon counting module 12 is used to obtain low spatial resolution photon counting cube data based on the single-photon arrival time distribution, and its output is connected to one input of the slice layering guided upsampling module 3.

[0046] The slice-layer guided upsampling module 3 is used to acquire high spatial resolution photon counting cube data based on the high spatial resolution intensity image and the low spatial resolution photon counting cube data. Specifically, the slice-layer guided upsampling module 3 includes a time-slot slicing module 31 and a guide filter 32. The input of the time-slot slicing module 31 is connected to the output of the time-correlated photon counting module 12, and it is used to acquire an initial high spatial resolution photon counting time-slot slice based on the low spatial resolution photon counting cube data. One input of the guide filter 32 is connected to the output of the time-slot slicing module 31, and the guide filter 32 is used to acquire a final high spatial resolution photon counting time-slot slice guided by the high spatial resolution intensity image based on the initial high spatial resolution photon counting time-slot slice and the high spatial resolution intensity image.

[0047] The intensity imaging module 2 is used to acquire a high spatial resolution intensity image of the target scene 01. The intensity imaging module 2 is an intensity detector, and its output is connected to the other input of the guide filter 32. The intensity imaging module 2 is the same as the general imaging process, and can be passively imaging to obtain laser illumination. However, the field of view of the intensity imaging needs to be registered with the single photon detection system through optical means to output a high spatial resolution intensity image.

[0048] The input of the 3D reconstruction module 4 is connected to the output of the guide filter 32, and is used to output a high spatial resolution depth map based on the high spatial resolution photon counting cube data.

[0049] Based on the aforementioned imaging system, the following describes a super-resolution imaging method for intensity-layered guided photon counting time-slot slices according to the present invention. (Refer to...) Figure 1 The method includes the following steps:

[0050] Step 1: Acquire high spatial resolution intensity images and single photon arrival time distributions of the same target scene 01;

[0051] Step 2: Obtain low spatial resolution photon counting cube data based on the single photon arrival time distribution using the time-correlated photon counting module;

[0052] Step 3: Slice the low spatial resolution photon counting cube data sequentially in the time dimension according to the time slot order to obtain the spatial distribution of photon counts in t different time slots, where t > 1;

[0053] Step 4: Based on the high spatial resolution intensity image and the spatial distribution of photon counts in t different time slots, obtain high spatial resolution photon count time slot slices guided by t high spatial resolution intensity images;

[0054] The spatial distribution of photon counting in t different time slots is sampled onto a high spatial resolution intensity image using a linear interpolation method to obtain t initial high spatial resolution photon counting time slot slices. Then, the correspondence parameters between the t initial high spatial resolution photon counting time slot slices and the high spatial resolution intensity image are calculated sequentially using a guide filter to obtain the final high spatial resolution photon counting time slot slice guided by the t high spatial resolution intensity images.

[0055] Step 5: Obtain high spatial resolution photon counting cube data based on all the acquired high spatial resolution photon counting time slot slices;

[0056] Step 6: The maximum likelihood estimation method is used to estimate the arrival time of each pixel photon from the high spatial resolution photon count cube data, thereby acquiring a high spatial resolution depth image and achieving high spatial resolution imaging of the target scene. In the presence of noise or sparse echoes, sparse regularization estimation or first-photon reconstruction methods can be used instead of maximum likelihood estimation to estimate the arrival time of each pixel photon from the high spatial resolution photon count cube data, acquiring a high spatial resolution depth image and achieving super-resolution imaging of the target scene.

[0057] Unlike conventional intensity-guided depth image upsampling schemes, traditional methods are based on the assumption of consistency between intensity and depth edge changes. They are suitable for targets with a single material, but when applied to complex targets with rich reflectivity information and diverse materials, they are prone to producing false depth textures or smooth depth image edges. In contrast, this invention considers that single-photon 3D imaging data contains changes in the temporal distribution of photon numbers. Therefore, each time slot is segmented. Based on the assumption of consistency between overall intensity information and intensity information at different times, it only acts on the two-dimensional space and does not affect the temporal distribution of photons. This can greatly reduce the inconsistency between intensity and depth images, achieving super-resolution imaging while increasing the realism of 3D reconstruction results.

Claims

1. A super-resolution imaging method for intensity-layered guided photon counting time-slot slices, characterized in that, Includes the following steps: Step 1: Acquire high spatial resolution intensity images and single photon arrival time distributions of the same target scene; Step 2: Obtain low spatial resolution photon counting cube data based on the single photon arrival time distribution; Step 3: Slice the low spatial resolution photon counting cube data sequentially in the time dimension according to the time slot order to obtain the spatial distribution of photon counting in t different time slots, where t is greater than 1; Step 4: Based on the high spatial resolution intensity image and the spatial distribution of photon counts in t different time slots, obtain high spatial resolution photon count time slot slices guided by t high spatial resolution intensity images; The spatial distribution of photon counting in t different time slots is sampled onto a high spatial resolution intensity image to obtain t initial high spatial resolution photon counting time slot slices; then the correspondence parameters between the t initial high spatial resolution photon counting time slot slices and the high spatial resolution intensity image are calculated sequentially to obtain the final high spatial resolution photon counting time slot slice guided by the t high spatial resolution intensity images. Step 5: Obtain high spatial resolution photon counting cube data based on all the acquired high spatial resolution photon counting time slot slices; Step 6: Estimate the arrival time of each pixel photon from the high spatial resolution photon counting cube data to obtain a high spatial resolution depth image, thereby achieving super-resolution imaging of the target scene.

2. The super-resolution imaging method for intensity-layered guided photon counting time-slot slices according to claim 1, characterized in that, In step 4: The spatial distribution of photon counts in t different time slots was sequentially sampled onto a high spatial resolution intensity image using a linear interpolation method.

3. The super-resolution imaging method for intensity-layered guided photon counting time-slot slices according to claim 1 or 2, characterized in that, Step 6 specifically involves: The arrival time of each pixel photon in high spatial resolution photon counting cube data is estimated using maximum likelihood estimation, sparse regularization estimation, or first photon reconstruction methods.

4. A super-resolution imaging system for intensity-layer guided photon counting time-slot slicing, used to implement the super-resolution imaging method for intensity-layer guided photon counting time-slot slicing as described in any one of claims 1-3, characterized in that: It includes a single-photon three-dimensional imaging module (1), an intensity imaging module (2), a slice-layer guided upsampling module (3), and a three-dimensional reconstruction module (4); The single-photon three-dimensional imaging module (1) is used to obtain the single-photon arrival time distribution of the target scene (01) and obtain low spatial resolution photon counting cube data based on the single-photon arrival time distribution. The intensity imaging module (2) is used to acquire a high spatial resolution intensity image of the target scene (01); The output ends of the single-photon three-dimensional imaging module (1) and the intensity imaging module (2) are respectively connected to the two input ends of the slice layering guided upsampling module (3). The slice layering guided upsampling module (3) is used to obtain high spatial resolution photon count cube data based on the high spatial resolution intensity image and the low spatial resolution photon count cube data. The input end of the three-dimensional reconstruction module (4) is connected to the output end of the slice layering guided upsampling module (3) and is used to obtain a high spatial resolution depth image based on the high spatial resolution photon counting cube data.

5. The super-resolution imaging system for intensity-layered guided photon counting time-slot slicing according to claim 4, characterized in that: The single-photon three-dimensional imaging module (1) includes a single-photon detector (11) and a time-correlated photon counting module (12); The single-photon detector (11) is used to obtain the single-photon arrival time distribution of the target scene (01), and its output is connected to the input of the time-correlated photon counting module (12). The time-correlated photon counting module (12) is used to obtain low spatial resolution photon counting cube data based on the single photon arrival time distribution, and its output is connected to one input of the slice layering guided upsampling module (3).

6. The super-resolution imaging system for intensity-layered guided photon counting time-slot slicing according to claim 5, characterized in that: The slice layering guided upsampling module (3) includes a time slot slicing module (31) and a guide filter (32); The input end of the time slot slicing module (31) is connected to the output end of the time-related photon counting module (12) and is used to obtain the initial high spatial resolution photon counting time slot slice based on the low spatial resolution photon counting cube data. One input of the guide filter (32) is connected to the output of the time slot slicing module (31), the other input is connected to the output of the intensity imaging module (2), and the output is connected to the input of the three-dimensional reconstruction module (4). The guide filter (32) is used to obtain the final high spatial resolution photon counting time slot slice guided by the high spatial resolution intensity image based on the initial high spatial resolution photon counting time slot slice and the high spatial resolution intensity image.

7. The super-resolution imaging system for intensity-layered guided photon counting time-slot slicing according to claim 6, characterized in that: The intensity imaging module (2) is an intensity detector.

Images