Array single-photon detection and two-dimensional camera common-path fusion device and pixel matching method

Abstract

The invention discloses an array single-photon detection and two-dimensional camera common-path fusion device and a pixel matching method, and belongs to the field of optical precision measurement. An array single-photon detection and two-dimensional camera common-path fusion device comprises a signal generator, a laser, a transmitting optical system, a receiving optical system, a dichroscope, a two-dimensional camera, an array single-photon detector, an array single-photon counter and a data acquisition and processing fusion module. The implementation method comprises the following steps: separating array single-photon detection wave band pulse laser returned by a target object and two-dimensional camera induction wave band ambient light by using a common receiving light path and a dichroscope to determine the pixel size and the number of an array single-photon detector and a two-dimensional camera; the transmission light offset d is calculated according to the thickness of the dichroscope; the transmission light offset d serves as a correction factor, and pixel matching of array single-photon detection three-dimensional imaging and two-dimensional camera images is achieved; an array single photon detection three-dimensional image and a two-dimensional camera image are superposed, and the high resolution of the two-dimensional image is utilized to enrich three-dimensional point cloud imaging information.

Description

Technical Field

[0001] The invention belongs to the field of optical precision measurement and relates to an array single-photon detection and two-dimensional camera common-path fusion device and a pixel matching method. Background Art

[0002] Long-distance high-precision target positioning and identification is of great significance in the fields of obstacle positioning, identification and warning, remote sensing, geological disaster warning, etc. Single-photon detection imaging technology belongs to the new generation of quantum imaging technology, with picosecond accuracy and photon-level sensitivity. It can realize long-distance three-dimensional imaging with time-correlated single-photon counting technology. Since it requires time accumulation, the speed of single-point scanning single-photon imaging is very slow. Array single-photon detection imaging can realize long-distance single-photon imaging of relatively fast dynamic targets, but the number of existing array single-photon pixels is relatively small, generally 64×64, 128×128, so the corresponding point cloud is relatively sparse, so it is difficult to distinguish the details of the target object at a long distance. Two-dimensional cameras are relatively mature and have grayscale and color. The number of pixels can easily reach millions of pixels or even higher, and rich details and texture information of the target object can be obtained, but there is no distance depth information. Image fusion of array single-photon detection imaging and two-dimensional cameras can realize multi-dimensional information of the target, which is easier to identify and classify the target. Existing multi-sensor image fusion methods mostly fuse independent sensor systems through field of view alignment. This can easily lead to field of view mismatch when zooming at different distances, and requires high back-end computing power. Summary of the invention

[0003] In order to realize multi-sensor fusion matching of target positioning and identification at long distances and different focal lengths, the purpose of the present invention is to provide an array single-photon detection and two-dimensional camera common-path fusion device and a pixel matching method, a signal generator is used to synchronize the laser and the array single-photon counter, and a common receiving optical system and a dichroic mirror are used to realize the same-field matching reception of the array single-photon detector band laser and the two-dimensional camera sensing band light, and the image fusion of the column single-photon detection three-dimensional imaging and the two-dimensional image is realized through the pixel correction matching algorithm, so as to obtain richer multi-dimensional information of the imaging target.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] The array single-photon detection and two-dimensional camera common-path fusion device disclosed in the present invention comprises a signal generator, a laser, a transmitting optical system, a receiving optical system, a dichroic mirror, a two-dimensional camera, an array single-photon detector, an array single-photon counter and a data acquisition processing fusion module. The signal generator gives synchronous pulse signals to the laser and the array single photon counter respectively. The pulse laser emitted by the laser is in the sensing band of the array single photon detector. The pulse laser is irradiated to the target object through the transmitting optical system. The pulse laser and the ambient light of the sensing band of the two-dimensional camera are not in the same band. The reflected light of the target object includes the pulse laser and the ambient light of the sensing band of the two-dimensional camera. The reflected light is received by the receiving optical system, and then the pulse laser in the sensing band of the array single photon detector and the ambient light in the sensing band of the two-dimensional camera are separated through the dichroic mirror. The pulse laser is detected by the array single photon detector and converted into an electrical signal, which is sent to the array single photon counter. After histogram accumulation and statistics, the pulse laser flight time and the target distance value corresponding to each pixel of the array single photon detector are extracted and solved, and the target distance value is transmitted to the data acquisition processing fusion module to generate a three-dimensional point cloud image; the ambient light is sensed and collected by the two-dimensional camera to generate a two-dimensional image, which is transmitted to the data acquisition processing fusion module. The data acquisition processing fusion module performs pixel correction matching on the three-dimensional point cloud image and the two-dimensional image to obtain the result to realize the image fusion of the three-dimensional imaging of the array single photon detection and the two-dimensional image, and obtain more abundant multi-dimensional information of the imaging target.

[0006] The data acquisition, processing and fusion module performs pixel correction and matching on the single-photon detector pixels in the array and the two-dimensional camera pixels. The specific implementation method is as follows:

[0007] The number of pixels in the array single photon detector is m 水平 ×n 竖直 , the pixel spacing is d1, and the number of pixels of the two-dimensional camera is a 水平 ×b 竖直 , the pixel spacing is d2, the thickness of the dichroic mirror is L, the translation of the transmitted light is d, since the dichroic mirror is placed at 45° relative to the incident light, the incident angle is 45°, and the refractive index of the dichroic mirror is n. Therefore, through geometric optics calculation, the translation of the transmitted light d is

[0008]

[0009] The direction of the translation of the transmitted light corresponding to the dichroic mirror is horizontal or vertical, then the number of pixels x1 of the array single-photon detector translated corresponding to the translation d of the transmitted light is

[0010]

[0011] The number of pixels x2 of the two-dimensional camera translation corresponding to the transmitted light translation d is

[0012]

[0013] Pixel matching is achieved by moving the position of the array single-photon detector pixel or the position of the two-dimensional camera pixel. If the array single-photon detector pixel is moved, the pixel matching is achieved by correcting and translating x1 array single-photon detector pixels along the translation direction of the transmitted light. If the two-dimensional camera pixel is moved, the pixel matching is achieved by correcting and translating x2 two-dimensional camera pixels along the translation direction of the transmitted light, thus achieving pixel matching and fusion of array single-photon detection three-dimensional imaging and two-dimensional camera images.

[0014] In the array single-photon counter, the method for calculating the flight time and the target distance value is as follows: the signal generator gives synchronous pulse signals to the laser and the array single-photon counter respectively, and the array single-photon counter records the starting time t0. The laser emits a pulse laser, which is irradiated to the target object through the transmitting optical system. The reflected pulse laser is received by the receiving optical system and passes through the dichroic mirror. It is received by the array single-photon detector and converted into an electrical signal, which is input into the array single-photon counter and records the end time t1. Since the signal after the pulse laser reflection is relatively weak, it requires the accumulation of multiple pulse cycles. After the histogram statistics, the pulse shape and position corresponding to the target object are restored. Through the pulse positioning method, the time point t corresponding to the target object is extracted. 目标 , get the target object flight time △t = t 目标 -t0, the target distance value D is obtained according to the photon flight time, where the refractive index corresponding to the laser wavelength in the air is n g .

[0015]

[0016] The array single photon detector is one of a Geiger mode avalanche diode array single photon detector, a superconducting nanowire array single photon detector, and a photomultiplier tube array single photon detector.

[0017] The array single photon detection and two-dimensional camera common light path pixel matching method is implemented based on the array single photon detection and two-dimensional camera common light path fusion device. The array single photon detection and two-dimensional camera common light path pixel matching method comprises the following steps:

[0018] The first step is to use the common receiving optical path and dichroic mirror to separate the array single-photon detection band pulse laser and the two-dimensional camera sensing band ambient light returned by the target object; the second step is to determine the pixel size and number of the array single-photon detector and the two-dimensional camera; the third step is to calculate the transmitted light offset d according to the thickness of the dichroic mirror; the fourth step is to use the transmitted light offset d as a correction factor to achieve pixel matching between the array single-photon detection three-dimensional imaging and the two-dimensional camera image; the fifth step is to superimpose the array single-photon detection three-dimensional image and the two-dimensional camera image, and use the high resolution of the two-dimensional image to enrich the three-dimensional point cloud imaging information.

[0019] Beneficial effects:

[0020] 1. The present invention discloses an array single-photon detection and two-dimensional camera common-path fusion device and a pixel matching method. A signal generator gives synchronous pulse signals to a laser and an array single-photon counter respectively. The pulse laser emitted by the laser is in the sensing band of the array single-photon detector. The pulse laser is irradiated to a target object through a transmitting optical system. The pulse laser and the ambient light in the sensing band of the two-dimensional camera are not in the same band. The reflected light of the target object includes the pulse laser and the ambient light in the sensing band of the two-dimensional camera. The reflected light is received by a receiving optical system, and then the pulse laser in the sensing band of the array single-photon detection and the ambient light in the sensing band of the two-dimensional camera are separated through a dichroic mirror. The pulse laser is detected by the array single-photon detector and converted into an electrical signal, which is sent to the array single-photon counter. After histogram accumulation and statistics, the pulse laser flight time is extracted and transmitted to a data acquisition, processing and fusion module to generate a three-dimensional point cloud image. The ambient light is sensed and collected by the two-dimensional camera to generate a two-dimensional image, which is transmitted to the data acquisition, processing and fusion module. The data acquisition, processing and fusion module performs pixel correction and matching on the three-dimensional point cloud image and the two-dimensional image to obtain a result to realize the image fusion of the three-dimensional imaging of the array single-photon detection and the two-dimensional image.

[0021] 2. Traditional multi-sensor fusion has different receiving optical systems and different fields of view, so the fusion process is complicated, the field of view registration is difficult, and the imaging range is short. The array single-photon detection and two-dimensional camera common-path fusion device and pixel matching method disclosed in the present invention use a common receiving optical system, a dichroic mirror, and a pixel correction matching method to achieve image fusion of array single-photon detection imaging and two-dimensional camera images, which can solve the problems of field of view mismatch, registration difficulties, and short imaging distance in existing multi-sensor fusion, and obtain richer multi-dimensional information of the imaging target. In addition, compared with traditional three-dimensional imaging radar, array single-photon detection imaging has photon-level sensitivity and picosecond-level accuracy, and does not require scanning imaging. After fusion with a two-dimensional camera, it can achieve high-frame rate, fast-moving and static targets at tens of kilometers of ultra-long-distance single-photon three-dimensional imaging and two-dimensional camera matching fusion imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of a common-path fusion device based on array single-photon detection and two-dimensional camera;

[0023] Figure 2 Schematic diagram of array single photon detection and two-dimensional camera pixel matching;

[0024] Among them, 1-signal generator, 2-laser, 3-transmitting optical system, 4-target object, 5-receiving optical system, 6-dichroic mirror, 7-two-dimensional camera, 8-array single photon detector, 9-array single photon counter, 10-data acquisition and processing fusion module DETAILED DESCRIPTION

[0025] In order to better illustrate the purpose and advantages of the present invention, the invention is further described below with reference to the accompanying drawings and examples.

[0026] Embodiment 1:

[0027] like Figure 1 As shown, the array single-photon detection and two-dimensional camera co-path fusion device disclosed in this embodiment includes a signal generator 1, a laser 2, a transmitting optical system 3, a receiving optical system 5, a dichroic mirror 6, a two-dimensional camera 7, an array single-photon detector 8, an array single-photon counter 9 and a data acquisition processing fusion module 10; the signal generator 1 gives synchronous pulse signals to the laser 2 and the array single-photon counter 9 respectively, the wavelength of the pulse laser emitted by the laser 2 is 1550nm, the array single-photon detector is made of indium gallium arsenide material, and the sensing band is near-infrared 900nm~1600nm, the wavelength of the pulse laser emitted by the laser 2 is in the sensing band of the array single-photon detector 8, the pulse laser passes through the transmitting optical system 3 to irradiate the target object 4, the pulse laser and the ambient light in the sensing band of the two-dimensional camera 7 are not in the same band, the sensing band of the two-dimensional camera is 400nm~900nm, which is mainly the visible light band, the reflected light of the target object 4 includes the pulse laser and the ambient light in the sensing band of the two-dimensional camera 7, the reflected light is received by the receiving optical system 5, and then the array single-photon detector is transmitted through the dichroic mirror 6. The pulse laser in the sensing band of the sub-detector 8 and the ambient light in the sensing band of the two-dimensional camera 7 are separated. The dichroic mirror 6 has a reflection band in the band of 400nm to 900nm and a transmission band of 900nm to 1600nm. The thickness L is 1mm and the refractive index is 1.5. Therefore, the 1550nm band sensed by the array single-photon detector 8 is transmitted, and the visible light band sensed by the two-dimensional camera 7 is reflected. The pulse laser is detected by the array single-photon detector 8 and converted into an electrical signal, which is sent to the array single-photon counter 9. After histogram accumulation and statistics, the pulse laser flight time and target distance value corresponding to each pixel of the array single-photon detector 8 are extracted and solved, and the target distance value is transmitted to the data acquisition, processing and fusion module 10 to generate a three-dimensional point cloud image; the ambient light is sensed and collected by the two-dimensional camera 7 to generate a two-dimensional image, which is transmitted to the data acquisition, processing and fusion module 10. The data acquisition, processing and fusion module 10 performs pixel correction and matching on the three-dimensional point cloud image and the two-dimensional image to obtain a result to realize the image fusion of the three-dimensional imaging of the array single-photon detection and the two-dimensional image, and obtain more abundant multi-dimensional information of the imaging target.

[0028] The data acquisition processing fusion module 10 performs pixel correction matching on the pixel of the array single photon detector 8 and the pixel of the two-dimensional camera 7. The specific implementation method is as follows:

[0029] like Figure 2 As shown, the number of pixels of the array single photon detector 8 is m水平 ×n 竖直 =64×64, the pixel pitch is d1=50μm, and the pixel number of the two-dimensional camera 7 is a 水平 ×b 竖直 =1000×1000, the pixel spacing is d2=10μm, the dichroic mirror 6 is reflective in the 400nm~900nm band, and is transmissive in the 900nm~1600nm band, the thickness is L=1mm, the translation amount d of the transmitted light, since the dichroic mirror 6 is placed at 45° relative to the incident light, the incident angle is 45°, and the refractive index of the dichroic mirror 6 is n=1.5, so through geometric optics calculation, the translation amount d of the transmitted light is

[0030]

[0031] The direction of the translation amount of the transmitted light corresponding to the dichroic mirror 6 is the horizontal direction or the vertical direction. Then the number of pixels x1 of the array single-photon detector 8 translated corresponding to the translation amount d of the transmitted light is

[0032]

[0033] The number of pixels x2 of the two-dimensional camera 7 translated corresponding to the transmitted light translation d is

[0034]

[0035] Pixel matching is achieved by moving the position of the array single photon detector 8 pixels or the position of the two-dimensional camera 7 pixels. If the array single photon detector 8 pixels are moved, 6.6 array single photon detector 8 pixels are corrected and translated along the translation direction of the transmitted light to achieve pixel matching. If the two-dimensional camera 7 pixels are moved, 33 two-dimensional camera 7 pixels are corrected and translated along the translation direction of the transmitted light to achieve pixel matching, and the pixel matching and fusion of the array single photon detection three-dimensional imaging and the two-dimensional camera 7 image are achieved.

[0036] In the array single-photon counter 9, the method for calculating the flight time and the target distance value is as follows: the signal generator 1 gives a synchronous pulse signal to the laser 2 and the array single-photon counter 9 respectively, and the array single-photon counter 9 records the starting time t0, the laser 2 emits a pulse laser, and the pulse laser passes through the transmitting optical system 3 to irradiate the target object 4, and the reflected pulse laser is received by the receiving optical system 5 and passes through the dichroic mirror 6, and is received by the array single-photon detector 8 and converted into an electrical signal, which is input into the array single-photon counter 9, and the end time t1 is recorded. Since the signal after the pulse laser is reflected is relatively weak, it requires the accumulation of multiple pulse cycles. After the histogram statistics, the pulse shape and position corresponding to the target object 4 are restored, and the time point t corresponding to the target object 4 is extracted through the pulse positioning method. 目标 , obtain the target object 4 flight time △t = t 目标-t0, the target distance value D is obtained according to the photon flight time, where the refractive index corresponding to the laser wavelength in the air is n g .

[0037]

[0038] The array single photon detector 8 is one of a Geiger mode avalanche diode array single photon detector 8 , a superconducting nanowire array single photon detector 8 , and a photomultiplier tube array single photon detector 8 .

[0039] The array single photon detection and two-dimensional camera common light path pixel matching method is implemented based on the array single photon detection and two-dimensional camera common light path fusion device. The array single photon detection and two-dimensional camera common light path pixel matching method has the following specific implementation steps:

[0040] The first step is to use the common receiving optical path and the dichroic mirror 6 to separate the array single-photon detection band pulse laser and the two-dimensional camera 7 sensing band ambient light returned by the target object 4; the second step is to determine the pixel size and number of the array single-photon detector 8 and the two-dimensional camera 7; the third step is to calculate the transmitted light offset d according to the thickness of the dichroic mirror 6; the fourth step is to use the transmitted light offset d as a correction factor to achieve pixel matching between the array single-photon detection three-dimensional imaging and the two-dimensional camera 7 image; the fifth step is to superimpose the array single-photon detection three-dimensional image and the two-dimensional camera 7 image, and use the high resolution of the two-dimensional image to enrich the three-dimensional point cloud imaging information.

[0041] The specific description above further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. Array single photon detection and two-dimensional camera co-path fusion device, characterized by: The invention comprises a signal generator, a laser, a transmitting optical system, a receiving optical system, a dichroic mirror, a two-dimensional camera, an array single-photon detector, an array single-photon counter and a data acquisition and processing fusion module; the signal generator respectively gives synchronous pulse signals to the laser and the array single-photon counter, the pulse laser emitted by the laser is in the sensing band of the array single-photon detector, the pulse laser is irradiated to the target object through the transmitting optical system, the pulse laser and the ambient light in the sensing band of the two-dimensional camera are not in the same band, the reflected light of the target object includes the pulse laser and the ambient light in the sensing band of the two-dimensional camera, the reflected light is received by the receiving optical system, and then the pulse laser in the sensing band of the array single-photon detector and the ambient light in the sensing band of the two-dimensional camera are combined through the dichroic mirror. The two-dimensional camera senses the ambient light in the band, and the pulsed laser is detected by the array single-photon detector and converted into an electrical signal, which is sent to the array single-photon counter. After histogram accumulation and statistics, the pulsed laser flight time and target distance value corresponding to each pixel of the array single-photon detector are extracted and solved, and the target distance value is transmitted to the data acquisition, processing and fusion module to generate a three-dimensional point cloud image; the ambient light is sensed and collected by the two-dimensional camera to generate a two-dimensional image, which is transmitted to the data acquisition, processing and fusion module. The data acquisition, processing and fusion module performs pixel correction and matching on the three-dimensional point cloud image and the two-dimensional image, and the result realizes the image fusion of the array single-photon detection three-dimensional imaging and the two-dimensional image, and obtains richer multi-dimensional information of the imaging target.

2. The array single photon detection and two-dimensional camera co-path fusion device as claimed in claim 1, characterized in that: The data acquisition, processing and fusion module performs pixel correction and matching on the single-photon detector pixels in the array and the two-dimensional camera pixels. The specific implementation method is as follows: The number of pixels in the array single photon detector is m 水平 ×n 竖直 , the pixel spacing is d1, and the number of pixels of the two-dimensional camera is a 水平 ×b 竖直 , the pixel spacing is d2, the thickness of the dichroic mirror is L, the translation of the transmitted light is d, since the dichroic mirror is placed at 45° relative to the incident light, the incident angle is 45°, and the refractive index of the dichroic mirror is n. Therefore, through geometric optics calculation, the translation of the transmitted light d is The direction of the translation of the transmitted light corresponding to the dichroic mirror is horizontal or vertical, then the number of pixels x1 of the array single-photon detector translated corresponding to the translation d of the transmitted light is The number of pixels x2 of the two-dimensional camera translation corresponding to the transmitted light translation d is Pixel matching is achieved by moving the array single-photon detector pixel position or the two-dimensional camera pixel position; if the array single-photon detector pixel is moved, then the array single-photon detector pixel is corrected and translated by x1 along the translation direction of the transmitted light to achieve pixel matching; if the two-dimensional camera pixel is moved, then the two-dimensional camera pixel is corrected and translated by x2 along the translation direction of the transmitted light to achieve pixel matching, thereby realizing pixel matching and fusion of array single-photon detection three-dimensional imaging and two-dimensional camera image.

3. The array single photon detection and two-dimensional camera co-path fusion device as claimed in claim 1, characterized in that: In the array single-photon counter, the method for calculating the flight time and the target distance value is as follows: the signal generator gives synchronous pulse signals to the laser and the array single-photon counter respectively, and the array single-photon counter records the starting time t0. The laser emits a pulse laser, which is irradiated to the target object through the transmitting optical system. The reflected pulse laser is received by the receiving optical system and passes through the dichroic mirror. It is received by the array single-photon detector and converted into an electrical signal, which is input into the array single-photon counter and records the end time t1. Since the signal after the pulse laser reflection is relatively weak, it requires the accumulation of multiple pulse cycles. After the histogram statistics, the pulse shape and position corresponding to the target object are restored. Through the pulse positioning method, the time point t corresponding to the target object is extracted. 目标 , get the target object flight time △t = t 目标 -t0, the target distance value D is obtained according to the photon flight time, where the refractive index corresponding to the laser wavelength in the air is n g ; 4. The array single photon detection and two-dimensional camera co-path fusion device as claimed in claim 1, characterized in that: The array single photon detector is one of a Geiger mode avalanche diode array single photon detector, a superconducting nanowire array single photon detector, and a photomultiplier tube array single photon detector.

5. A method for matching common optical path pixels of array single photon detection and two-dimensional camera, which is implemented based on the common optical path fusion device of array single photon detection and two-dimensional camera as claimed in claim 1, 2, 3 or 4, characterized in that: The following steps are included: In the first step, the array single-photon detection band pulse laser and the two-dimensional camera sensing band ambient light returned by the target object are separated by using a common receiving optical path and a dichroic mirror; The second step is to determine the pixel size and number of the array single-photon detector and the two-dimensional camera; the third step is to calculate the transmitted light offset d according to the thickness of the dichroic mirror; the fourth step is to use the transmitted light offset d as a correction factor to achieve pixel matching between the array single-photon detection three-dimensional imaging and the two-dimensional camera image; the fifth step is to superimpose the array single-photon detection three-dimensional image and the two-dimensional camera image, and use the high resolution of the two-dimensional image to enrich the three-dimensional point cloud imaging information.

Images