Direct time-of-flight 3D imaging method and device based on speckle projection

Abstract

The application discloses a direct time-of-flight 3D imaging device based on speckle projection, comprising: a speckle projection module for emitting multiple light beams and forming spaced speckles on a target surface; a SPAD array receiving module for receiving the speckle reflection light beams of the target surface and recording the light receiving time; a control processing module for controlling the speckle projection module to emit light beams and recording the light emitting time, and processing the depth distance information according to the light flight time obtained from the light emitting time and the light receiving time. The application also discloses a direct time-of-flight 3D imaging method based on speckle projection. The application can directly record the light emitting time and the light receiving time, and the device has the characteristics of long imaging distance, high signal-to-noise ratio, strong anti-interference, high precision and low power consumption.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of photoelectric imaging, in particular to a direct time-of-flight 3D imaging method and device based on speckle projection. BACKGROUND

[0002] 3D imaging devices can greatly enrich the user experience and improve product competitiveness. Unlike traditional 2D imaging devices such as cameras, which can only obtain the planar 2D information of an object, 3D imaging devices can also obtain the depth information of an object and construct a three-dimensional 3D model, so 3D imaging devices are widely used in industrial measurement, part modeling, medical diagnosis, security monitoring, machine vision, biometric identification, augmented reality AR, virtual reality VR and other fields, and have great application value.

[0003] 3D imaging technology is divided into active and passive types, with structured light and time-of-flight technology being the mainstream of active type, and binocular vision being the mainstream of passive type. Due to the influence of objective factors such as external environment and surface texture properties of the shooting object, and the complexity of the feature point automatic matching algorithm, the current 3D imaging consumer electronics field has not been popularized, and the passive application of structured light and time-of-flight technology (TOF) is the mainstream in the market. The structured light technology scheme is characterized by containing a speckle projector and an IR imaging module, and the accuracy within a distance of 1m reaches sub-millimeter, which meets the highest 3D face recognition accuracy requirement of the financial payment industry, but the long-distance imaging accuracy will decrease sharply, and also consumes certain calculation resources for depth algorithm, which greatly limits the application scene range. The time-of-flight technology, i.e. the TOF technology scheme, is characterized by containing an illumination emission module and a TOF sensor receiving module, and has a long imaging distance and a wide application scene, such as rear modeling and 3D sensing of smart phones, 3D sensing of AR / VR devices to external environment, laser 3D ranging system of cars, etc. The disadvantage of TOF technology is that the accuracy in the whole imaging range is not as high as that of structured light technology, but more application scenes do not require such high accuracy, so TOF has a wider application prospect.

[0004] TOF is the full name of Time-Of-Flight, that is, flight time. The time interval from the moment of emitting light to the moment of reflecting the light to the receiving end is measured, and the distance measurement can be realized according to the principle of constant light speed. There are two TOF technology routes at present: I-TOF (indirect-ToF) and D-TOF (direct-ToF). The I-TOF technology is commonly used in the current market. A time periodically modulated laser is emitted by a laser emitting device to the surface of an object. The returned light produces a time delay relative to the incident light in time sequence, which is specifically manifested as a phase delay. The size of the phase delay has a corresponding calculation relationship with the flight time of the light. The I-TOF technology indirectly obtains the flight time of the light by measuring the phase delay, and then realizes distance measurement. Since the flight time of the light is indirectly obtained by measuring the phase, the measurement of the phase is actually indirectly realized by relying on the energy strength of the received light. Thus, the problem of inaccurate phase measurement is caused, and the phase measurement is easily disturbed by ambient light. Once the distance continues to rise, for example, above 10 m, the reflected and received light will be sharply attenuated, and even be submerged by ambient light. The measurement accuracy will sharply decrease. At the same time, the I-TOF technology essentially uses the light energy average integration method to obtain the distance value of each pixel on the sensor. Therefore, the problem of multi-path interference is inevitably caused, which leads to problems such as unclear object contour, distorted corner shape, and inability to distinguish adjacent two objects.

[0005] The D-TOF technology directly measures the flight time Δt. In the pulse debugging signal mode, each pixel inside the sensor directly measures the round-trip time of the photons. The D-TOF technology has the advantages of high sensitivity, avoids various problems caused by the measurement of the phase difference of the round-trip signal, and does not have the process of photoelectron accumulation. The measurement accuracy of the D-TOF technology is less affected by light noise. The requirement of low signal-to-noise ratio is beneficial to the reduction of system power consumption. SUMMARY

[0006] In order to solve the above technical problems, the present application provides a direct flight time 3D imaging method and device based on speckle projection, which directly realizes time recording at two time points of light emission and light reception. The device has the characteristics of long imaging distance, high signal-to-noise ratio, strong anti-interference, high precision and low power consumption.

[0007] The specific technical scheme adopted by the present application is as follows:

[0008] The present application provides a direct flight time 3D imaging device based on speckle projection, comprising:

[0009] The speckle projection module is used for emitting a plurality of light beams and forming a plurality of spaced speckles on the target surface.

[0010] The SPAD array receiving module is configured to receive the speckle reflection light beam of the target surface and record the light receiving time;

[0011] The control processing module is configured to control the speckle projection module to emit the light beam and record the light emitting time, and process the depth distance information according to the light flight time obtained from the light emitting time and the light receiving time.

[0012] The following also provides preferred modes, but not as an additional limitation to the above general scheme, but only as a further supplement or preference, without technical or logical contradiction, each preferred mode can be combined with the above general scheme alone, and can also be combined between multiple preferred modes.

[0013] As preferred, the speckle projection module comprises a light source and an ultra-short focal collimating lens.

[0014] The number n of light emitting holes of the light source is greater than 500, and the light beam is emitted with angular divergence.

[0015] The focal length f of the ultra-short focal collimating lens is less than 1.5 mm.

[0016] The speckle projection module of the present application only contains two key optical elements, a laser emitter and an ultra-short focal collimating lens, and does not set a diffraction element DOE. A large number (n>500) of light emitting holes are used to generate a speckle projection pattern with high enough density, which can make up for the replication function without using DOE. Further, since an ultra-short focal collimating lens with a focal length f<1.5 mm is used, the distortion of the projected speckle pattern can be ignored.

[0017] As preferred, the light source is a laser emitter, including a vertical cavity surface laser emitter or a surface array laser combined by a single point laser.

[0018] The laser emitter as the light source can be a surface array laser, such as a vertical cavity surface laser emitter, or a surface array combined by a single point laser, such as an edge emitting laser. The emission wavelength can be selected according to the application needs, and the preferred vertical cavity surface laser emitter VCSEL with regular dot array distribution is used, and the emission wavelength is 940 nm.

[0019] As preferred, the light emitting holes are regularly arranged or row-column misaligned.

[0020] The light emitting holes in the present application can be regular or row-column misaligned, and the regularly distributed or misaligned light emitting holes have equal spacing. The misaligned distribution can be in various optional forms, such as adjacent two rows misaligned or adjacent two columns misaligned.

[0021] As preferred, the light emitting holes are grouped and sequentially emit light beams.

[0022] In a preferred embodiment, the light emitting holes form a scanning speckle by being illuminated in groups, and the speckle reflected light beams are received by the SPAD array receiving module, which can reduce the integration complexity of the SPAD sensor, and reduce the sensor area and pixel size.

[0023] Preferably, the ultra-short focal collimating lens expands the light beams to exit with a field of view angle a>20°.

[0024] Preferably, the SPAD array receiving module comprises, in sequence along the optical path, an imaging receiving lens, a narrowband filter, and a SPAD sensor.

[0025] Preferably, the SPAD sensor comprises a plurality of binning macro-pixel units corresponding to the speckles, each binning macro-pixel unit having a plurality of sub-SPAD pixels, and the binning macro-pixel units are consistent with the arrangement and grouping of the light emitting holes.

[0026] Binning is an image reading mode that combines the photoelectric conversion of adjacent pixels together to be read out in the mode of one pixel. The advantage of binning is to combine several pixels as one pixel to improve sensitivity and output speed.

[0027] The application also provides a direct time-of-flight 3D imaging method based on speckle projection, characterized in that it comprises:

[0028] The control processing module drives the speckle projection module to emit a plurality of light beams to form a plurality of spaced speckles on the target surface, and records the light emission time;

[0029] The SPAD array receiving module receives the speckle reflected light beams of the target surface, and records the light receiving time;

[0030] The light flight time obtained from the light emission time and the light receiving time is processed to generate depth distance information.

[0031] Preferably, the speckle projection module comprises a light source and an ultra-short focal collimating lens.

[0032] The number n of light emitting holes of the light source is greater than 500, the light emitting holes are divided into a plurality of groups, and a group of depth distance information is obtained after the light emitting holes are illuminated in sequence, and the complete distance depth information is generated after the depth distance information of the plurality of groups is fused.

[0033] The focal length f of the ultra-short focal collimating lens is less than 1.5 mm.

[0034] Compared with the prior art, the application has the following advantages:

[0035] The application avoids using diffraction optical elements, has simple and compact structure, and low material cost. One optical element is reduced, the light energy loss of the system is reduced, the energy utilization rate of the laser light source is improved, and the entire 3D imaging system has higher signal-to-noise ratio under the same emission power. Moreover, the emission module does not produce optical distortion, and the receiving module does not need to consider compensating for distortion, greatly reducing the design difficulty of the receiving lens group and the system matching complexity between the emission module and the receiving module. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 Fig. 1 is a structural diagram of the 3D imaging device in the application;

[0037] Figure 2 Fig. 3 is a structural diagram of the speckle projection module;

[0038] Figure 3 Fig. 4 is a contrast diagram of the projection speckle pattern distortion, wherein (a) is a speckle pattern with distortion, and (b) is a speckle pattern without distortion;

[0039] Figure 4 Fig. 5 is a light-emitting hole arrangement distribution diagram, wherein (a) is regularly arranged light-emitting holes, and (b) is staggered distributed light-emitting holes;

[0040] Figure 5 Fig. 6 is a plurality of optional arrangement and distribution modes of the light-emitting hole;

[0041] Figure 6 Fig. 7 is a structural diagram of the SPAD array receiving module;

[0042] Figure 7 Fig. 8 is a wavelength transmittance curve diagram;

[0043] Figure 8 Fig. 9 is a pixel distribution diagram in the SPAD sensor;

[0044] Figure 9 Fig. 10 is a cross-sectional schematic diagram of the SPAD sensor;

[0045] Figure 10 Fig. 11 is a grouped scanning speckle diagram;

[0046] Figure 11 Fig. 12 is a working schematic diagram of the 3D imaging device in the application;

[0047] Figure 12 Fig. 13 is a working diagram of the grouped SPAD pixel;

[0048] Figure 13 Fig. 14 is a pulse current control diagram. DETAILED DESCRIPTION

[0049] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the invention is not limited to the specific embodiments disclosed below.

[0050] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0051] like Figure 1 The direct time-of-flight 3D imaging device based on speckle projection shown includes a control and processing module 10, a speckle projection module 11, and a SPAD array receiving module 13.

[0052] The speckle projection module 11 includes a light source, which emits multiple beams of light. After passing through the internal optical system of the module, the beams of light are finally distributed as speckles 121 on the target surface (such as the receiving screen 12).

[0053] SPAD array receiving module 13 includes a binning macropixel unit corresponding to speckle 121. The binning macropixel unit is used to receive the speckle reflected beam from the target surface and trigger the internal precision timing circuit to record the light receiving time.

[0054] The control processing module 10 controls the speckle projection module 11 to emit a laser beam. It can also control the speckle emitted by the speckle projection module to scan the target surface. Based on the TOF value of the corresponding SPAD pixel obtained in each scan, it processes and generates depth and distance data and transmits it to an external device.

[0055] The control and processing module 10 is a comprehensive system module containing a high-speed laser drive circuit and a power management circuit. Its main function is to coordinate and control the speckle projection module 11 and the SPAD array receiving module 13, and to perform data calculation and processing. After receiving the high-speed synchronous start signal from the SPAD array receiving module 13, the control and processing module 11 modulates and shapes the signal, amplifies its power, generates a drive pulse, and then drives the speckle projection module 11 to pulse-modulate the emission of its internal light source, projecting multiple laser beams with precisely controlled angles. These beams hit the object in front, such as the receiving screen 12, forming speckles 121 covering the screen. Simultaneously, the SPAD array receiving module 13 starts working synchronously and records the initial moment of light emission. The reflected beams from the speckle 121 are received by the SPAD array receiving module 13, triggering the internal precision timing circuit to record the light reception time. The time difference between the light reception time and the light emission time is the desired light flight time Δt. The distance d can then be calculated using the following basic formula:

[0056] d = At * c / 2

[0057] In the formula, c represents the speed of light, which is about equal to the constant 3 x 108m / s in vacuum. Various algorithm models can be used in practical applications to improve environmental adaptability and calculation accuracy, but they cannot be separated from this core light speed ranging formula.

[0058] Each speckle 121 corresponds to each binning macro-pixel unit in the SPAD array receiving module 13, so the light flight time At corresponding to each binning macro-pixel unit can be obtained, and then a whole distance depth map of the front object scene can be obtained, achieving the purpose of 3D imaging.

[0059] As shown in Figure 2 In the most preferred embodiment, the speckle projection module 11 mainly includes a laser emitter 111 and an ultra-short focal collimating lens 112. The innovation of this embodiment is the use of an ultra-short focal collimating lens, whose focal length f is very small, limited to f < 1.5mm, which can realize laser beam collimation while having a very large exit field angle a, limited to a > 20°. The difference from the projection module used in other 3D imaging schemes is that this embodiment only contains two key optical elements, the laser emitter 111 and the ultra-short focal collimating lens 112, while the traditional projection module is composed of three parts, the laser emitter + collimating mirror + diffractive element DOE. Therefore, the invention scheme of this embodiment has the advantages of simple structure and low material cost, and avoids the production complexity brought by the assembly of the diffractive element DOE, reduces one optical element, also reduces the light energy loss of the system, improves the energy utilization rate of the laser light source, and under the same emission power, the whole 3D imaging system also has a higher signal-to-noise ratio.

[0060] The traditional speckle projection module composed of a laser emitter + a collimating mirror + a diffractive element DOE will produce a large diffraction distortion when the beam is diffracted, and a speckle pattern distortion will be produced on the receiving screen as shown in Figure 3 (a). In this preferred embodiment, since no diffractive element is used, the optical distortion produced by the ultra-short focal collimating lens 112 itself is small and controllable, so the speckle pattern projected by the speckle projection module 11 of this embodiment has no distortion, as shown in Figure 3 (b).

[0061] In another preferred embodiment, the light source has n > 500 light emitting holes, emitting an angularly divergent light beam. As the light source, the laser emitter 111 can be a surface array laser such as a vertical cavity surface emitting laser, or a surface array laser composed of single-point lasers such as an edge-emitting laser, and the emission wavelength can be selected according to the application. In this embodiment, a vertical cavity surface emitting laser VCSEL with a regular point array distribution is preferably used, and the emission wavelength is 940 nm. The laser emitter 111 is composed of a plurality of discrete light emitting holes 1110, and the number of light emitting holes n is greater than 500; while the number of light emitting holes of the VCSEL laser emitter used in the prior art 3D imaging product is much lower than 500. One of the features of the technical solution claimed in this application is that the light source has a large number of light emitting holes to generate a speckle projection pattern with high enough density to compensate for the replication function without using a DOE. Each light emitting hole emits a laser beam 1111 with a certain divergence angle, which collectively constitutes the first light beam 101.

[0062] In another preferred embodiment, the light emitting holes of the laser emitter 111 are arranged in a regular pattern, which can be horizontally and vertically arranged, as shown in Figure 4 (a), or staggered, as shown in Figure 4 (b). The staggered arrangement can be various, such as Figure 5 which shows that the rows are staggered by 1 / 3, 1 / 4, and the columns are staggered by 1 / 2, 1 / 5. The advantage of staggered arrangement is that it can increase the pitch of the light emitting holes, achieve higher heat dissipation, and improve the light emitting efficiency. The staggered arrangement in this application includes but is not limited to the above-mentioned arrangement, and any row stagger, column stagger, and row-column stagger after grouping can be used according to actual needs.

[0063] In another preferred embodiment, the light emitting holes can be controlled to emit laser light in groups, as shown in Figure 4 (b), where the light emitting holes 1151 are the first group and the light emitting holes 1152 are the second group. The number of groups can be set according to needs, such as three groups, four groups, five groups, etc. All light emitting holes can also be turned on together, i.e., only one group.

[0064] The ultra-short focal collimating lens 112 is composed of one lens in the schematic diagram, or can be composed of other number of lenses, which functions to collimate the light beams emitted by all light emitting points 1110 of the laser emitter 111, and expand the beams out with a large field angle a to form a collimated light beam 1112, and the collimated light beams corresponding to each light emitting hole collectively constitute the second light beam 102.

[0065] As shown in Figure 6As shown, the SPAD array receiving module 13 mainly includes a SPAD sensor 131, a narrow-band filter 132 and an imaging receiving lens 133, and the direction of light travel is indicated by an arrow 134. After the speckle light beam emitted by the speckle projection module 11 hits an object in the application scene, the reflected light beam 1113 reaches the SPAD sensor 131 through the imaging receiving lens 133 and the narrow-band filter 132.

[0066] In another preferred embodiment, the plurality of binning macro-pixel units corresponding to the speckles on the SPAD sensor each have a plurality of sub-SPAD pixels, and the binning macro-pixel units are consistent with the arrangement and grouping of the light-emitting holes. The imaging receiving lens 133 in this embodiment is composed of one lens, or can be composed of other numbers of lenses together, and its function is to receive all the speckle reflected light beams from the receiving screen 12 and accurately control each light beam to reach the corresponding binning macro-pixel unit on the SPAD sensor 131. Since the speckle projection module 11 does not produce optical distortion, the imaging receiving lens 133 does not need to consider distortion compensation, greatly reducing the design difficulty of the lens group and the system matching complexity between the emission module and the receiving module.

[0067] The function of the narrow-band filter 132 is to allow only the light of the corresponding wavelength emitted by the speckle projection module 11 to pass through, and to cut off other wavelengths of light, thereby achieving the effect of filtering other bands of light and reducing environmental light interference. The preferred wavelength transmittance curve is as shown in Figure 7 .

[0068] SPAD is the abbreviation of Single Photon Avalanche Diode, i.e. single photon avalanche diode, which works in Geiger mode APD (Avalanche Photon Diode), has super strong photosensitive capacity to single photon, and can make the detector have high sensitivity and gain by using avalanche multiplication effect, has great advantages in the application of weak light detection, high-speed imaging and other fields.

[0069] As shown in Figure 8 , the SPAD sensor 131 is composed of a plurality of SPAD pixel units 1311, and the gating control of each SPAD pixel can be realized through the X-direction addressing control circuit 1312 and the Y-direction addressing control circuit 1313, so that the corresponding SPAD pixel is in a working photosensitive state or a closed state. Usually, a larger reverse bias is applied to the corresponding pixel to make it in a critical avalanche state, which is equivalent to entering the "exposure" working state. The pixel units in the actual SPAD sensor do not have Figure 8The arrangement is so compact that due to the influence of complex internal integrated circuits and manufacturing processes, there will be gaps between each SPAD pixel unit 1311, and the filling rate can be obtained by the ratio of the effective light-sensitive pixel unit area to the total array area. Generally, the filling rate of the back-illuminated SPAD array sensor is higher than that of the front-illuminated SPAD array sensor, and thus the photon detection efficiency is higher. The preferred embodiment adopts a back-illuminated SPAD sensor.

[0070] The SPAD sensor usually further integrates quenching circuits, digital time converters TDCs, memories, digital processing logic, and the like, as shown in Figure 9 The cross-sectional view of the SPAD sensor is shown, mainly including a photon absorption layer 1314, a charge control layer 1315, and a multiplication layer 1316, which are connected to the underlying controller circuit, quenching circuit, analog front end, TDC, histogram memory, and other logic circuits through a wafer bonding process. Under the action of the controller, after the pixel unit enters the “exposure” state, the incident photons can be absorbed and converted to generate a multiplication avalanche, trigger an output photoelectric current, generate a voltage signal on the internal series resistance, and after being processed by the analog front end circuit, reach the high-precision timing circuit TDC to record the trigger time, that is, the time of flight Δt of the light to the SPAD pixel unit. At the same time, the quenching circuit is triggered to stop the avalanche effect of the multiplication layer in time and restore it to the initial reverse bias state, preparing for the next photon reception avalanche trigger.

[0071] Since the SPAD has the ability to respond to weak single photons, it has high sensitivity and gain through avalanche multiplication effect, and can greatly improve the imaging distance, ranging accuracy, and anti-environmental light interference ability of the system under the same light emission power. At the same time, the 3D imaging device uses the SPAD light-sensitive pixel to receive single photons to trigger the TDC timing system to obtain the TOF value, rather than using the average integration of light energy to calculate the phase difference to obtain the TOF value, so it does not have the problem of multipath interference, and the contours between multiple objects in the depth map are clear, and the quality is much higher than that of the I-TOF technical solution.

[0072] In another embodiment, a direct time-of-flight 3D imaging method based on speckle projection is provided, based on the 3D imaging device described in the foregoing embodiment, comprising:

[0073] The control processing module drives the speckle projection module to emit multiple light beams to form a spaced distribution of speckles on the target surface, and records the light emission time;

[0074] The SPAD array receiving module receives the speckle reflected light beams of the target surface, and records the light receiving time;

[0075] According to the light flight time obtained from the light emission time and the light receiving time, the depth distance information is processed and generated.

[0076] In this embodiment, the light-emitting holes on the light source are divided into three groups, which are sequentially lit, and the specific steps are as follows:

[0077] As shown in Figure 10 , the laser emitter 111 in the speckle projection module 11 adopts a vertical cavity surface emitting laser (VCSEL), and the arrangement of the light-emitting holes adopts a form of periodic misplacement of 1 / 3 between rows. All the VCSEL light-emitting holes are divided into three groups: the first group of light-emitting holes 1151, the second group of light-emitting holes 1152, and the third group of light-emitting holes 1153. The number of light-emitting holes in the figure is only illustrative, and the actual total number of light-emitting holes n>500.

[0078] First, under the first pulsed current driving of the control processing module 10, the first group of light-emitting holes 1151 of the laser emitter 11 is lit, as shown in Figure 11 , and each light beam forms a first speckle pattern 1251 on the front receiving screen 12 after passing through the ultra-short focal collimating lens 112, as shown by the black solid circle spots in the figure.

[0079] As shown in Figure 12 , at the same time when the first pulsed current starts to act, the corresponding first group of pixels 1351 of the SPAD sensor 131 will be turned on, so that the pixels in this group are in a light-sensing working state and can receive the sensing photons. The gray pixels in the figure represent this state. Other pixels are in an off state, which saves the chip power consumption. The moment when the first pulsed current starts to act is the starting time t1 of timing.

[0080] After the light beams emitted by the first group of light-emitting holes 1151 reach the receiving screen 12 to form the first speckle pattern 1251, the reflected light beams (partly diffuse reflection and specular reflection) will be collected by the imaging receiving lens of the SPAD array receiving module 13, and finally form the black solid focused light spot distribution shown in Figure 12 on the pixel surface of the SPAD sensor. Moreover, at the moment when the light beams reach the pixel surface, the SPAD pixels will be triggered to generate avalanche current, which will be detected and recorded by the TDC timing circuit to record the photon arrival time t2. Then, the corresponding light beam flight time Δt=t2-t1 can be obtained, and then the distance depth information of the corresponding object point of the light spot can be obtained according to d=Δt*c / 2.

[0081] In the embodiment, the projection module and the receiving module are strictly matched in optical design, so that each speckle point is accurately focused in the corresponding binning macro-pixel unit on the SPAD pixel surface, such as the speckle point 121 on the receiving screen 12 corresponding to the binning macro-pixel unit 1311 on the SPAD sensor pixel surface. In the embodiment, the macro-pixel unit is set to contain 3*3 sub-SPAD pixels, which is larger than the actual focused light spot range, to ensure that the imaging range of the light spot in various cases does not exceed the binning macro-pixel unit. The design size of the macro-pixel unit fully considers the following factors: 1) the baseline distance influence of the projection module and the receiving module; 2) the light spot imaging principle of near large and far small, taking into account different ranging ranges; 3) optical element processing error, assembly error, temperature drift. Each binning macro-pixel unit is provided with a TDC timing circuit.

[0082] After obtaining the distance depth information corresponding to the first speckle pattern 1251, the second group of light emitting holes 1152 of the laser emitter 11 is lit under the drive of the second pulse current, forming a second speckle pattern 1252 on the front receiving screen 12, as shown by the dotted line circle spot. Figure 12 At the same time when the second pulse current starts to act, the corresponding second group of pixels 1352 of the SPAD sensor 131 is opened, so that the group of pixels is in a photosensitive working state, and other pixels are in a closed state. The moment when the second pulse current starts to act is the starting time t3 of timing. After the light beams emitted by the second group of light emitting holes 1152 reach the receiving screen 12 to form the second speckle pattern 1252, the reflected light beams (part of the diffuse reflection and specular reflection) are collected by the imaging receiving lens of the SPAD array receiving module 13, and finally form the dotted line focused light spot distribution shown in Figure 12 When the light beams reach the pixel surface, the SPAD pixels are triggered to generate avalanche current, which is further detected and recorded by the TDC timing circuit to reach time t4. Then the corresponding light beam flight time Δt=t4-t3 is obtained, and then the distance depth information of the object point corresponding to the light spot can be obtained according to d=Δt*c / 2.

[0083] Similarly, the third group of light emitting holes 1153 correspond to the third pulse current drive, the third speckle pattern 1253 and the third group of pixels 1353, and finally the distance depth information corresponding to the third speckle pattern 1253 is obtained.

[0084] After the above steps, the speckle projection scanning of the entire picture is completed. According to the obtained three groups of distance depth information, the distance depth information of the entire picture can be obtained through fusion, that is, the direct time-of-flight 3D imaging based on speckle projection is completed.

[0085] Adopting this 3D imaging mode of grouping scanning speckle, the number requirement of TDC timing circuit, memory and other logic circuits of the SPAD sensor at the same time can be greatly reduced, the same set of TDC timing circuit, memory and other logic circuits can be reused in each group of scanning process, the integration complexity of the SPAD sensor is reduced, and the sensor area and pixel size are reduced.

[0086] In another embodiment, the emission hole arrangement of the laser emitter can also have other various forms, such as 1 / 2, 1 / 4, 1 / 5 row and row period misalignment arrangement, or column and column period misalignment arrangement, or regular arrangement without misalignment. The advantage of misalignment arrangement is that the spacing of the light-emitting holes can be increased to achieve higher heat dissipation and improve light-emitting efficiency. The grouping number and manner of the light-emitting holes can also be various, such as adjacent rows or adjacent columns forming a group, different local regions forming a group, etc., as long as the arrangement, grouping number and grouping manner of the binning macro-pixel of the SPAD sensor are consistent with the emission holes.

[0087] In another embodiment, the first pulse current, the second pulse current, the third pulse current and the Nth pulse current, wherein each pulse current can contain K (K>500) sub-pulse currents, and each sub-pulse current has a width of S (S<10ns), such as Figure 13 Each group of light-emitting holes is continuously lit K times, and each light-emitting time is S. At the same time when each sub-pulse current starts to act, the corresponding grouped pixels of the SPAD sensor are reset to be opened, so that the group of pixels is in a light-sensing working state. Therefore, in each large pulse current period, the SPAD is reset to be opened K times, and a flight time Δt is obtained each time. Due to the randomness of photon avalanche triggering, environmental light interference, device inherent noise interference and time jitter, the Δt value of each time is not the same. Finally, the histogram distribution of the flight time Δt about the counting number can be obtained, from which a more accurate flight time ΔT can be extracted to improve the ranging accuracy of the system.

[0088] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A direct time-of-flight 3D imaging device based on speckle projection, characterized in that, include: A speckle projection module is used to emit multiple beams of light and form a speckle pattern with intervals on the target surface. The speckle projection module includes a light source and an ultra-short focal length collimating lens. The number of light-emitting holes n>500 and emits a beam of light with diverging angles. The light-emitting holes are arranged in a regular pattern or staggered in rows and columns. The ultra-short focal length collimating lens expands the beam of light with diverging angles and emits it with an emission field of view α>20°. The focal length f of the ultra-short focal length collimating lens is less than 1.5mm. The SPAD array receiving module is used to receive the speckle reflected beam from the target surface and record the moment of light reception. The control processing module is used to control the speckle projection module to emit a light beam and record the light emission time, and to process and generate depth distance information based on the light flight time obtained from the light emission time and the light reception time.

2. The direct time-of-flight 3D imaging device based on speckle projection according to claim 1, characterized in that, The light source is a laser emitter, including a vertical cavity surface laser emitter or a surface array laser composed of single-point lasers.

3. The direct time-of-flight 3D imaging device based on speckle projection according to claim 1, characterized in that, The light-emitting holes are grouped and illuminated sequentially to emit light beams.

4. The direct time-of-flight 3D imaging device based on speckle projection according to claim 1, characterized in that, The SPAD array receiving module includes an imaging receiving lens, a narrowband filter, and a SPAD sensor arranged sequentially along the optical path.

5. The direct time-of-flight 3D imaging device based on speckle projection according to claim 1, characterized in that, The SPAD sensor includes multiple binning macropixel units corresponding to the speckle, each binning macropixel unit having multiple sub-SPAD pixels, and the arrangement and grouping of the binning macropixel units are consistent with those of the light-emitting apertures.

6. A direct time-of-flight 3D imaging method based on speckle projection, characterized in that, The speckle projection module includes a light source and an ultra-short focal length collimating lens, wherein the focal length of the ultra-short focal length collimating lens is f < 1.5 mm; the 3D imaging method includes: The control and processing module drives the light source of the speckle projection module to emit multiple beams. After each beam passes through the ultra-short focal length collimating lens, it forms a speckle pattern with intervals on the target surface and records the light emission time. The SPAD array receiving module receives the speckle reflected beam from the target surface and records the moment of light reception. Based on the time of flight of light obtained from the light emission and light reception times, depth and distance information are generated. The number of light-emitting holes in the light source is n>500. The light-emitting holes are divided into multiple groups. After being lit in sequence, a set of depth and distance information is obtained. When a set of depth and distance information is obtained, a set of pixels corresponding to the SPAD sensor of the SPAD array receiving module is turned on, so that the set of pixels is in a photosensitive working state and can receive photons. Other pixels are in a closed state. After multiple sets of depth and distance information are fused, complete distance and depth information is generated.

Images