ITOF ranging system and method to eliminate multipath error

Abstract

The application discloses an ITOF ranging system and method for eliminating multipath error, comprising: a transmitter for alternately transmitting a floodlight beam and a spot pattern light beam to a target scene; a receiver for receiving the floodlight beam reflected back by the target and generating a first electric signal, and receiving the spot pattern light beam reflected back by the target and generating a second electric signal; a control and processor for processing the first electric signal to generate a first Rawphase image, processing the second electric signal to generate a second Rawphase image, and processing the first Rawphase image and the second Rawphase image to obtain a dense depth image eliminating multipath interference. In the application, different modes of probe light beams are used to quantitatively determine the deviation amount caused by the multipath error, and the deviation amount of the multipath error is eliminated to obtain a more accurate dense depth image, thereby improving the accuracy of the ITOF ranging system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical technology, in particular to an ITOF ranging system for eliminating multipath error and a method for eliminating multipath error. BACKGROUND

[0002] Time of flight (TOF) is a method of depth measurement. It is widely used because of its simple principle, small size, large measurement distance range and strong anti-interference ability. According to different ranging methods, there are two TOF technology routes: indirect time of flight (ITOF) and direct time of flight (DTOF). DTOF directly emits light pulses to the measured object and measures the time interval between the reflected light pulse and the emitted light pulse to obtain the flight time of the light, thereby directly calculating the depth of the measured object. ITOF measures the flight time by emitting modulated light of a specific frequency and detecting the phase difference between the reflected modulated light and the emitted modulated light. Because the pixels of the ITOF sensor are relatively small, higher image resolution can be achieved.

[0003] In the prior art, a floodlight ITOF ranging system projects uniform illumination light to a target object. Although dense depth information of the measured target object can be obtained, it is easily affected by multipath interference in the measurement environment. For example, when measuring the corners, part of the light signal is reflected to the pixel unit, and another part of the light signal is reflected to other target points and then reflected or multiple reflected to the pixel unit, so that the light signal collected by the pixel unit contains interference signals reflected by other target points, which is called multipath interference phenomenon, reducing the accuracy of the ITOF ranging system.

[0004] Therefore, the prior art still needs to be improved and developed in view of the above defects. SUMMARY

[0005] The main purpose of the present application is to provide an ITOF ranging system and method for eliminating multipath error to solve the technical problems in the prior art.

[0006] To achieve the above object, the first aspect of the present application provides an ITOF ranging system for eliminating multipath error, comprising: a transmitter for alternately transmitting a floodlight beam and a speckle pattern light beam to a target scene; a receiver for receiving the floodlight beam reflected back by the target and generating a first electrical signal, and receiving the speckle pattern light beam reflected back by the target and generating a second electrical signal; a control and processor for processing the first electrical signal to generate a first Rawphase map, processing the second electrical signal to generate a second Rawphase map, and processing the first Rawphase map and the second Rawphase map to obtain a dense depth map eliminating multipath interference.

[0007] In some embodiments, the control and processor generates a sparse error Rawphase map according to the first Rawphase map and the second Rawphase map, interpolates the sparse error Rawphase map to obtain a dense error Rawphase map, and corrects the first Rawphase map using the dense error Rawphase map to obtain the dense depth map eliminating multipath interference.

[0008] In some embodiments, the control and processor processes the first Rawphase map and the second Rawphase map to generate a first depth map and a second depth map, obtains a sparse error depth map according to the first depth map and the second depth map, interpolates the sparse error depth map to obtain a dense error depth map, and corrects the first depth map using the dense error depth map to obtain a dense depth map eliminating multipath interference.

[0009] In some embodiments, the transmitter comprises: a floodlight illumination unit and a speckle illumination unit; the floodlight illumination unit is used to emit a floodlight beam towards the target scene; the speckle illumination unit is used to emit a speckle pattern light beam towards the target scene. Alternatively, the transmitter comprises a light source array and a liquid crystal element; the light beam emitted by the light source array forms a speckle pattern light beam after passing through the liquid crystal element in a transparent state; the light beam emitted by the light source array forms a floodlight beam after passing through the liquid crystal element in a diffusion state. Alternatively, the transmitter comprises a first light source array, a second light source array, a collimating mirror and a diffractive optical element; the distance between the first light source array and the collimating mirror is equal to the focal length of the collimating mirror; the distance between the second light source array and the collimating mirror is not equal to the focal length of the collimating mirror.

[0010] In some embodiments, the ITOF ranging system further comprises a floodlight illumination mode and a speckle working mode; when in the floodlight illumination mode, the transmitter is used to emit a floodlight beam to the target scene; when in the speckle working mode, the transmitter emits the speckle pattern light beam to the target scene.

[0011] The application also provides an ITOF ranging method for eliminating multipath error, characterized in that the method comprises: alternately emitting a floodlight beam and a speckle pattern light beam towards a target scene; receiving part of the floodlight beam and part of the speckle pattern light beam reflected by the target and outputting a first electric signal and a second electric signal; generating a first Rawphase map and a second Rawphase map according to the first electric signal and the second electric signal respectively, and processing the first Rawphase map and the second Rawphase map to obtain a dense depth map eliminating multipath interference.

[0012] In some embodiments, processing the first Rawphase map and the second Rawphase map to obtain the dense depth map eliminating multipath interference comprises: generating a sparse error Rawphase map according to the first Rawphase map and the second Rawphase map, interpolating the sparse error Rawphase map to obtain a dense error Rawphase map, and correcting the first Rawphase map by using the dense error Rawphase map to obtain the dense depth map eliminating multipath interference.

[0013] In some embodiments, processing the first Rawphase map and the second Rawphase map to obtain the dense depth map eliminating multipath interference comprises: processing the first Rawphase map and the second Rawphase to obtain a first depth map and a second depth map; obtaining a sparse error depth map according to the first depth map and the second depth map, interpolating the sparse error depth map to obtain a dense error depth map, and correcting the first depth map by using the dense error depth map to obtain the dense depth map eliminating multipath interference.

[0014] As can be seen from the above, in the application, the emitter alternately emits a floodlight beam and a speckle pattern light beam towards a target scene; the receiver receives the floodlight beam reflected by the target and generates a first electric signal, and receives the speckle pattern light beam reflected by the target and generates a second electric signal; the control and processor processes the first electric signal to generate a first Rawphase map, processes the second electric signal to generate a second Rawphase map, and processes the first Rawphase map and the second Rawphase map to obtain a dense depth map eliminating multipath interference. In the application, different modes of probe light beams are used to quantitatively determine the deviation amount caused by multipath error, and the deviation amount of multipath error is eliminated to obtain a more accurate dense depth image, thereby improving the accuracy of the ITOF ranging system. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0016] Figure 1 is a structural schematic diagram of a preferred embodiment of the ITOF ranging system of the present application;

[0017] Figure 2 is a light path schematic diagram of a preferred embodiment of the ITOF ranging system of the present application;

[0018] Figure 3 is a schematic diagram of the image projected by the floodlight emitting unit provided by the embodiments of the present application;

[0019] Figure 4 is a schematic diagram of the image projected by the speckle emitting unit provided by the embodiments of the present application;

[0020] Figure 5 is a flow schematic diagram of a preferred embodiment of the ITOF ranging method for eliminating multipath error of the present application. DETAILED DESCRIPTION

[0021] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0022] It should be understood that the term "comprising" as used in the specification and the appended claims indicates the presence of the recited features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0023] It should also be understood that the terms used in the present application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clearly indicated by the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0024] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0027] In existing technologies, ITOF ranging systems acquire depth information of target objects by projecting a uniform floodlight beam onto the target scene. However, during detection, they are susceptible to multipath interference in the measurement environment. For example, when measuring corners, some light signals may be reflected back to the pixel unit, while other light signals may be reflected again or multiple times to other target points before entering the pixel unit. This results in the pixel unit acquiring interference signals reflected back from other target points, a phenomenon known as multipath interference, which reduces the measurement accuracy of the ITOF ranging system. While ITOF ranging systems that project a speckled array beam are less susceptible to multipath interference, they can only measure a sparse depth image of the target scene, resulting in lower detection resolution.

[0028] like Figure 1 As shown, an ITOF ranging system is proposed, including: a transmitter 10, a receiver 20, and a control and processor 30 connected to the transmitter and the receiver. The transmitter 10 is used to continuously emit a time-modulated beam of light into the target scene. At least a portion of the emitted beam is reflected by a target point in the target area to form a reflected beam. At least a portion of the reflected beam is received by the receiver and generates an electrical signal. The control and processor 30 synchronizes the trigger signals of the transmitter 10 and the receiver 20, and processes the received electrical signal to calculate the flight time of the reflected beam relative to the emitted beam 30. Furthermore, the depth information of the target is calculated based on the flight time.

[0029] The transmitter 10 includes a light source and a driver. The light source can be a single light source such as a light-emitting diode (LED), an edge-emitting laser (EEL), or a vertical-cavity surface-emitting laser (VCSEL), or it can be a VCSEL array light source chip formed by generating multiple VCSEL light sources on a single semiconductor substrate. The driver drives the light source to emit a laser beam toward a target region with a specific timing amplitude modulation. In some embodiments, the transmitter 10 may further include emitting optical elements disposed in the transmission optical path of the emitted beam to collimate, expand, diffract, or modulate the emitted beam before emitting it outward.

[0030] In one embodiment, such as Figure 2 As shown, the emitter 10 includes a floodlight illumination unit 101 and a spot illumination unit 102. The floodlight illumination unit 101 emits a floodlight beam toward a target area; the spot illumination unit 102 emits a spot pattern beam toward the target area. The floodlight illumination unit 101 includes a single point light source for emitting a floodlight beam toward the target area. To improve the field of view and light energy uniformity of the emitted beam, it may also include a diffusion element disposed on the outgoing optical path of the emitted beam to modulate the emitted beam. Alternatively, the floodlight illumination unit 101 may also include a light source array and a diffusion element. The spot beam emitted by the light source array is modulated by the diffusion element to form a uniform floodlight beam projected onto the target area. The spot illumination unit 102 includes a light source array and an emitting optical element. The light source array emits a spot pattern beam, which is expanded by the emitting optical element to form a higher density spot pattern beam projected onto the target area. The emitting optical element may be a diffractive optical element, a microlens array, etc. In this embodiment, the illumination light emitted by the transmitter can be infrared light in the 850nm or 940nm band, or visible light in other bands; no limitation is made here.

[0031] Those skilled in the art will understand that Figure 2 The block diagram shown is only a partial diagram of the structure related to the solution of this application, and does not constitute a limitation on the ITOF ranging system on which the solution of this application is applied. In this embodiment, as shown... Figure 2 As shown, the floodlight unit 101 and the spotlight unit 102 are arranged symmetrically with respect to the receiver 20. The arrangement of the floodlight unit 101 and the spotlight unit 102 can also be adjusted according to actual needs, but it is necessary to ensure that the emitted light can cover the field of view area of ​​the receiver 20 and / or the area where the target object is located. Figure 3 As shown, Figure 3 This is a schematic diagram of the projected image from the floodlight unit 101 provided in this embodiment of the application. In practical applications of ITOF ranging systems, the intensity of the uniform illumination beam is modulated over time according to a preset period. Figure 4As shown, Figure 4 is a schematic diagram of the spot illumination unit 102 projecting an image. In the actual application of the ITOF ranging system, the intensity of the speckle array light beam is modulated according to a preset period in time, wherein the speckle array light beam emitted by the spot illumination unit 102 can be set and adjusted according to actual needs, so that the speckle array light beam can be regularly arranged or randomly arranged, which is not limited here.

[0032] The flood illumination unit 101 and the spot illumination unit 102 in the present application can be two independent emitters, or can be integrated in the same emitter through some special design to realize the switching of the flood light beam and the spot pattern light beam. In an embodiment, the emitter 10 includes a light source array and a liquid crystal, and the light source array is preferably an array VCSEL chip composed of a plurality of VCSEL light sources arranged on a semiconductor substrate. The liquid crystal element has a transparent state and a diffusion state, and is in the transparent state when turned on and in the diffusion state when turned off. The spot pattern light beam emitted by the VCSEL array light source is modulated by the liquid crystal element in the diffusion state to form a flood light beam, and is modulated by the liquid crystal element in the transparent state to form a spot pattern light beam. In this way, the adjustment of the emitted light beam can be realized by converting the two states of the liquid crystal element. In some specific embodiments, the light source array can be regular or irregular. The light source array can be configured to include a first sub-light source array and a second sub-light source array, and the optical aperture of the first sub-light source in the first sub-light source array is smaller than the optical aperture of the second sub-light source in the second sub-light source array. The first sub-light source has a small optical aperture, and can emit a first light beam with a large divergence angle, which is modulated by the liquid crystal element in the diffusion state to form a flood light beam. The second light source has a large optical aperture, and can emit a first light beam with a small divergence angle, which is modulated by the liquid crystal element in the transparent state to form a spot pattern light beam.

[0033] In one embodiment, the transmitter 10 comprises a first light source array, a second light source array, a collimating mirror and a diffractive optical element. The first light source array can be a regular light source array or an irregular light source array, while the second light source array is a regular light source array. The first light source array is located on the object focal plane of the collimating mirror, i.e. the distance between the first light source array and the collimating mirror is equal to the focal length of the collimating mirror. The plurality of spot beams emitted by the first light source array are collimated by the collimating mirror and then incident on the diffractive optical element. The diffractive optical element replicates the collimated spot beams to form a spot pattern beam which is projected onto the target area. The second light source array is not located on the object focal plane of the collimating mirror, i.e. the distance between the second light source array and the collimating mirror is not equal to the focal length of the collimating mirror, which can be greater than or less than the focal length of the collimating mirror. The plurality of spot beams emitted by the second light source array are collimated by the collimating mirror to form discrete spots, which are replicated by the diffractive optical element to form a floodlight beam which is projected onto the target area. In another embodiment, the collimating mirror can be a zoom lens. The first light source array and the second light source array are arranged on the same circuit substrate. When a spot pattern beam needs to be projected, the focal length of the collimating mirror is adjusted to be equal to the distance between the first light source array and the collimating mirror, and the first light source array is controlled to emit light for projecting the spot pattern beam. When a floodlight beam needs to be projected, the focal length of the collimating mirror is adjusted to be not equal to the distance between the first light source array and the collimating mirror, and the second light source array is controlled to emit light for projecting the floodlight beam.

[0034] The receiver 20 comprises an image sensor, a filtering unit and a lens unit. Specifically, the lens unit is configured to receive at least part of the reflected light beams reflected by the target in the scene and image them on at least part of the image sensor; the filtering unit is configured as a narrowband filter matching the wavelength of the light source, which is configured to suppress the background light noise in the remaining waveband. The image sensor comprises at least one acquisition unit, which comprises at least two taps, each of which is configured to store and read or discharge the charge signal generated by the acquisition of the reflected light beams (or ambient light) under the control of the corresponding electrode. In actual applications, the image sensor can switch the taps in a certain order to acquire the corresponding reflected light beams in a single frame period (or a single exposure time), and convert the acquired reflected light beams into corresponding electrical signals, so that the image sensor can output a Rawphase image in each frame period. The pixel value of each pixel in the Rawphase image is the amount of charge accumulated by each tap due to the acquisition of the reflected light beams (or ambient light), and the Rawphase image also contains the phase difference information of the reflected light beams relative to the emitted light beams. At this time, the image sensor outputs a Rawphase image in each frame period T, and then the control and processing unit 30 can calculate the depth information of the target according to the Rawphase image output by the image sensor.

[0035] Further, the control and processing unit further comprises a working mode control unit for controlling the working mode of the emitter 10, the emitter 10 executes the flood working mode, the speckle working mode or the mixed working mode under the control of the working mode control unit. When executing the flood working mode, such as detecting the target at the near distance or obtaining the depth image with higher resolution, the control and processing unit controls the emitter 10 to emit the flood light beam, for example, the working mode control unit controls the flood illumination unit 101 to be turned on and the speckle illumination unit 102 to be turned off; the flood illumination unit 101 emits the uniform flood light beam towards the target object, the receiver 20 receives the reflected signal reflected back by the target object, and the control and processing unit 30 calculates the first depth image according to the reflected signal received by the receiver 20 through the ITOF measurement principle. When executing the speckle working mode, such as detecting the target at the far distance or obtaining the depth image with higher precision, the control and processing unit controls the emitter 10 to emit the speckle pattern light beam, for example, the working mode control unit controls the speckle illumination unit 102 to be turned on and the flood illumination unit 101 to be turned off; the speckle illumination unit 102 emits the speckle pattern light beam towards the target object, the receiver 20 receives the reflected signal reflected back by the target object, and the control and processing unit 30 calculates the second depth image according to the reflected signal received by the receiver 20 through the ITOF measurement principle. In most detection applications, both the detection precision and the resolution need to be improved, at this time, if the flood illumination mode is executed, the detection precision is reduced due to the influence of the multipath interference in the measurement environment, and then the mixed working mode is executed to eliminate the multipath interference problem.

[0036] In one embodiment, when the control and processing unit controls the emitter 10 to execute the mixed working mode, that is, the control and processing unit controls the emitter 10 to continuously and alternately emit the flood light beam and the speckle pattern light beam, for example, the control and processing unit controls the flood illumination unit 101 and the speckle illumination unit 102 to be alternately turned on. In a specific implementation, the control and processing unit first controls the flood illumination unit 101 to be turned on and the speckle illumination unit 102 to be turned off to obtain the first Rawphase image; then controls the speckle illumination unit 102 to be turned on and the flood illumination unit 101 to be turned off to obtain the second Rawphase image; and the control and processing unit is further used to obtain the dense depth image eliminating the multipath interference according to the first Rawphase image and the second Rawphase image. Specifically, the first Rawphase image and the second Rawphase image are subtracted to obtain the sparse error Rawphase image, the control and processing unit performs interpolation on the sparse error Rawphase image based on the assumption of the spatial smoothness of the multipath interference to obtain the dense error Rawphase image, and the dense error Rawphase image is used to correct the multipath interference error in the first Rawphase image to obtain the Rawphase image with higher precision, so as to calculate the dense depth image with high precision according to the Rawphase.

[0037] In yet another embodiment, the control and processing processor processes the first Rawphase image and the second Rawphase image to generate a first depth image and a second depth image, and then subtracts the first depth image from the second depth image to obtain a sparse error depth image. In this embodiment, subtracting the first depth image from the second depth image can reduce the influence of noise. In addition, the control and processing processor interpolates the sparse error depth image based on the assumption of spatial smoothness of multipath interference to obtain a dense error depth image, that is, processing is performed on the difference image of the first depth image and the second depth image, and the signal-to-noise ratio of the difference is higher. Then, the control and processing processor subtracts the first depth image from the dense error depth image to obtain a dense depth image that eliminates multipath interference, thereby improving the accuracy of the ITOF ranging system.

[0038] Since the multipath interference is mainly diffuse scattering, the light signal will be incident in a certain area of the space after being reflected by multiple target points, which will interfere with multiple pixels, causing the pixels to receive interference signals after receiving the reflected light signal. By subtracting the first Rawphase image / first depth image from the second Rawphase image / second depth image, the interference signal collected by a certain pixel can be determined. Then, based on the assumption of spatial smoothness of multipath interference, that is, the interference signal amounts collected by adjacent pixels are close, the interference signal amount collected by each pixel can be determined, thereby eliminating the interference signal amount collected by the pixel to obtain the effective signal amount collected by the pixel. Further, a more accurate detection result can be obtained according to the effective signal amount.

[0039] In this embodiment, when the control transmitter 10 performs the hybrid working mode, the sequence and frame rate at which the flood illumination unit 101 and the spot illumination unit 102 are turned on can be adjusted according to actual application, which is not limited here.

[0040] As shown in Figure 5 The application also provides an ITOF ranging method for eliminating multipath error. The ITOF ranging method for eliminating multipath error includes the following steps:

[0041] Step S501: Alternately emitting a flood light beam and a speckle pattern light beam towards a target scene;

[0042] The control transmitter 10 alternately emits a flood light beam and a speckle pattern light beam towards a target scene, that is, the control transmitter 10 performs a hybrid working mode. Details can be seen from the foregoing description.

[0043] Step S502: Receiving part of the flood light beam and part of the speckle pattern light beam reflected by the target and outputting a first electric signal and a second electric signal;

[0044] Specifically, the control receiver 20 receives the reflected signal reflected by the target object and generates an electrical signal, when the floodlight beam is emitted, the receiver receives the reflected floodlight beam to generate a first electrical signal; when the speckle pattern beam is emitted, the receiver receives the reflected speckle pattern beam to generate a second electrical signal.

[0045] In step S503, a first Rawphase image and a second Rawphase image are respectively generated according to the first electrical signal and the second electrical signal, and the first Rawphase image and the second Rawphase image are processed to obtain a dense depth image eliminating multipath interference.

[0046] In one embodiment, the control and processor 30 receive the first electrical signal to process to obtain the first Rawphase image, and receive the second electrical signal to process to obtain the second Rawphase image, the specific processing process can be seen from the foregoing description, which will not be repeated here. Among them, the resolution in the first Rawphase image is greater than that in the second Rawphase image, and the accuracy of the first Rawphase image is less than that of the second Rawphase image, because the first Rawphase is detected by emitting a floodlight beam, the light intensity distribution of the floodlight beam is uniform when it irradiates to the target scene, and the reflected light beam will also be scattered to different pixels to cause crosstalk, or irradiate to multiple target points in the target scene and then continuously reflect to the pixels. These interference signals are collectively referred to as multipath interference. The second Rawphase image does not include interference signals. In this application, the amount of interference caused by the multipath interference signal in the detection process is determined by the second Rawphase image, the existence of the interference amount is eliminated to correct the accumulated charge amount, and finally a high-precision, high-resolution depth image is obtained. First, the first Rawphase image and the second Rawphase image are collected by the same image sensor, so the pixels have a one-to-one correspondence, and the only difference is that the resolutions of the two are different, that is, part of the pixels in the second Rawphase image have no pixel value. For the pixels with pixel values in the second Rawphase image, the corresponding pixels in the first Rawphase image are found and subtracted, to obtain a sparse error Rawphase image. The sparse error Rawphase image is used to represent the deviation caused by the multipath interference. According to the spatial smoothness assumption of the multipath interference error, the sparse error Rawphase is interpolated to obtain a dense error Rawphase image. The interpolation method can use many methods such as adjacent interpolation, bilinear interpolation, bilinear interpolation, etc. In this application, it is not specifically limited. Further, the first Rawphase image is subtracted from the dense error Rawphase image to obtain a dense Rawphase image that does not contain interference signals. The dense Rawphase image is used to calculate a dense depth image.

[0047] In yet another embodiment, the control and processing unit 30 receives the first Rawphase image for processing to obtain the first depth image and receives the second Rawphase image for processing to obtain the second depth image, the processing of which is described above and will not be repeated here. The resolution of the first depth image is greater than that of the second depth image, and the accuracy of the second depth image is greater than that of the first depth image. This is because the first depth image is obtained by detecting the floodlight beam, and the light intensity is uniform when the floodlight beam is irradiated to the target scene. However, the reflected light beam will be scattered and incident to different pixels, causing crosstalk, or the light beam will be continuously reflected after being irradiated to multiple target points in the target scene and then incident to the pixels. These interference signals are collectively referred to as multi-path interference, which reduces the accuracy of the floodlight beam detection. In contrast, the speckle pattern beam has concentrated beam energy and a small market angle for each spot, and the multi-path interference is less likely to occur, so the detection accuracy is higher. After obtaining the first depth image and the second depth image, the first depth image and the second depth image are subtracted to obtain a sparse error depth image. Since the first depth image and the second depth image are both generated by collecting light signals from the same image sensor, the pixels in the depth images have a one-to-one correspondence. The difference is that some pixels in the second depth image have no depth value, while the pixel values in the first depth image are inaccurate depth values. Therefore, for each second pixel with a depth value in the second depth image, a difference value is obtained by subtracting the first pixel in the first depth image corresponding to the second pixel. Finally, a sparse error depth image is obtained, and the resolution of the sparse error depth image is approximately equal to that of the second depth image. The sparse error depth image is used to represent the deviation caused by multi-path interference. Since multi-path interference is mainly diffuse scattering, the depth error caused by multi-path interference has spatial smoothness. Therefore, the sparse multi-path error depth image is interpolated to obtain a dense error depth image. The interpolation method can be a variety of methods such as nearest neighbor interpolation, bilinear interpolation, and bilinear interpolation, which are not specifically limited in the present application. Further, the dense error depth image represents the depth error deviation caused by multi-path interference. The first depth image and the dense error depth image are subtracted to correct the depth values in the first depth image to obtain a dense depth image that eliminates multi-path interference.

[0048] In the embodiment, the first depth image and the second depth image are subtracted to obtain a sparse error depth image containing a multi-path interference signal, and based on the assumption of spatial smoothness of the multi-path interference, the sparse error depth image is interpolated to obtain a dense error depth image. The prior art usually subtracts the first depth image from the combined depth image to obtain the second depth image, finds the unlit area in the second depth image according to a preset customization, and interpolates the unlit area in the second depth image to extract the multi-path interference. However, since the unlit area itself has a low signal-to-noise ratio, the measurement result in the prior art is easily affected by noise, and repeated calculation of multiple groups of difference image frames is required, resulting in increased processing time and reduced efficiency. In the embodiment, the first depth image and the second depth image are subtracted, and processing is performed on the difference value image of the first depth image and the second depth image. The difference value has a higher signal-to-noise ratio, and the assumption of sparsity of the multi-path interference is introduced, so that repeated calculation or processing of the image is not required.

[0049] As can be seen from the above, in the ITOF ranging method for eliminating multi-path error, two different modes of probe light beams are projected to obtain an error amount caused by multi-path interference, so that the influence of multi-path interference can be eliminated, and the accuracy of the ITOF ranging system is improved.

[0050] It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional units and modules is exemplified, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above integrated unit can be realized in the form of hardware or software. In addition, the specific names of the functional units and modules are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, which will not be described here.

[0052] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0053] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0054] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand; it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements, which do not deviate from the spirit and scope of the technical solutions of the embodiments of the present application, should be included in the protection scope of the present application.

Claims

1. An ITOF ranging system that eliminates multipath errors, characterized in that, The method comprises: a transmitter for alternately transmitting a floodlight beam and a speckle pattern light beam to a target scene; a receiver for receiving the floodlight beam reflected back by the target and generating a first electrical signal, and receiving the speckle pattern light beam reflected back by the target and generating a second electrical signal; a control and processor for processing the first electrical signal to generate a first Rawphase image, processing the second electrical signal to generate a second Rawphase image, and processing the first Rawphase image and the second Rawphase image to obtain a dense depth image with multipath interference eliminated; the processing of the first Rawphase image and the second Rawphase image to obtain the dense depth image with multipath interference eliminated comprises: the control and processor generate a sparse error Rawphase image according to the first Rawphase image and the second Rawphase image, interpolate the sparse error Rawphase image to obtain a dense error Rawphase image, and correct the first Rawphase image using the dense error Rawphase image to obtain the dense depth image with multipath interference eliminated; or the control and processor process the first Rawphase image and the second Rawphase image to generate a first depth image and a second depth image, obtain a sparse error depth image according to the first depth image and the second depth image, interpolate the sparse error depth image to obtain a dense error depth image, and correct the first depth image using the dense error depth image to obtain the dense depth image with multipath interference eliminated.

2. The ITOF ranging system of claim 1, wherein, The transmitter comprises a floodlight illumination unit and a speckle illumination unit; the floodlight illumination unit is used for transmitting the floodlight beam towards the target scene; and the speckle illumination unit is used for transmitting the speckle pattern light beam towards the target scene.

3. The ITOF ranging system of claim 1, wherein, The transmitter comprises a light source array and a liquid crystal element; the light beam emitted by the light source array forms the speckle pattern light beam after passing through the liquid crystal element in a transparent state; and the light beam emitted by the light source array forms the floodlight beam after passing through the liquid crystal element in a diffusion state.

4. The ITOF ranging system of claim 1, wherein, The transmitter comprises a first light source array, a second light source array, a collimating mirror and a diffractive optical element; the distance between the first light source array and the collimating mirror is equal to the focal length of the collimating mirror; and the distance between the second light source array and the collimating mirror is not equal to the focal length of the collimating mirror.

5. The ITOF ranging system of claim 1, wherein, The ITOF ranging system further comprises a floodlight illumination mode and a speckle working mode; when in the floodlight illumination mode, the transmitter is used for transmitting the floodlight beam towards the target scene; and when in the speckle working mode, the transmitter transmits the speckle pattern light beam towards the target scene.

6. An ITOF ranging method that eliminates multipath errors, characterized by, The method for eliminating multipath error of ITOF ranging is applied to the ITOF ranging system for eliminating multipath error according to any one of claims 1-5, and the method comprises: alternately transmitting a floodlight beam and a speckle pattern light beam towards a target scene; receiving part of the floodlight beam and part of the speckle pattern light beam reflected by the target and outputting a first electrical signal and a second electrical signal; generating a first Rawphase image and a second Rawphase image according to the first electrical signal and the second electrical signal respectively, processing the first Rawphase image and the second Rawphase image to obtain a dense depth image eliminating multipath interference; processing the first Rawphase image and the second Rawphase image to obtain a dense depth image eliminating multipath interference, comprising: generating a sparse error Rawphase image according to the first Rawphase image and the second Rawphase image, interpolating the sparse error Rawphase image to obtain a dense error Rawphase image, and correcting the first Rawphase image by using the dense error Rawphase image to obtain the dense depth image eliminating multipath interference; or, processing the first Rawphase image and the second Rawphase image to obtain a dense depth image eliminating multipath interference, comprising: processing the first Rawphase image and the second Rawphase image to obtain a first depth image and a second depth image, generating a sparse error depth image according to the first depth image and the second depth image, interpolating the sparse error depth image to obtain a dense error depth image, and correcting the first depth image by using the dense error depth image to obtain the dense depth image eliminating multipath interference.

Images