TOF depth camera based on dot light projection

Abstract

The present invention provides a time-of-flight (TOF) depth camera based on dot-matrix light projection, comprising: a structured light projector for projecting structured light onto a target; an infrared camera for collecting infrared speckle images and multiple infrared reference images; and a processor module for generating a speckle depth map based on the phase difference of multiple frames of infrared speckle images. The speckle depth map includes the infrared speckle image and depth data. The corresponding infrared reference image is determined based on the depth data. A reference region corresponding to the speckle in the infrared reference image is determined based on the pixel coordinates of the speckle. Each subregion in the reference region is matched with the corresponding speckle region to determine the target subregion with the highest correlation. The center offset between the target subregion and the reference region is determined. When the offset is greater than a preset threshold, the speckle region is determined to be a multipath interference point. The present invention can detect mirror multipath and eliminate multipath interference, enabling the application of a TOF depth camera based on dot-matrix light projection on a sweeping robot.

Description

Technical Field

[0001] The present invention relates to a depth camera, and in particular to a TOF depth camera based on dot matrix light projection. Background Art

[0002] Time-of-flight (TOF) depth cameras emit a floodlight beam in a specific wavelength band, then use a sensor to receive the reflected beam from objects in the measured space and measure the beam's flight time in space to calculate distance, thereby acquiring a depth image of the measured space. TOF depth cameras can simultaneously obtain both grayscale and depth images and are widely used in 3D depth vision-related technologies such as gesture recognition, face recognition, 3D modeling, somatosensory gaming, machine vision, assisted focus, security, and autonomous driving.

[0003] Traditional TOF depth cameras assume that the received light beam is reflected only once in the target scene. However, in actual scenes, there are always mirror-reflective or diffuse-reflective material surfaces that reflect the incident light in all directions. In this way, the TOF sensor may receive a superposition of a single reflected beam and multiple reflected beams, thereby interfering with the accuracy of the TOF depth camera's distance measurement. This effect is called multipath interference.

[0004] Traditional TOF depth cameras typically consist of a light projector and a light receiving sensor. The light projector emits a flood beam into space to provide illumination, while the light receiving sensor receives the reflected flood beam for imaging. The depth calculator calculates the time of flight by using the phase delay between the emitted and received light, thereby obtaining distance information. This approach to depth measurement has some limitations, such as interference from ambient light that can affect measurement accuracy. This is especially true when the ambient light intensity is higher than the intensity of the directly reflected light, resulting in the light receiving sensor receiving a signal dominated by ambient light. This is typically the case when there are multiple mirror images.

[0005] Specular multipath is a common problem in robot vacuums. The ground is made of highly reflective tiles. The signal received by the corresponding pixel on the ground is a combination of direct reflection (primary path) and light signals from multiple reflections from objects (secondary paths). For highly reflective objects, the intensity of the primary path is much lower than that of the secondary path, causing the measured ground depth to deviate from the actual depth. Summary of the Invention

[0006] In view of the defects in the prior art, an object of the present invention is to provide a TOF depth camera based on dot matrix light projection.

[0007] According to the TOF depth camera based on dot matrix light projection provided by the present invention, 1. A TOF depth camera based on dot matrix light projection is characterized by comprising the following modules:

[0008] A structured light projector, used to project structured light toward a target;

[0009] an infrared camera, configured to receive structured light reflected by the target and generate an infrared speckle image and a plurality of infrared reference images, wherein the infrared reference images are generated by capturing a reference target at a plurality of different distances using a depth camera;

[0010] A processor module is configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image.

[0011] Preferably, when determining the reference area corresponding to the scattered speckles in the infrared reference image, the following steps are included:

[0012] Step M1: Acquire depth data of each scattered speckle, and determine the distance between the target surface point corresponding to the scattered speckle and the depth camera according to the depth data;

[0013] Step M2: determining an infrared reference image collected at the same distance according to the distance;

[0014] Step M3: determining a reference area corresponding to the scattered speckle in the corresponding infrared reference image according to the pixel coordinates of the scattered speckle.

[0015] Preferably, when determining that the scattered speckle is a multipath interference point, the following steps are included:

[0016] Step N1: Calculating the correlation between the scattered speckle area and a sub-area at the upper left corner of the corresponding reference area;

[0017] Step N2: moving the sub-region by one pixel each time from left to right and from top to bottom, and calculating the correlation between the sub-region and the speckle region until the target sub-region with the highest correlation is determined;

[0018] Step N3: determining a center offset between the target sub-region and the scattered speckle region, and when the offset is greater than a preset threshold, determining that the scattered speckle is a multipath interference point.

[0019] Preferably, the correlation r is calculated as follows:

[0020]

[0021] Among them, A mn is the amplitude of the reference area, is the average amplitude of the reference area; B mn is the amplitude of the speckle area, is the average amplitude of the speckle area; m and n are the pixel coordinate ranges.

[0022] Preferably, the infrared reference image acquisition includes the following steps:

[0023] Step S101: Projecting dot matrix light toward the reference target at a distance through the structured light projector;

[0024] Step S102: generating an infrared reference image by receiving the dot matrix light reflected by the reference target through the infrared camera;

[0025] Step S103: Repeat steps S101 to S102 to obtain infrared reference images at multiple different distances.

[0026] Preferably, the structured light projector includes a light source, a light source driver, and a light modulator;

[0027] The light source driver is connected to the light source and is used to drive the light source to emit light;

[0028] The light modulator is used to modulate the light projected by the light source into structured light and then project the structured light toward a target.

[0029] Preferably, the infrared camera includes a lens, a filter, and an image sensor arranged along the optical path, and the image sensor is provided with at least four receiving windows; the pulse width of the receiving window is greater than or less than the pulse width of the structured light;

[0030] The image sensor is configured to receive at least four optical signals of the structured light through at least four receiving windows; the at least four receiving windows are sequentially arranged in time sequence, and each speckle depth map is generated according to the optical signal received by each receiving window.

[0031] Preferably, the structured light is a dot matrix light; the dot matrix light is distributed in the following preset shapes: linear, triangular, quadrilateral, circular, hexagonal, pentagonal, random arrangement, spatially coded arrangement and quasi-lattice arrangement.

[0032] Preferably, the size of the scattered speckle area can be set to a 5×5 pixel area, and the scattered speckle area includes a scattered speckle;

[0033] The size of the reference area can be set to an 8×8 pixel area; and the threshold is set to two pixels.

[0034] The TOF depth camera based on dot-matrix light projection provided by the present invention includes the following modules:

[0035] A structured light projector, used to project structured light toward a target;

[0036] Infrared camera, used to collect infrared speckle images;

[0037] A memory for storing a plurality of infrared reference images, wherein the infrared reference images are pre-collected and generated by the depth camera at a plurality of different distances from a reference target.

[0038] A processor module is configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] In the present invention, the offset of each speckle in a captured infrared speckle image is determined by using an infrared reference image. When the offset is greater than a preset threshold, the speckle region is determined to be a multipath interference point. The multipath interference points in the infrared speckle image are then removed to generate a target speckle image. This allows for the detection of mirror multipath and the removal of multipath interference, thus enabling the application of a TOF depth camera based on speckle array projection on a sweeping robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without inventive work. Other features, purposes and advantages of the present invention will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0042] Figure 1 Schematic diagram of a module of a TOF depth camera based on dot-matrix light projection in an embodiment of the present invention;

[0043] Figure 2 A flowchart of the steps of determining a reference area corresponding to scattered speckles in an embodiment of the present invention;

[0044] Figure 3 Flowchart of the steps for determining speckles as multipath interference points in an embodiment of the present invention;

[0045] Figure 4 Flowchart of the steps for collecting infrared reference images in an embodiment of the present invention;

[0046] Figure 5 2 is a schematic diagram of a module of a structured light projector according to an embodiment of the present invention;

[0047] Figure 6 Schematic diagram of the module of the infrared camera in an embodiment of the present invention;

[0048] Figure 7 (a), (b), and (c) are schematic diagrams of the non-periodic arrangement of lattice light in an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of a module of a TOF depth camera based on dot-matrix light projection in a modified example of the present invention. DETAILED DESCRIPTION

[0050] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several variations and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0051] The terms "first," "second," "third," "fourth," and the like (if any) in the description and claims of the present invention and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatus.

[0052] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0053] The TOF depth camera based on dot matrix light projection provided by the present invention is intended to solve the problems existing in the prior art.

[0054] The following describes in detail the technical solutions of the present invention and how the technical solutions of this application solve the above-mentioned technical problems using specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following embodiments of the present invention are described in conjunction with the accompanying drawings.

[0055] Figure 1 FIG is a schematic diagram of a module of a TOF depth camera based on dot-matrix light projection in an embodiment of the present invention, as shown in FIG. Figure 1 As shown, the TOF depth camera based on dot matrix light projection provided by the present invention includes the following modules:

[0056] A structured light projector, used to project structured light toward a target;

[0057] an infrared camera, configured to receive structured light reflected by the target and generate an infrared speckle image and a plurality of infrared reference images, wherein the infrared reference images are generated by capturing a reference target at a plurality of different distances using a depth camera;

[0058] A processor module is configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image.

[0059] In an embodiment of the present invention, the infrared camera is an infrared detector, which receives the dot matrix light reflected by the target person through the infrared detector.

[0060] The infrared speckle image is collected by a depth camera at a distance of 30 to 80 centimeters from the target person. The depth camera is a time-of-flight camera, and a light projector of the time-of-flight camera can project a dot matrix light toward the target.

[0061] In the embodiment of the present invention, the multipath interference points may be directly deleted to generate the target speckle image.

[0062] In an embodiment of the present invention, an infrared reference image is used to determine the offset of each speckle in a captured infrared speckle image. When the offset is greater than a preset threshold, the speckle region is determined to be a multipath interference point. The multipath interference points in the infrared speckle image are then removed to generate a target speckle image. This allows for the detection of mirror multipath and the removal of multipath interference, thereby enabling the application of a TOF depth camera based on speckle array projection on a sweeping robot.

[0063] Figure 2 FIG. 1 is a flow chart of the steps of determining the reference area corresponding to the scattered speckles in an embodiment of the present invention. Figure 2 As shown, when determining the reference area corresponding to the scattered speckles in the infrared reference image, the following steps are included:

[0064] Step M1: Acquire depth data of each scattered speckle, and determine the distance between the target surface point corresponding to the scattered speckle and the depth camera according to the depth data;

[0065] Step M2: determining an infrared reference image collected at the same distance according to the distance;

[0066] Step M3: determining a reference area corresponding to the scattered speckle in the corresponding infrared reference image according to the pixel coordinates of the scattered speckle.

[0067] In an embodiment of the present invention, each speckle is extracted from the infrared speckle image, and the distance is determined according to the depth data of the speckle, thereby determining the corresponding infrared reference image.

[0068] A reference area at a corresponding position is determined in the infrared reference image according to the pixel coordinates of each scattered spot.

[0069] Figure 3 FIG. 1 is a flow chart of the steps of determining scattered speckles as multipath interference points in an embodiment of the present invention. Figure 3 As shown, when determining that the scattered speckle is a multipath interference point, the following steps are included:

[0070] Step N1: Calculating the correlation between the scattered speckle area and a sub-area at the upper left corner of the corresponding reference area;

[0071] Step N2: moving the sub-region by one pixel each time from left to right and from top to bottom, and calculating the correlation between the sub-region and the speckle region until the target sub-region with the highest correlation is determined;

[0072] Step N3: determining a center offset between the target sub-region and the scattered speckle region, and when the offset is greater than a preset threshold, determining that the scattered speckle is a multipath interference point.

[0073] In an embodiment of the present invention, the size of the speckle area can be set to a 5×5 pixel area, which includes one scattered speckle; the size of the reference area can be set to an 8×8 pixel area; and the threshold can be set to two pixels, that is, when the center offset is greater than or equal to two pixels, the scattered speckle is determined to be a multipath interference point.

[0074] In the embodiment of the present invention, the speckle region is traversed through each pixel in the reference region in a pixel-by-pixel manner in a left-to-right and top-to-bottom order.

[0075] In the embodiment of the present invention, the calculation method of the correlation r is as follows:

[0076]

[0077] Among them, A mn is the amplitude of the reference area, is the average amplitude of the reference area; B mn is the amplitude of the speckle area, is the average amplitude of the speckle area; m and n are the pixel coordinate ranges.

[0078] In the embodiment of the present invention, the amplitude may also be represented by any physical quantity selected from the group consisting of grayscale value, pixel value, illumination, luminous flux and radiation power.

[0079] Figure 4 FIG. 1 is a flow chart of steps for collecting infrared reference images in an embodiment of the present invention. Figure 4 As shown, the infrared reference image acquisition includes the following steps:

[0080] Step S101: Projecting dot matrix light toward the reference target at a distance through the structured light projector;

[0081] Step S102: generating an infrared reference image by receiving the dot matrix light reflected by the reference target through the infrared camera;

[0082] Step S103: Repeat steps S101 to S102 to obtain infrared reference images at multiple different distances.

[0083] In an embodiment of the present invention, multiple infrared reference images at different distances can be pre-collected and stored in a memory for use by the depth camera. The reference target can be a checkerboard or a flat plate.

[0084] Figure 5FIG. 1 is a schematic diagram of a module of a structured light projector according to an embodiment of the present invention. Figure 5 As shown, the structured light projector includes a light source, a light source driver and a light modulator;

[0085] The light source driver is connected to the light source and is used to drive the light source to emit light;

[0086] The light modulator is used to modulate the light projected by the light source into structured light and then project the structured light toward a target.

[0087] In an embodiment of the present invention, the optical modulator is a diffraction grating (DOE) or a spatial light modulator (SLM).

[0088] Figure 6 FIG. 1 is a schematic diagram of a module of an infrared camera according to an embodiment of the present invention. Figure 6 As shown, the infrared camera includes a lens, a filter, and an image sensor arranged along the optical path, and the image sensor is provided with at least four receiving windows; the pulse width of the receiving window is greater than or less than the pulse width of the structured light;

[0089] The image sensor is configured to receive at least four optical signals of the structured light through at least four receiving windows; the at least four receiving windows are sequentially arranged in time sequence, and each speckle depth map is generated according to the optical signal received by each receiving window.

[0090] The image sensor includes a plurality of light detectors distributed in an array;

[0091] The lens is an optical imaging lens, which is used to ensure that the direction vector of the collimated light beam passing through the lens and entering the light detector array is in a one-to-one correspondence with the light detector;

[0092] The light detector is used to receive the collimated light beam reflected by the target object.

[0093] In an embodiment of the present invention, in order to filter background noise, a narrowband filter is usually installed in the optical imaging lens so that the light detector array can only pass the incident collimated light beam of a preset wavelength. The preset wavelength can be the wavelength of the incident collimated light beam, such as 950 nanometers, or it can be between 50 nanometers less than the incident collimated light beam and 50 nanometers greater than the incident collimated light beam. The light detector array can be arranged periodically or non-periodically. Depending on the number of discrete dot matrix lights required, the light detector array can be a combination of multiple single-point light detectors or a sensor chip that integrates multiple light detectors. In order to further optimize the sensitivity of the light detector, the illumination spot of a discrete dot matrix light on the target person can correspond to one or more light detectors. When multiple light detectors correspond to the same illumination spot, the signal of each detector can be connected through a circuit, so that they can be merged into a light detector with a larger detection area.

[0094] In the embodiment of the present invention, the light detector may be a CMOS light sensor, a CCD light sensor or a SPAD light sensor.

[0095] In an embodiment of the present invention, the structured light is a dot matrix light; the dot matrix light is distributed in the following preset shapes: linear, triangular, quadrilateral, circular, hexagonal, pentagonal, random arrangement, spatially coded arrangement, and quasi-lattice arrangement.

[0096] Figure 7 (a), (b), and (c) are schematic diagrams of the non-periodic arrangement of the dot matrix light in an embodiment of the present invention. Figure 7 As shown in (a), the spatial coding arrangement is specifically to omit a part of the light beam in the periodic arrangement, thereby realizing the spatial coding of the arrangement position. The coding that can be used in practice is not limited to Figure 7 Example (a); Figure 7 As shown in (b), the random arrangement is specifically the random distribution of the collimated light beams, so that the similarity of the arrangement modes at different positions is very small or close to zero, as shown in FIG. Figure 7 As shown in (c), the quasi-crystalline lattice arrangement is specifically a non-periodic arrangement of collimated light beams at adjacent positions at close range and a periodic arrangement at long distances. Since the present invention is limited by the optical system during implementation, the actual arrangement of the collimated light beams in the cross section may be distorted, such as stretching, twisting, etc. The energy distribution of each collimated light beam in the cross section can be a circle, annulus, ellipse, or other shapes. In this arrangement as shown in 7, this arrangement is conducive to uniform sampling of non-determined targets and optimizes the effect of the final 3D depth map.

[0097] Figure 8 Schematic diagram of the module of the TOF depth camera based on dot matrix light projection in a modified example of the present invention, as shown in FIG. Figure 8As shown, the TOF depth camera based on dot matrix light projection provided by the present invention includes the following modules:

[0098] A structured light projector, used to project structured light toward a target;

[0099] an infrared camera, configured to receive the structured light reflected by the target and generate an infrared speckle image;

[0100] A memory for storing a plurality of infrared reference images, wherein the infrared reference images are pre-collected and generated by the depth camera at a plurality of different distances from a reference target.

[0101] A processor module is configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image.

[0102] In a modified embodiment of the present invention, a plurality of infrared reference images at different distances may be pre-collected and stored in a memory for retrieval and use by the processor module.

[0103] In an embodiment of the present invention, the offset of each speckle in a captured infrared speckle image is determined using an infrared reference image. When the offset is greater than a preset threshold, the speckle region is determined to be a multipath interference point. Multipath interference points in the infrared speckle image are then removed to generate a target speckle image. This allows for the detection of mirror multipath and the removal of multipath interference, enabling the application of a TOF depth camera based on speckle array projection on a sweeping robot.

[0104] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be apparent to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0105] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various variations or modifications within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A TOF depth camera based on dot-matrix light projection, characterized in that: Includes the following modules: A structured light projector, used to project structured light toward a target; an infrared camera, configured to receive structured light reflected by the target and generate an infrared speckle image and a plurality of infrared reference images, wherein the infrared reference images are generated by capturing a reference target at a plurality of different distances using a depth camera; a processor module, configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image; When determining the reference area corresponding to the scattered speckles in the infrared reference image, the following steps are included: Step M1: Acquire depth data of each scattered speckle, and determine the distance between the target surface point corresponding to the scattered speckle and the depth camera according to the depth data; Step M2: determining an infrared reference image collected at the same distance according to the distance; Step M3: determining a reference area corresponding to the scattered speckle in the corresponding infrared reference image according to the pixel coordinates of the scattered speckle.

2. The TOF depth camera based on dot-matrix light projection according to claim 1, characterized in that: When determining that the scattered speckle is a multipath interference point, the following steps are included: Step N1: Calculating the correlation between the scattered speckle area and a sub-area at the upper left corner of the corresponding reference area; Step N2: moving the sub-region by one pixel each time from left to right and from top to bottom, and calculating the correlation between the sub-region and the speckle region until the target sub-region with the highest correlation is determined; Step N3: determining a center offset between the target sub-region and the scattered speckle region, and when the offset is greater than a preset threshold, determining that the scattered speckle is a multipath interference point.

3. The TOF depth camera based on dot-matrix light projection according to claim 2, characterized in that: The calculation method of the correlation r is as follows: Among them, A mn is the amplitude of the reference area, is the average amplitude of the reference area; B mn is the amplitude of the speckle area, is the average amplitude of the speckle area; m and n are the pixel coordinate ranges.

4. The TOF depth camera based on dot-matrix light projection according to claim 1, wherein: The infrared reference image acquisition includes the following steps: Step S101: Projecting dot matrix light toward the reference target at a distance through the structured light projector; Step S102: generating an infrared reference image by receiving the dot matrix light reflected by the reference target through the infrared camera; Step S103: Repeat steps S101 to S102 to obtain infrared reference images at multiple different distances.

5. The TOF depth camera based on dot-matrix light projection according to claim 1, characterized in that: The structured light projector includes a light source, a light source driver, and a light modulator; The light source driver is connected to the light source and is used to drive the light source to emit light; The light modulator is used to modulate the light projected by the light source into structured light and then project the structured light toward a target.

6. The TOF depth camera based on dot-matrix light projection according to claim 1, characterized in that: The infrared camera includes a lens, a filter, and an image sensor arranged along the optical path, wherein the image sensor is provided with at least four receiving windows; the pulse width of the receiving window is greater than or less than the pulse width of the structured light; The image sensor is configured to receive at least four optical signals of the structured light through at least four receiving windows; The at least four receiving windows are arranged sequentially in time sequence, and each speckle depth map is generated according to the optical signal received by each receiving window.

7. The TOF depth camera based on dot-matrix light projection according to claim 1, characterized in that: The structured light is a dot matrix light; the dot matrix light is distributed in the following pre-set shapes: linear, triangular, quadrilateral, circular, hexagonal, pentagonal, random arrangement, spatially coded arrangement and quasi-lattice arrangement.

8. The TOF depth camera based on dot-matrix light projection according to claim 1, wherein: The size of the scattered speckle area can be set to a 5×5 pixel area, and the scattered speckle area includes a scattered speckle; The size of the reference area can be set to an 8×8 pixel area; and the threshold is set to two pixels.

9. A TOF depth camera based on dot-matrix light projection, characterized in that: Includes the following modules: A structured light projector, used to project structured light toward a target; an infrared camera, configured to receive the structured light reflected by the target and generate an infrared speckle image; A memory for storing a plurality of infrared reference images, wherein the infrared reference images are pre-collected and generated by the depth camera at a plurality of different distances from a reference target. a processor module, configured to generate a speckle depth map of a target based on phase differences of multiple frames of infrared speckle images, the speckle depth map including an infrared speckle image and depth data of each speckle in the infrared speckle image; determine a corresponding infrared reference image based on the depth data of each speckle; further determine a reference region corresponding to the speckle in the infrared reference image based on pixel coordinates of the speckle; match each subregion in the reference region with the corresponding speckle region to determine a target subregion with the highest correlation; determine a center offset between the target subregion and the reference region; and when the offset is greater than a preset threshold, determine the speckle region as a multipath interference point, thereby removing the multipath interference point from the infrared speckle image to generate a target speckle image; When determining the reference area corresponding to the scattered speckles in the infrared reference image, the following steps are included: Step M1: Acquire depth data of each scattered speckle, and determine the distance between the target surface point corresponding to the scattered speckle and the depth camera according to the depth data; Step M2: determining an infrared reference image collected at the same distance according to the distance; Step M3: determining a reference area corresponding to the scattered speckle in the corresponding infrared reference image according to the pixel coordinates of the scattered speckle.

Images