Depth image imaging system and method

Abstract

The present application provides a depth image imaging system and method, the imaging system comprising: a light source device, located on one side of the target scene, for emitting at least a predetermined light; a filtering device, located on one side of the target scene, for filtering the reflected light of the target scene to obtain a first filtered light and a second filtered light, the first filtered light being the predetermined light obtained by reflection and filtering of the target scene; a first imaging device and a second imaging device, located on the light-emitting side of the filtering device, for imaging the first filtered light and the second filtered light of a predetermined band respectively; a processing device, for determining the predetermined light according to the second filtered light, and calculating the depth of the target scene according to the first filtered light. The imaging system can more accurately measure the depth of the target scene.

Description

Technical Field

[0001] The present application relates to the field of imaging, and in particular, to a depth imaging system and method. Background Art

[0002] At present, there are deep imaging systems on the market that can perform depth imaging of the subject in addition to traditional color images, and combine depth information with color image information for various scenarios such as biometrics, AR games, three-dimensional map construction, robots, etc., such as: Time of Flight (TOF), structured light, binocular vision, etc. At present, TOF technology has the advantages of high frame rate, low software and hardware costs, and long ranging distance, so it is the most widely used method for obtaining depth information. However, because the current TOF technology also has some inherent disadvantages, such as low resolution, partial image missing, depth value detection needs to be deduplicated, etc., especially when using a single TOF lens for depth ranging, it is easy to cause various blanks or errors in the depth image.

[0003] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country for those skilled in the art. Summary of the invention

[0004] The main purpose of the present application is to provide a depth image imaging system and method to solve the technical problem that when using a single TOF lens to perform depth ranging on complex scenes with many reflective surfaces, it is easy to cause some pixel points to be missing or have large errors.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a depth image imaging system is provided, including: a light source device, located on one side of the target scene, for emitting at least predetermined light; a filtering device, located on one side of the target scene, for filtering the reflected light of the target scene to obtain a first filtered light and a second filtered light, wherein the first filtered light is the predetermined light obtained by being reflected and filtered by the target scene; a first imaging device and a second imaging device, located on the light-emitting side of the filtering device, for respectively imaging the first filtered light and the second filtered light of a predetermined band; a processing device, for determining the predetermined light based on the second filtered light, and calculating the depth of the target scene based on the first filtered light.

[0006] Optionally, the processing device also includes: an acquisition unit, used to acquire a first image and a second image obtained by imaging the second filtered light by the first imaging device and the second imaging device; a first processing unit, used to determine an estimated depth based on a coordinate position difference between the first image and the second image; and a second processing unit, used to determine the predetermined light based on the estimated depth.

[0007] Optionally, the processing device also includes: a first calculation unit, used to calculate the first depth of the first imaging device based on the first filtered light incident on the first imaging device; a second calculation unit, used to calculate the second depth of the second imaging device based on the first filtered light incident on the second imaging device; and a third processing unit, used to obtain the depth of the target scene based on the first depth and the second depth and weights corresponding to the first depth and the second depth.

[0008] Optionally, the third processing unit includes: a determination module for determining weights corresponding to the first depth and the second depth, wherein the weights are determined by at least one of the following methods: modulation parameters of the predetermined light generated by the light source device, wherein the modulation parameters include at least one of modulation amplitude and modulation frequency; imaging parameters of the first imaging device and the second imaging device, wherein the imaging parameters include at least one of optical parameters and imaging quality.

[0009] Optionally, the imaging system also includes: a polarizing device, located on one side of the filtering device, used to make the predetermined light incident on the first imaging device and the second imaging device have mutually orthogonal polarization directions to obtain corresponding polarized light; and an analyzing device, located between the polarizing device and the imaging device, used to make the corresponding polarized light incident on the first imaging device and the second imaging device respectively.

[0010] Optionally, the first filtered light incident on the first imaging device and the predetermined light incident on the second imaging device have different modulation parameters.

[0011] Optionally, the first imaging device and the second imaging device are arranged at intervals on the light output side of the filtering device.

[0012] Optionally, the predetermined light is modulated infrared light, and the light corresponding to the second filtered light is visible light.

[0013] Optionally, the filtering device is a Fabry-Perot interferometer cavity with adjustable cavity thickness.

[0014] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a depth image imaging method is provided, including: emitting at least predetermined light; filtering the reflected light of the target scene to obtain a first filtered light and a second filtered light, wherein the first filtered light is obtained by reflecting and filtering the predetermined light from the target scene; imaging the first filtered light and the second filtered light of a predetermined band respectively; determining the predetermined light based on the second filtered light, and calculating the depth of the target scene based on the first filtered light.

[0015] Applying the technical solution of the present application, in the above-mentioned depth image imaging system, the light source device is located on one side of the target scene and is used to emit at least a predetermined light; the filtering device is located on one side of the target scene and is used to filter the reflected light of the target scene to obtain a first filtered light and a second filtered light, wherein the first filtered light and the predetermined light are obtained by reflection and filtering of the target scene; the first imaging device and the second imaging device are located on the light-emitting side of the filtering device and are used to image the first filtered light and the second filtered light of a predetermined band respectively; the processing device is used to determine the predetermined light according to the second filtered light, and calculate the depth of the target scene according to the first filtered light. It is possible to collect depth information from multiple angles, avoid image blanks or errors caused by a single TOF lens, improve the accuracy and reliability of depth measurement and imaging effects, and thus solve the technical problem in the related art that when a single TOF lens is used to perform depth ranging on a complex scene with many reflective surfaces, it is easy to cause the measurement of some pixels to be missing or the error to be large. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a schematic structural diagram of a depth image imaging system according to an embodiment of the present application;

[0018] Figure 2 is a flow chart of a depth image imaging method according to an embodiment of the present application;

[0019] FIG3( a ) is a schematic diagram of light modulation by a TOF lens in a depth image imaging system according to an embodiment of the present application;

[0020] FIG3( b ) is a schematic diagram of light modulation by another TOF lens in a depth image imaging system according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of a depth image imaging device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element, or there may be intermediate elements. Moreover, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element, or "connected" to the other element through a third element.

[0026] In an embodiment of the present application, a depth image imaging system 10 is provided. Figure 1 is a schematic diagram of the structure of a depth image imaging system according to an embodiment of the present application, such as Figure 1 As shown, the imaging system 10 includes:

[0027] A light source device 20, located at one side of the target scene 70, for emitting at least a predetermined light;

[0028] The filtering device 30 is located at one side of the target scene 70 and is used to filter the reflected light of the target scene 70 to obtain a first filtered light and a second filtered light, wherein the first filtered light is obtained by reflecting and filtering the target scene 70;

[0029] The first imaging device 40 and the second imaging device 50 are located at the light exit side of the filtering device 30 and are used to respectively image the first filtered light and the second filtered light of the predetermined wavelength band;

[0030] The processing device 60 is used to determine the predetermined light according to the second filtered light, and calculate the depth of the target scene 70 according to the first filtered light.

[0031] As an optional embodiment, the above-mentioned light source device has the function of emitting multiple types of light, for example, it can emit different modulated lights, and the modulated lights include but are not limited to infrared light, etc. It should be noted that when the light source device emits predetermined light to the target scene, the relevant parameters such as the frequency and amplitude of the predetermined light can be flexibly adjusted. In the specific implementation process, the light source device can emit the required predetermined light according to different application scenarios. Among them, the light source device is not limited to the above-mentioned structure, and can also include multiple light source devices, and the multiple light source devices emit different predetermined lights, that is, predetermined lights with different frequencies and different amplitudes can be obtained.

[0032] As an optional embodiment, the light source device emits at least two predetermined lights, and the emission time periods of the predetermined lights are spaced apart from each other. Then, one or at least two predetermined lights are used for depth ranging. In the specific implementation process, the depth ranging results of the two predetermined lights can be combined with each other to reduce errors and compensate for image deficiencies, and can also be used to remove overlapping measurement results of additional kλ / 2 (k is a positive integer) in depth ranging. In turn, the accuracy of the imaging system can be improved.

[0033] As an optional embodiment, the filtering device can realize filtering processing of the reflected light of the target scene, wherein the filtering processing is based on a predetermined band to filter the reflected light of the target scene. Of course, the predetermined band can be set manually, or the default form of the imaging system can be adopted. For example, for visible light, the predetermined band can be selected within the wavelength range of 380nm-780nm, and the infrared light can be selected within the wavelength range of 780nm-3000nm. In the specific implementation process, it is not limited to the above-described content.

[0034] It should be noted that the first filtered light corresponds to predetermined light, which may be infrared light, etc.; the second filtered light corresponds to visible light, which may be visible light under natural conditions or visible light emitted by a light source device. In a specific implementation, taking visible light as an example, when the reflected light of the target scene is visible light, after passing through the filtering device, the visible light will be filtered to obtain the second filtered light; taking infrared light as an example, when the reflected light of the target scene is infrared light, after passing through the filtering device, the infrared light will be filtered to obtain the first filtered light.

[0035] Moreover, both visible light and infrared light can be filtered according to a predetermined wavelength band to obtain the light required for imaging. For the imaging system, through the above-mentioned filtering process, some useless or interfering light can be filtered out, effectively retaining the light required for imaging, thereby improving the subsequent imaging effect and the accuracy of depth calculation.

[0036] As an optional embodiment, in the imaging system of the present application, the imaging device includes a first imaging device and a second imaging device, and both imaging devices can realize imaging of the same target scene, so that different imaging devices can realize depth ranging of the same target scene from different positions to reduce the error of depth ranging. In the embodiment of the present application, the above-mentioned imaging device is a TOF lens.

[0037] As an optional embodiment, the above-mentioned imaging device includes an array of two-dimensional pixels, which can be a well-known imaging device such as CMOS, CCD, etc., wherein each pixel images a different spatial point in the scene, and may also include an equipped narrow-band filter, etc. However, in the specific implementation, it is not limited to the contents described above.

[0038] As an optional embodiment, the above-mentioned processing device includes a processor and a memory, the processing unit is stored in the memory as a program unit, and the processor executes the above-mentioned program unit stored in the memory to implement corresponding functions.

[0039] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the depth of the target scene to be measured can be calculated by adjusting kernel parameters.

[0040] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory can be used to store calculation data related to the depth of the target scene, as well as corresponding algorithms, programs, etc.

[0041] In the above-mentioned depth image imaging system of the present application, the light source device is located on one side of the target scene, and is used to emit at least a predetermined light; the filtering device is located on one side of the target scene, and is used to filter the reflected light of the target scene to obtain a first filtered light and a second filtered light, and the first filtered light and the predetermined light are obtained by reflection and filtering of the target scene; the first imaging device and the second imaging device are respectively located on the light-emitting side of the filtering device, and are used to image the first filtered light and the second filtered light of the predetermined band respectively; the processing device is used to determine the predetermined light according to the second filtered light, and calculate the depth of the target scene according to the first filtered light. In this system, the filtering device can be used to switch between visible imaging and infrared imaging, and the depth can be estimated under visible imaging. Then, under infrared imaging, the appropriate modulated light is selected according to the depth estimation, and the scene depth is calculated.

[0042] It should be noted that in the specific implementation process, the filter device is adjusted to transmit visible light, the depth is estimated using the triangular parallax method, and then two appropriate modulation frequencies of infrared light are set according to the estimated depth. At this time, the filter device is adjusted to transmit infrared light, and the two imaging devices perform depth measurement through two modulation frequencies respectively, and then deduplication and mixing are performed according to the results of the depth measurement to obtain the depth of the target scene. In this way, depth information can be collected from multiple angles, reducing image blanks or errors caused by a single TOF depth imaging lens.

[0043] Furthermore, the above-mentioned imaging system of the present application can be applied to target scenes with multiple reflective surfaces, so that depth information can be collected from multiple angles, avoiding image blanks or errors caused by a single TOF lens, and improving the accuracy and reliability of depth measurement and imaging effects, thereby solving the technical problem in the related art that when a single TOF lens is used to perform depth ranging on complex scenes with many reflective surfaces, it is easy to cause the measurement of some pixels to be missing or the error to be large.

[0044] Optionally, the processing device also includes: an acquisition unit, used to acquire the first image and the second image obtained by imaging the second filtered light by the first imaging device and the second imaging device; a first processing unit, used to determine the estimated depth based on the coordinate position difference between the first image and the second image; and a second processing unit, used to determine the predetermined light based on the estimated depth.

[0045] As an optional embodiment, for the visible light reflected by the target scene, the visible light is filtered by a filtering device before entering the imaging device, and then a second filtered light is obtained, and then the second filtered light is incident on the first imaging device and the second imaging device, and further, the first imaging device and the second imaging device respectively image the second filtered light reflected by the target scene. The first imaging device images the second filtered light to obtain a first image, and the second imaging device images the second filtered light to obtain a second image.

[0046] As an optional embodiment, the first image and the second image include a target object in the same target scene, and the coordinate position difference between the target object in the first image and the second image can be further obtained, and then the estimated depth is obtained using the triangulation parallax method, wherein the estimated depth is depth measurement data obtained based on visible light.

[0047] As an optional embodiment, determining the estimated depth based on the coordinate position difference between the first image and the second image also includes: using a trigonometric formula to calculate and generate the estimated depth, where z = f(1+b / Δl), where f is the focal length of the lens, b is the baseline length, Δl is the coordinate position difference between the pixel in the first image and the second image, and z is the depth.

[0048] As an optional embodiment, the predetermined light is determined according to the estimated depth, wherein the predetermined light is modulated infrared light. In a specific implementation, at least two predetermined lights are determined according to the estimated depth, and at least two predetermined lights have different frequencies and / or amplitudes. Preferably, the predetermined light is two different modulated infrared lights.

[0049] Optionally, the processing device also includes: a first calculation unit, used to calculate the first depth of the first imaging device based on the first filtered light incident on the first imaging device; a second calculation unit, used to calculate the second depth of the second imaging device based on the first filtered light incident on the second imaging device; and a third processing unit, used to obtain the depth of the target scene based on the first depth and the second depth and weights corresponding to the first depth and the second depth.

[0050] As an optional embodiment, for the infrared light reflected by the target scene, the infrared light is filtered by a filtering device before entering the imaging device, and then a first filtered light is obtained, and then the first filtered light is incident on the first imaging device and the second imaging device. Further, the first imaging device and the second imaging device respectively image the first filtered light reflected by the target scene. The first imaging device images the first filtered light to obtain a third image, and the second imaging device images the first filtered light to obtain a fourth image.

[0051] By calculating the depth separately through two imaging lenses and combining them into a depth map, data gaps can be filled and the overall depth detection accuracy can be improved.

[0052] Optionally, the third processing unit includes: a determination module for determining weights corresponding to the first depth and the second depth, wherein the weights are determined by at least one of the following methods: modulation parameters of the predetermined light generated by the light source device, wherein the modulation parameters include at least one of the modulation amplitude and the modulation frequency; imaging parameters of the first imaging device and the second imaging device, wherein the imaging parameters include at least one of the optical parameters and the imaging quality.

[0053] As an optional embodiment, there are many ways to determine the weights, including: modulation parameters of the predetermined light generated by the light source device, imaging parameters of the first imaging device and the second imaging device. In the specific implementation process, it is not limited to the methods described above.

[0054] First, the depth and depth error are estimated based on the difference in imaging between the two lenses in visible light imaging. Then, the infrared light is modulated specifically according to the estimated depth and error size, and the modulation amplitude and frequency suitable for the estimated depth are output. This can reduce errors and reduce energy consumption.

[0055] Optionally, the imaging system also includes: a polarizing device, located on one side of the filtering device, used to make the predetermined light incident on the first imaging device and the second imaging device have mutually orthogonal polarization directions to obtain corresponding polarized light; and an analyzing device, located between the polarizing device and the imaging device, used to make the corresponding polarized light incident on the first imaging device and the second imaging device respectively.

[0056] As an optional embodiment, the above-mentioned polarizing device can adjust different predetermined lights into corresponding polarized lights. The polarized lights obtained after adjustment can not only reduce the mutual interference of depth measurements of the two imaging devices when the imaging time overlaps, but also avoid other influences such as ambient light on the imaging results, thereby further ensuring the accuracy of the imaging system.

[0057] As an optional embodiment, the above-mentioned polarization analyzer can adjust the polarization direction of the above-mentioned polarized light, eliminate multi-path reflection, and ensure that the polarized light corresponding to the predetermined light accurately enters the imaging device, further improving the accuracy and reliability of the imaging system.

[0058] Optionally, the first filtered light incident on the first imaging device and the predetermined light incident on the second imaging device have different modulation parameters.

[0059] As an optional embodiment, two imaging devices are configured to perform depth measurement using different modulation parameters, respectively, so that the measurement results can be used to deduplicate the detection results.

[0060] Optionally, the first imaging device and the second imaging device are arranged at intervals on the light-exiting side of the filtering device.

[0061] It should be noted that by arranging the first imaging device and the second imaging device at intervals on the light-emitting side of the filtering device, it is possible to simultaneously perform depth measurement of the target scene from different positions or angles, thereby reducing measurement errors. This can effectively avoid the problem in the related art that when a single imaging device is used to perform depth measurement on a complex scene with a large number of reflective surfaces, which easily causes the measurement of some pixels to be missing or the error to be large.

[0062] Optionally, the predetermined light is modulated infrared light, and the light corresponding to the second filtered light is visible light.

[0063] As an optional embodiment, the predetermined light may be modulated infrared light, wherein different modulated infrared lights have different characteristics such as frequency and amplitude. In order to enable the imaging system to be applicable to target scenes at different depths, the modulated infrared light may be set according to the needs of the target scene. It should be noted that, in a specific implementation, the amplitude of the modulated infrared light with a larger frequency is smaller than the amplitude of the modulated infrared light with a smaller frequency.

[0064] As an optional embodiment, the light corresponding to the second filtered light is visible light, and the visible light may be ambient light or natural light, that is, the visible light reflected in the target scene may be filtered to obtain the second filtered light.

[0065] Optionally, the filtering device is a Fabry-Perot interference cavity with adjustable cavity thickness, and the adjustable cavity thickness means that the height of the cavity can be adjusted.

[0066] As an optional embodiment, by adjusting the cavity thickness of the above-mentioned filtering device, that is, adjusting the predetermined band of filtering, that is, the adjustable wavelength range, filtering imaging is performed within the visible light and infrared light wavelength ranges respectively.

[0067] As an optional embodiment, the filtering device preferably adopts a Fabry-Perot (FP) interference cavity, and changes the thickness of the cavity participating in the resonance by means of piezoelectric ceramics, micro-electro-mechanical systems (MEMS), angle clarity or liquid crystal filling, so as to achieve the purpose of changing the peak transmission wavelength. For example, the transmission peak wavelength of the FP interference cavity under vertical incidence is 2nd / k, where n is the refractive index of the cavity, d is the cavity thickness, k is a positive integer, and the half-height width of the transmission peak can also be adjusted by n, d, k and the reflectivity of the cavity surface.

[0068] The filter device includes a liquid crystal layer or a variable thickness layer, and the surface of the liquid crystal layer or the variable thickness layer is a highly reflective surface. Preferably, the angle or thickness of the filter device can be adjusted by voltage, and the refractive index of the filter cavity material can also be changed, so as to effectively filter light of different wavelengths.

[0069] In an embodiment of the present application, a depth image imaging method is provided. Figure 2 is a flowchart of a depth image imaging method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0070] Step S102, emitting at least a predetermined light;

[0071] Step S104, filtering the reflected light of the target scene to obtain first filtered light and second filtered light, where the first filtered light is the predetermined light obtained by being reflected from the target scene and filtered;

[0072] Step S106, imaging the first filtered light and the second filtered light of the predetermined wavelength band respectively;

[0073] Step S108, determining the predetermined light according to the second filtered light, and calculating the depth of the target scene according to the first filtered light.

[0074] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0075] The present application utilizes at least emitting predetermined light; filtering the reflected light of the target scene to obtain a first filtered light and a second filtered light, wherein the first filtered light is obtained by reflecting and filtering the predetermined light from the target scene; imaging the first filtered light and the second filtered light of a predetermined band respectively; determining the predetermined light according to the second filtered light, and calculating the depth of the target scene according to the first filtered light. The imaging method can switch between visible imaging and infrared imaging through a filtering device, estimate the depth under visible imaging, and then select a suitable modulated light according to the depth estimation under infrared imaging, and then calculate the scene depth.

[0076] Furthermore, the imaging method can be applied to target scenes with multiple reflective surfaces, so that depth information can be collected from multiple angles, avoiding image blanks or errors caused by a single TOF lens, and improving the accuracy and reliability of depth measurement and imaging effects, thereby solving the technical problem in the related art that when a single TOF lens is used to perform depth ranging on complex scenes with many reflective surfaces, it is easy to cause the measurement of some pixels to be missing or the error to be large.

[0077] As an optional embodiment, in the above-mentioned depth image imaging method, determining the predetermined light based on the second filtered light includes: acquiring a first image and a second image obtained by imaging the second filtered light by a first imaging device and a second imaging device; determining an estimated depth based on a coordinate position difference between the first image and the second image; and determining the predetermined light based on the estimated depth.

[0078] As an optional embodiment, in the above-mentioned depth image imaging method, calculating the depth of the target scene based on the first filtered light includes: calculating the first depth of the first imaging device based on the first filtered light incident on the first imaging device; calculating the second depth of the second imaging device based on the first filtered light incident on the second imaging device; and obtaining the depth of the target scene based on the first depth and the second depth and weights corresponding to the first depth and the second depth.

[0079] As an optional embodiment, in the above-mentioned depth image imaging method, the method includes: determining weights corresponding to the first depth and the second depth, wherein the weights are determined by at least one of the following methods: modulation parameters of the predetermined light generated by the light source device, wherein the modulation parameters include at least one of the modulation amplitude and the modulation frequency; imaging parameters of the first imaging device and the second imaging device, wherein the imaging parameters include at least one of the optical parameters and the imaging quality.

[0080] As an optional embodiment, in the above-mentioned depth image imaging method, the method also includes: making the predetermined light incident on the first imaging device and the second imaging device have mutually orthogonal polarization directions to obtain corresponding polarized light; and respectively incident the corresponding polarized light on the first imaging device and the second imaging device.

[0081] As an optional embodiment, in the above-mentioned depth image imaging method, the method further includes: the first filtered light incident on the first imaging device and the predetermined light incident on the second imaging device have different modulation parameters.

[0082] As an optional embodiment, the first imaging device and the second imaging device are arranged at intervals on the light-exiting side of the filtering device.

[0083] As an optional embodiment, the predetermined light is modulated infrared light, and the light corresponding to the second filtered light is visible light.

[0084] As an optional embodiment, the filtering device is a Fabry-Perot interferometer cavity with adjustable cavity thickness.

[0085] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a storage medium is provided, which stores a program, wherein when the program is executed by a processor, the processor is controlled to execute any one of the above-mentioned depth image imaging methods.

[0086] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a computer device is provided, including: a memory and a processor, the memory storing a computer program; the processor is used to execute the computer program stored in the memory, and when the computer program is running, the processor executes any one of the above-mentioned depth image imaging methods.

[0087] An optional implementation of the present application is described below.

[0088] It should be noted that, in the following optional implementation manner, the TOF lens is equivalent to an imaging device, and the filter is equivalent to a filtering device.

[0089] When using a single TOF lens to image or measure the depth of complex scenes with many reflective surfaces, it is easy to cause some pixels to be missing or have large errors. This is one of the reasons why the resolution of TOF lenses is lower than that of other depth measurement methods.

[0090] Based on this, in order to make it suitable for scenes with higher precision requirements, the preferred implementation of the present application can use two TOF lenses to simultaneously measure depth from different positions, and combine the measured depth information to reduce the measurement errors of each other. In addition, the advantages of the two TOF lenses can be used to first estimate the depth in the visible light band using the ambient light in natural conditions.

[0091] Further, in a preferred embodiment of the present application, the filter preferably adopts a Fabry-Perot (FP) interferometer cavity, and changes the thickness of the cavity participating in the resonance by means of piezoelectric ceramics, MEMS, angle clarity or liquid crystal filling, so as to achieve the purpose of changing the peak value of the transmission wavelength. For example, the transmission peak wavelength of the FP interferometer cavity under vertical incidence is 2nd / k, where n is the refractive index of the cavity, d is the thickness of the cavity, k is a positive integer, and the half-height width of the transmission peak can also be adjusted by n, d, k and the reflectivity of the cavity surface. By adjusting d, the transmission peak appears in the visible light range, such as 587nm, 486nm, 656nm, etc.

[0092] After the filter processing, two TOF lenses are used to image the scene in visible light, and the two TOF lenses obtain the first image and the second image respectively. The first image and the second image can generate a preliminary depth map according to the trigonometric formula by the difference in the coordinate position of the same feature in the image, that is, z = f (1 + b / Δl), where f is the focal length of the lens, b is the baseline length, Δl is the position difference of the pixel in the first and second images, and z is the depth.

[0093] It should be noted that, since it is necessary to switch between infrared light and visible light imaging, it is necessary to use a filter with a variable transmission peak. As a preferred embodiment, voltage is used to adjust the angle or thickness of the filter, or to change the refractive index of the filter cavity material (such as liquid crystal, etc.).

[0094] Among them, the estimated depth z is then used to determine the modulation frequency of the modulated infrared light for TOF suitable for the current scene, and the modulation amplitude can be further modified after the modulation frequency is determined. Since z = θλ / 4π in phase-type TOF detection, where θ is the phase difference between the reflected light and the modulated light, the maximum is 2π, and λ is the modulation wavelength, the maximum detection distance should be controlled within λ / 2. After obtaining the estimated depth, the modulation frequency of the modulated light source can be changed to ensure that the maximum detection distance is not exceeded. It is also possible to select a modulation amplitude that can provide sufficient signal-to-noise ratio in the current scene according to the determined modulation frequency to reduce power consumption. After obtaining the estimated depth map, the user can also be preliminarily prompted with problems in the scene and suggestions for response, such as prompting that the scene is too close or too far from the lens, too many objects in the scene may increase the error, the scene is too monotonous and lacks depth changes, etc. to suggest the user to move the shooting direction. This process can be continuous until the user has moved to a scene suitable for applications such as AR, before which only visible light detection is performed without entering the infrared light emission stage.

[0095] Thereafter, infrared light with a determined modulation frequency and amplitude is emitted to the scene, and the filter is adjusted as above until the transmission peak appears in the infrared band, such as 940nm, and TOF ranging is started. Since two TOF lenses are used at the same time, at least two modulated lights that can be distinguished from each other by being separated in time or orthogonal in polarization can also be output. The modulation frequencies and amplitudes corresponding to the two modulated lights can be substantially different and the two TOF lenses can use one of the two modulated lights for depth ranging. Figure 3(a) is a schematic diagram of modulated light of a TOF lens in a depth image imaging system according to an embodiment of the present application, and Figure 3(b) is a schematic diagram of modulated light of another TOF lens in a depth image imaging system according to an embodiment of the present application. As shown in Figures 3(a) and 3(b), the depth ranging results of the two modulated lights can be used to combine with each other to reduce errors and compensate for image deficiencies, and can also be used to remove overlapping measurement results of additional kλ / 2 (k is a positive integer) in depth ranging.

[0096] Finally, the depths detected by the two TOFs will be combined by weight to form the final depth map. If the modulated light of the same frequency and amplitude is imaged, the weights can be equal, which is only used to compensate for errors such as image blanks. However, when the two TOF lenses respectively image different frequencies and amplitudes that are distinguished from each other in time or polarization, the weights of the two lenses should be different. For example, the weight of the lens with a modulation frequency that corresponds more to the estimated depth during visible light detection is increased, the weight of the lens with a higher modulation amplitude is increased, or the weight of the lens with a smaller overall imaging error is increased. The depth value obtained by the final weighted combination will be able to obtain reduced errors and higher image resolution. In addition, the two TOF lenses should be calibrated and calibrated in advance so that the coordinates of their imaging can correspond to each other and be combined into an image, and the calibration matrix can be directly used in each depth measurement after it is determined.

[0097] The embodiment of the present application provides a depth image imaging device, Figure 4 is a schematic diagram of a depth image imaging device according to an embodiment of the present application, and the imaging device includes any one of the depth image imaging systems 10 described above.

[0098] The above imaging device can be applied to, including but not limited to, servers, PCs, PADs, mobile phones, etc. In a specific application, the above Figure 4 The device shown is a mobile phone, and the imaging system 10 is located in the mobile phone. Figure 4 Only the external structure of the mobile phone is shown. From the outside, the position of the imaging system 10 is Figure 4 as shown in .

[0099] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0100] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0101] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A depth image imaging system, characterized in that: include: A light source device, located at one side of the target scene, for emitting at least a predetermined light; A filtering device, located at one side of the target scene, for filtering the reflected light of the target scene to obtain a first filtered light and a second filtered light, wherein the first filtered light is obtained by reflecting and filtering the predetermined light from the target scene; A first imaging device and a second imaging device are located at the light exit side of the filtering device, and are used to respectively image the first filtered light and the second filtered light of a predetermined wavelength band; a processing device, configured to determine the predetermined light according to the second filtered light, and calculate the depth of the target scene according to the first filtered light, The processing device also includes: An acquisition unit, configured to acquire a first image and a second image obtained by imaging the second filtered light by the first imaging device and the second imaging device; A first processing unit, configured to determine an estimated depth according to a coordinate position difference between the first image and the second image; The second processing unit is used to determine the predetermined light according to the estimated depth, including: determining a modulation frequency according to the estimated depth, and using the light with the modulation frequency as the predetermined light suitable for the target scene for the first imaging device and the second imaging device.

2. The imaging system according to claim 1, characterized in that The processing device also includes: A first calculation unit, configured to calculate a first depth of the first imaging device according to the first filtered light incident on the first imaging device; a second calculation unit, configured to calculate a second depth of the second imaging device according to the first filtered light incident on the second imaging device; The third processing unit is configured to obtain the depth of the target scene according to the first depth, the second depth, and weights corresponding to the first depth and the second depth.

3. The imaging system according to claim 2, characterized in that The third processing unit includes: a determination module, used to determine the weights corresponding to the first depth and the second depth, wherein the weights are determined by at least one of the following methods: modulation parameters of the predetermined light generated by the light source device, wherein the modulation parameters include at least one of a modulation amplitude and a modulation frequency; imaging parameters of the first imaging device and the second imaging device, wherein the imaging parameters include at least one of an optical parameter and an imaging quality.

4. The imaging system according to claim 1, characterized in that The imaging system further comprises: A polarizing device, located at one side of the filtering device, used to make the predetermined light incident on the first imaging device and the second imaging device have mutually orthogonal polarization directions to obtain corresponding polarized light; The polarization analyzer device is located between the polarizing device and the imaging device, and is used for respectively causing the corresponding polarized light to be incident on the first imaging device and the second imaging device.

5. The imaging system according to claim 1, characterized in that The first filtered light incident on the first imaging device and the predetermined light incident on the second imaging device have different modulation parameters.

6. The imaging system according to any one of claims 1 to 5, characterized in that: The first imaging device and the second imaging device are arranged at intervals on the light exiting side of the filtering device.

7. The imaging system according to any one of claims 1 to 5, characterized in that: The predetermined light is modulated infrared light, and the light corresponding to the second filtered light is visible light.

8. The imaging system according to any one of claims 1 to 5, characterized in that: The filtering device is a Fabry-Perot interference cavity with adjustable cavity thickness.

9. A depth image imaging method, characterized in that: include: at least emits the intended light; Filtering the reflected light of the target scene to obtain first filtered light and second filtered light, wherein the first filtered light is obtained by reflecting and filtering the predetermined light from the target scene; respectively imaging the first filtered light and the second filtered light of a predetermined wavelength band; determining the predetermined light according to the second filtered light, and calculating the depth of the target scene according to the first filtered light, Determining the predetermined light according to the second filtered light comprises: Acquire a first image and a second image obtained by imaging the second filtered light with a first imaging device and a second imaging device; Determining an estimated depth according to a coordinate position difference between the first image and the second image; Determining the predetermined light according to the estimated depth includes: determining a modulation frequency according to the estimated depth, and using light with the modulation frequency as predetermined light suitable for the target scene for the first imaging device and the second imaging device.

10. The imaging method according to claim 9, characterized in that: Calculating the depth of the target scene according to the first filtered light includes: Calculating a first depth of the first imaging device according to the first filtered light incident on the first imaging device; Calculating a second depth of the second imaging device according to the first filtered light incident on the second imaging device; The depth of the target scene is obtained according to the first depth, the second depth, and weights corresponding to the first depth and the second depth.

11. The imaging method according to claim 10, characterized in that: The method includes: determining weights corresponding to the first depth and the second depth, wherein the weights are determined by at least one of the following methods: modulation parameters of predetermined light generated by a light source device, wherein the modulation parameters include at least one of a modulation amplitude and a modulation frequency; imaging parameters of the first imaging device and the second imaging device, wherein the imaging parameters include at least one of an optical parameter and an imaging quality.

12. The imaging method according to claim 9, characterized in that: The method further comprises: The predetermined light incident on the first imaging device and the second imaging device has mutually orthogonal polarization directions to obtain corresponding polarized light; The corresponding polarized light is incident on the first imaging device and the second imaging device respectively.

13. The imaging method according to claim 9, characterized in that: The method further comprises: The first filtered light incident on the first imaging device and the predetermined light incident on the second imaging device have different modulation parameters.

14. The imaging method according to any one of claims 9 to 13, characterized in that: The first imaging device and the second imaging device are arranged at intervals on the light exiting side of the filtering device.

15. The imaging method according to claim 14, characterized in that: The filtering device is a Fabry-Perot interference cavity with adjustable cavity thickness.

16. The imaging method according to any one of claims 9 to 13, characterized in that: The predetermined light is modulated infrared light, and the light corresponding to the second filtered light is visible light.

17. A storage medium, characterized in that: The storage medium stores a program, wherein when the program is executed by the processor, the processor is controlled to execute the depth image imaging method described in any one of claims 9 to 16.

18. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program; the processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor executes the depth image imaging method described in any one of claims 9 to 16.

Images