Systems, methods, and apparatuses for generating depth images

Abstract

Embodiments of the present disclosure disclose a system, a method, and an apparatus for generating a depth image. A specific implementation of the system includes: an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic apertures are configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device. The control device is configured to: obtain a first photo and a second photo, where the first photo and the second photo are photos obtained by the image sensor capturing a target scene according to first shooting configuration information and second shooting configuration information respectively; generate a depth image of the target scene based on the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information. This implementation reduces the economic cost of generating a depth image and improves the image resolution of the generated depth image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly, to systems, methods, and devices for generating depth images. Background Art

[0002] As more and more applications rely on three-dimensional information, scene depth measurement has become increasingly important. For example, scene depth measurement is required in scenarios such as autonomous vehicles, augmented reality / virtual reality, 3D immersive experiences, 3D games, and smartphone 3D applications.

[0003] Currently, the main methods for measuring the distance of objects in a scene by imaging (excluding scanning lidar) are as follows: First, stereoscopic imaging vision, binocular imaging, and triangulation; Second, using computer vision based on images and object projections; Third, illuminating the scene with a pre-determined illumination pattern (structured light), analyzing the scene using the illumination, and finding the depth information using previously obtained calibration information; Fourth, using a time-of-flight (TOF) camera (multiple photodetectors) to measure the distance; Fifth, using a Pockels cell to adjust the transmission of the signal received from the scene to determine the distance of an object in the scene. Summary of the Invention

[0004] Embodiments of the present disclosure propose systems, methods, and devices for generating depth images.

[0005] In a first aspect, an embodiment of the present disclosure provides a system for generating a depth image. The system includes: an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic aperture is configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device. The control device is configured to: obtain a first photo and a second photo, where the first photo and the second photo are respectively photos obtained by the image sensor capturing a target scene according to first shooting configuration information and second shooting configuration information. The first shooting configuration information includes a first light transmittance function and a first light intensity function, and the second shooting configuration information includes a second light transmittance function and a second light intensity function. The first light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the first photo with respect to the image sensor for capturing the first photo and time. The second light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the second photo with respect to the image sensor for capturing the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the second photo and time. The first light transmittance function is not a constant and / or the second light transmittance function is not a constant, the first light transmittance function is different from the second light transmittance function and / or the first light intensity function is different from the second light intensity function; generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0006] In some embodiments, the control device is further configured to: generate a three-dimensional model of the target scene according to the first photo, the second photo, and the depth image.

[0007] In some embodiments, at least one set of dynamic aperture and corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor; and obtaining a first photo and a second photo includes: simultaneously performing a first control operation, a second control operation, and a third control operation, wherein the first control operation includes controlling the light intensity of the first illumination light emitted by the illumination source to the target scene within a preset exposure duration according to a first light intensity function, wherein the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the second control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to a first light transmittance function, the third control operation includes controlling the first image sensor to take a photo of the target scene; determining the photo taken by the first image sensor as the first photo; simultaneously performing a fourth control operation, a fifth control operation, and a sixth control operation, wherein the fourth control operation includes controlling the light intensity of the second illumination light emitted by the illumination source to the target scene within the preset exposure duration according to a second light intensity function, wherein the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1, the fifth control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to a second light transmittance function, the sixth control operation includes controlling the first image sensor to take a photo of the target scene; determining the photo taken by the first image sensor as the second photo.

[0008] In some embodiments, at least one set of dynamic apertures and corresponding image sensors includes a first dynamic aperture and a corresponding first image sensor, and a second dynamic aperture and a corresponding second image sensor; and obtaining a first photo and a second photo includes: simultaneously performing a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation, wherein the seventh control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function, the eighth control operation includes controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to a second light transmittance function, the ninth control operation includes controlling an illumination source to emit third illumination light to a target scene within the preset exposure duration, wherein a pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, the third illumination light includes first illumination light that is reflected by the target scene and causes corresponding light to reach the first image sensor through an optical system and the first dynamic aperture, the third illumination light includes second illumination light that is reflected by the target scene and causes corresponding light to reach the second image sensor through the optical system and the second dynamic aperture, an intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, an intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function, and the tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture photos of the target scene; determining the photo captured by the first image sensor as the first photo, and determining the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0009] In some embodiments, the optical system includes a filter configured to separate light of at least one wavelength belonging to a first preset wavelength set and light of at least one wavelength belonging to a second preset wavelength set, the third illumination light includes first illumination light of at least one wavelength belonging to the first preset wavelength set and second illumination light of at least one wavelength belonging to the second preset wavelength set, the first illumination light is reflected by the target scene and causes corresponding light to reach the first image sensor through the filter and the first dynamic aperture, and the second illumination light is reflected by the target scene and causes corresponding light to reach the second image sensor through the filter and the second dynamic aperture; and controlling the illumination source to emit the third illumination light to the target scene within the preset exposure duration includes: simultaneously performing an eleventh control operation and a twelfth control operation, wherein the eleventh control operation includes controlling the illumination source to emit the first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having an intensity conforming to the first light intensity function to the target scene within the preset exposure duration, and the twelfth control operation includes controlling the illumination source to emit the second illumination light including light of at least one wavelength belonging to the second preset wavelength set and having an intensity conforming to the second light intensity function to the target scene within the preset exposure duration.

[0010] In some embodiments, the optical system includes a polarizer; the polarizer is configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The third illumination light includes first illumination light with a polarization state of the first preset polarization state and second illumination light with a polarization state of the second preset polarization state. The first illumination light is reflected by the target scene, and through the polarizer and the first dynamic aperture, the corresponding light reaches the first image sensor. The second illumination light is reflected by the target scene, and through the polarizer and the second dynamic aperture, the corresponding light reaches the second image sensor; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a thirteenth control operation and a fourteenth control operation, where the thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state of the first preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a first light intensity function, and the fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state of the second preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a second light intensity function.

[0011] In some embodiments, the optical system includes a first beam splitting component, a first filter component, and a second filter component. The first filter component transmits light with at least one wavelength belonging to a first preset wavelength set, and the second filter component transmits light with at least one wavelength belonging to a second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor are located on a first side of the first beam splitting component, and the target scene, the second filter component, the second dynamic aperture, and the second image sensor are located on a second side of the first beam splitting component; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a fifteenth control operation and a sixteenth control operation, where the fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having a light intensity conforming to a first light intensity function to the target scene within the preset exposure duration, and the sixteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having a light intensity conforming to a second light intensity function to the target scene within the preset exposure duration.

[0012] In some embodiments, the optical system includes a second beam splitting component and a third filter component. The second beam splitting component is configured to split light and transmit light with at least one wavelength belonging to a first preset wavelength set. The third filter component is configured to transmit light with at least one wavelength belonging to a second preset wavelength set. The first dynamic aperture and the first image sensor are located on a first side of the second beam splitting component, and the target scene, the third filter component, the second dynamic aperture, and the second image sensor are located on a second side of the second beam splitting component. And controlling the illumination source to emit third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a seventeenth control operation and an eighteenth control operation, wherein the seventeenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to a first optical intensity function to the target scene within the preset exposure duration, and the eighteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to a second optical intensity function to the target scene within the preset exposure duration.

[0013] In some embodiments, at least one set of a dynamic aperture and a corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor. The first image sensor is an image sensor array in which first image sensor pixel units and second image sensor pixel units are alternately arranged. The first image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a first preset wavelength set, and the second image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a second preset wavelength set; and obtaining a first photo and a second photo, including: simultaneously performing a nineteenth control operation, a twentieth control operation, a twenty-first control operation, and a twenty-second control operation. Among them, the nineteenth control operation includes controlling an illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to a first optical intensity function to a target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light including light of at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to a second optical intensity function to the target scene within the preset exposure duration. Wherein, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light of a wavelength belonging to the first preset wavelength set within the preset exposure duration according to a first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light of a wavelength belonging to the second preset wavelength set within the preset exposure duration according to a second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; obtaining the photo taken by the first image sensor, and determining the obtained photo as a third photo; generating a first photo with the pixel values of each pixel point corresponding to the first image sensor pixel units in the third photo; generating a second photo with the pixel values of each pixel point corresponding to the second image sensor pixel units in the third photo.

[0014] In some embodiments, at least one set of dynamic aperture and the corresponding image sensor includes a first dynamic aperture and the corresponding first image sensor and a second image sensor. The first image sensor is located on the first side of the first dynamic aperture, and the second image sensor and the target scene are located on the second side of the first dynamic aperture; and obtaining a first photo and a second photo includes: simultaneously performing a twenty-third control operation, a twenty-fourth control operation, and a twenty-fifth control operation, where the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to a first light transmittance function and a second light transmittance function respectively, the twenty-fourth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration, where the first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing the corresponding light to reach the first image sensor, and the second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing the corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, the light intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function, the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, and the twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture photos of the target scene; determining the photo captured by the first image sensor as the first photo, and determining the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0015] In some embodiments, according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information, generating a depth image of the target scene includes: for each pixel point with coordinates (m, n) in the first photo and the second photo, establishing a first equation and a second equation, where the first equation is:

[0016]

[0017] where R(m, n) is the reflectivity of the area in the target scene corresponding to the pixel point with coordinates (m, n), f1 is the first light intensity function, t is a time variable, is the light intensity of the first illumination light when it reaches the area in the target scene corresponding to the pixel point with coordinates (m, n) in the first photo from the illumination source and then returns to the first image sensor at time t, t d (m, n) is the duration for the light to reach the area in the target scene corresponding to the pixel point with coordinates (m, n) from the position where the illumination source is located, c is the speed of light, z(m,n) is the distance between the region corresponding to the pixel at coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position of the first image sensor and the region corresponding to the pixel at coordinates (m,n) in the target scene, and the second side is the line segment connecting the position of the first image sensor and the position of the illumination source, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m,n) is the pixel value of the pixel at coordinates (m,n) in the first photo; The second equation is:

[0018]

[0019] where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the region corresponding to the pixel at coordinates (m,n) in the second photo in the target scene and then returns to the first image sensor at time t, h2 is the second light transmission function, and S2(m,n) is the pixel value of the pixel at coordinates (m,n) in the second photo; The third equation is obtained from the first equation and the second equation, and the third equation is:

[0020]

[0021] Solve the third equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; generate a depth image of the target scene based on the determined depth value of each pixel. In some embodiments, the control device is further configured to: for each shooting configuration information in the set of shooting configuration information, obtain a third photo and a fourth photo corresponding to the shooting configuration information based on the shooting configuration information, and generate a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo, where the shooting configuration information includes an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter, the third photo and the fourth photo corresponding to the shooting configuration information are photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information respectively, the third shooting parameter includes a third light transmission function and a third light intensity function, the fourth shooting parameter includes a fourth light transmission function and a fourth light intensity function, the third light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time, the fourth light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time, the third light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the third photo and time, the fourth light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the fourth photo and time, the third light transmission function is not a constant and / or the fourth light transmission function is not a constant, the third light transmission function is different from the fourth light transmission function and / or the third light intensity function is different from the fourth light intensity function, the exposure start time parameters in the shooting configuration information in the set of shooting configuration information are different from each other and / or the exposure duration parameters in the shooting configuration information in the set of shooting configuration information are different from each other, the third shooting parameters in the shooting configuration information in the set of shooting configuration information are all the same and the fourth shooting parameters in the shooting configuration information in the set of shooting configuration information are all the same or the third shooting parameters in the shooting configuration information in the set of shooting configuration information are different from each other and the fourth shooting parameters in the shooting configuration information in the set of shooting configuration information are different from each other; generate a depth image of the target scene based on the generated partial scene depth images of the target scene corresponding to each shooting configuration information pair in the set of preset shooting configuration information pairs.

[0022] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to a second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the second light transmittance function.

[0023] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to a second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function.

[0024] In some embodiments, the dynamic aperture is further configured to, under the control of a control device, make the wavelength of the light incident on the dynamic aperture different from the wavelength of the light emitted from the dynamic aperture, and the wavelength of the light emitted from the dynamic aperture is related to a preset wavelength sensitive range of an image sensor corresponding to the dynamic aperture.

[0025] In some embodiments, the control device is further configured to: before acquiring the first photo and the second photo, simultaneously perform a twenty-sixth control operation and a twenty-seventh control operation. The twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function. The twenty-seventh control operation includes controlling the first image sensor to capture a photo of a target scene, and determining the photo captured by the first image sensor as a background light photo; and generating a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information, including: generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0026] In some embodiments, generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: for each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo, establishing a fourth equation, a fifth equation, and a sixth equation, where the fourth equation is:

[0027]

[0028] Among them, S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area corresponding to the pixel point with coordinates (m, n) in the target scene under the background light, h1 is the first light transmission function, t is the time variable, t0 is the starting time of exposure, and τ is the preset exposure duration; The fifth equation is:

[0029]

[0030] Among them, R(m, n) is the reflectivity of the area corresponding to the pixel point with coordinates (m, n) in the target scene, and f1 is the first light intensity function. is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor. t d (m, n) is the duration for the light to reach the area corresponding to the pixel point with coordinates (m, n) in the target scene from the position where the illumination source is located. c is the speed of light, z(m, n) is the distance between the area corresponding to the pixel point with coordinates (m, n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side. Among them, the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel point with coordinates (m, n) in the target scene, and the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located. h1 is the first light transmission function, t0 is the starting time of exposure, τ is the preset exposure duration, and S1(m, n) is the pixel value of the pixel point with coordinates (m, n) in the first photo; The sixth equation is:

[0031]

[0032] Among them, f2 is the second light intensity function. is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the second photo in the target scene from the illumination source at time t and then returns to the first image sensor. h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; According to the fourth equation, the fifth equation, and the sixth equation, the seventh equation is obtained. The seventh equation is:

[0033]

[0034] Solve the seventh equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S b (m,n), S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; generate a depth image of the target scene based on the determined depth value of each pixel.

[0035] In some embodiments, the first light transmission function is related to the coordinates of each pixel in the first photo, and the second light transmission function is related to the coordinates of each pixel in the second photo.

[0036] In some embodiments, the dynamic aperture is an image intensifier.

[0037] In some embodiments, the dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

[0038] In some embodiments, the dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of a control device.

[0039] In a second aspect, embodiments of the present disclosure provide a method for generating a depth image, which is applied to a control device in a system for generating a depth image. The system for generating a depth image includes an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic aperture is configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device. The method includes: obtaining a first photo and a second photo, where the first photo and the second photo are respectively photos obtained by the image sensor photographing the target scene according to the first shooting configuration information and the second shooting configuration information. The first shooting configuration information includes a first light transmission function and a first light intensity function, and the second shooting configuration information includes a second light transmission function and a second light intensity function. The first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the first photo with respect to the image sensor for photographing the first photo and time. The second light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the second photo with respect to the image sensor for photographing the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the second photo and time. The first light transmission function is not a constant and / or the second light transmission function is not a constant, the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function; generating a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0040] In some embodiments, the method further includes: generating a three-dimensional model of the target scene based on the first photo, the second photo, and the depth image.

[0041] In some embodiments, at least one set of dynamic aperture and the corresponding image sensor includes a first dynamic aperture and the corresponding first image sensor; and obtaining the first photo and the second photo includes: simultaneously performing a first control operation, a second control operation, and a third control operation, wherein the first control operation includes controlling the light intensity of the first illumination light emitted by the illumination source to the target scene within a preset exposure duration according to a first light intensity function, wherein the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the second control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to a first light transmittance function, the third control operation includes controlling the first image sensor to take a photo of the target scene; determining the photo taken by the first image sensor as the first photo; simultaneously performing a fourth control operation, a fifth control operation, and a sixth control operation, wherein the fourth control operation includes controlling the light intensity of the second illumination light emitted by the illumination source to the target scene within a preset exposure duration according to a second light intensity function, wherein the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1, the fifth control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to a second light transmittance function, the sixth control operation includes controlling the first image sensor to take a photo of the target scene; determining the photo taken by the first image sensor as the second photo.

[0042] In some embodiments, at least one set of a dynamic aperture and a corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor, and a second dynamic aperture and a corresponding second image sensor; and obtaining a first photo and a second photo includes: simultaneously performing a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation, wherein the seventh control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function, the eighth control operation includes controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to a second light transmittance function, the ninth control operation includes controlling an illumination source to emit third illumination light to a target scene within the preset exposure duration, wherein a pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, the third illumination light includes first illumination light that reaches the first image sensor after being reflected by the target scene and passing through an optical system and the first dynamic aperture, the third illumination light includes second illumination light that reaches the second image sensor after being reflected by the target scene and passing through the optical system and the second dynamic aperture, an intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, an intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function, and the tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture photos of the target scene; determining the photo captured by the first image sensor as the first photo, and determining the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0043] In some embodiments, the optical system includes a filter configured to separate light of at least one wavelength belonging to a first preset wavelength set and light of at least one wavelength belonging to a second preset wavelength set, the third illumination light includes first illumination light of at least one wavelength belonging to the first preset wavelength set and second illumination light of at least one wavelength belonging to the second preset wavelength set, the first illumination light reaches the first image sensor after being reflected by the target scene and passing through the filter and the first dynamic aperture, and the second illumination light reaches the second image sensor after being reflected by the target scene and passing through the filter and the second dynamic aperture; and controlling the illumination source to emit the third illumination light to the target scene within the preset exposure duration includes: simultaneously performing an eleventh control operation and a twelfth control operation, wherein the eleventh control operation includes controlling the illumination source to emit the first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having an intensity conforming to the first light intensity function to the target scene within the preset exposure duration, and the twelfth control operation includes controlling the illumination source to emit the second illumination light including light of at least one wavelength belonging to the second preset wavelength set and having an intensity conforming to the second light intensity function to the target scene within the preset exposure duration.

[0044] In some embodiments, the optical system includes a polarizer; the polarizer is configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The third illumination light includes first illumination light with a polarization state of the first preset polarization state and second illumination light with a polarization state of the second preset polarization state. The first illumination light is reflected by the target scene, and through the polarizer and the first dynamic aperture, the corresponding light reaches the first image sensor. The second illumination light is reflected by the target scene, and through the polarizer and the second dynamic aperture, the corresponding light reaches the second image sensor; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a thirteenth control operation and a fourteenth control operation, where the thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state of the first preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a first light intensity function, and the fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state of the second preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a second light intensity function.

[0045] In some embodiments, the optical system includes a first beam splitting component, a first filter component, and a second filter component. The first filter component transmits light with at least one wavelength belonging to a first preset wavelength set, and the second filter component transmits light with at least one wavelength belonging to a second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor are located on a first side of the first beam splitting component, and the target scene, the second filter component, the second dynamic aperture, and the second image sensor are located on a second side of the first beam splitting component; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a fifteenth control operation and a sixteenth control operation, where the fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and with a light intensity conforming to a first light intensity function to the target scene within the preset exposure duration, and the sixteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and with a light intensity conforming to a second light intensity function to the target scene within the preset exposure duration.

[0046] In some embodiments, the optical system includes a second beam splitting component and a third filter component. The second beam splitting component is configured to split light and transmit light having at least one wavelength belonging to a first preset wavelength set. The third filter component is configured to transmit light having at least one wavelength belonging to a second preset wavelength set. The first dynamic aperture and the first image sensor are located on a first side of the second beam splitting component, and the target scene, the third filter component, the second dynamic aperture, and the second image sensor are located on a second side of the second beam splitting component. And controlling the illumination source to emit third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a seventeenth control operation and an eighteenth control operation, wherein the seventeenth control operation includes controlling the illumination source to emit first illumination light including light having at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to a first optical intensity function to the target scene within the preset exposure duration, and the eighteenth control operation includes controlling the illumination source to emit second illumination light including light having at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to a second optical intensity function to the target scene within the preset exposure duration.

[0047] In some embodiments, at least one set of dynamic aperture and the corresponding image sensor includes a first dynamic aperture and the corresponding first image sensor. The first image sensor is an image sensor array in which first image sensor pixel units and second image sensor pixel units are alternately arranged. The first image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a first preset wavelength set, and the second image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a second preset wavelength set; and obtaining a first photo and a second photo, including: simultaneously performing a nineteenth control operation, a twentieth control operation, a twenty-first control operation, and a twenty-second control operation. Among them, the nineteenth control operation includes controlling the illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set and with an intensity conforming to a first intensity function to the target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light of at least one wavelength belonging to the second preset wavelength set and with an intensity conforming to a second intensity function to the target scene within the preset exposure duration. Wherein, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light of wavelengths belonging to the first preset wavelength set within the preset exposure duration according to a first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light belonging to the second preset wavelength set within the preset exposure duration according to a second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; obtaining the photo taken by the first image sensor, and determining the obtained photo as a third photo; generating a first photo with the pixel values of each pixel point corresponding to the first image sensor pixel units in the third photo; generating a second photo with the pixel values of each pixel point corresponding to the second image sensor pixel units in the third photo.

[0048] In some embodiments, at least one set of dynamic apertures and corresponding image sensors includes a first dynamic aperture and a corresponding first image sensor and a second image sensor, the first image sensor is located on a first side of the first dynamic aperture, and the second image sensor and the target scene are located on a second side of the first dynamic aperture; and obtaining a first photo and a second photo includes: simultaneously performing a twenty-third control operation, a twenty-fourth control operation, and a twenty-fifth control operation, wherein the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to a first light transmittance function and a second light transmittance function respectively, the twenty-fourth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration, wherein, for the first illumination light included in the third illumination light, after being reflected by the target scene and transmitted through the first dynamic aperture, the corresponding light reaches the first image sensor, and for the second illumination light included in the third illumination light, after being reflected by the target scene and reflected by the first dynamic aperture, the corresponding light reaches the second image sensor, the light intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, the light intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function, the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, and the twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture photos of the target scene; determining the photo captured by the first image sensor as the first photo, and determining the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0049] In some embodiments, generating a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: for each pixel point with coordinates (m, n) in the first photo and the second photo, establishing a first equation and a second equation, wherein the first equation is:

[0050]

[0051] wherein, R(m, n) is the reflectivity of the area in the target scene corresponding to the pixel point with coordinates (m, n), f1 is the first light intensity function, t is a time variable, is the light intensity of the first illumination light when it reaches the area in the target scene corresponding to the pixel point with coordinates (m, n) in the first photo from the illumination source and then returns to the first image sensor at time t, t d (m, n) is the duration for the light to reach the area in the target scene corresponding to the pixel point with coordinates (m, n) from the position where the illumination source is located, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel at coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel at coordinates (m,n) in the target scene, and the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m,n) is the pixel value of the pixel at coordinates (m,n) in the first photo; The second equation is:

[0052]

[0053] where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel at coordinates (m,n) in the second photo in the target scene and then returns to the first image sensor at time t, h2 is the second light transmission function, and S2(m,n) is the pixel value of the pixel at coordinates (m,n) in the second photo; The third equation is obtained from the first equation and the second equation, and the third equation is:

[0054]

[0055] Solve the third equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; Based on the determined depth value of each pixel, generate the depth image of the target scene.

[0056] In some embodiments, the method further includes: for each shooting configuration information in the set of shooting configuration information, obtaining a third photo and a fourth photo corresponding to the shooting configuration information based on the shooting configuration information, and generating a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo, where the shooting configuration information includes an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter, the third photo and the fourth photo corresponding to the shooting configuration information are photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and respectively shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information, the third shooting parameter includes a third light transmission function and a third light intensity function, the fourth shooting parameter includes a fourth light transmission function and a fourth light intensity function, the third light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time, the fourth light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time, the third light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the third photo and time, the fourth light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the fourth photo and time, the third light transmission function is not a constant and / or the fourth light transmission function is not a constant, the third light transmission function is different from the fourth light transmission function and / or the third light intensity function is different from the fourth light intensity function, the exposure start time parameters in each shooting configuration information in the set of shooting configuration information are different from each other and / or the exposure duration parameters in each shooting configuration information in the set of shooting configuration information are different from each other, the third shooting parameters in each shooting configuration information in the set of shooting configuration information are the same and the fourth shooting parameters in each shooting configuration information in the set of shooting configuration information are the same or the third shooting parameters in each shooting configuration information in the set of shooting configuration information are different from each other and the fourth shooting parameters in each shooting configuration information in the set of shooting configuration information are different from each other; generating a depth image of the target scene based on the partial scene depth images of the target scene corresponding to each shooting configuration information pair in the set of preset shooting configuration information pairs.

[0057] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to a second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the second light transmittance function.

[0058] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to a second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function.

[0059] In some embodiments, the dynamic aperture is further configured to make the wavelength of the light incident on the dynamic aperture different from the wavelength of the light emitted from the dynamic aperture under the control of a control device, and the wavelength of the light emitted from the dynamic aperture is related to a preset wavelength sensitive range of an image sensor corresponding to the dynamic aperture. In some embodiments, before acquiring the first photo and the second photo, the method further includes: simultaneously performing a twenty-sixth control operation and a twenty-seventh control operation, the twenty-sixth control operation including controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function, the twenty-seventh control operation including controlling the first image sensor to capture a photo of a target scene, and determining the photo captured by the first image sensor as a background light photo; and generating a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information, including: generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0060] In some embodiments, generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: for each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo, establishing a fourth equation, a fifth equation, and a sixth equation, where the fourth equation is:

[0061]

[0062] Among them, S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area corresponding to the pixel point with coordinates (m, n) in the target scene under the background light, h1 is the first light transmission function, t is the time variable, t0 is the start time of exposure, and τ is the preset exposure duration; The fifth equation is:

[0063]

[0064] Among them, R(m, n) is the reflectivity of the area corresponding to the pixel point with coordinates (m, n) in the target scene, f1 is the first light intensity function, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the first photo in the target scene and then returns to the first image sensor at time t, t d (m, n) is the duration for the light to reach the area corresponding to the pixel point with coordinates (m, n) in the target scene from the position where the illumination source is located, c is the speed of light, z(m, n) is the distance between the area corresponding to the pixel point with coordinates (m, n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel point with coordinates (m, n) in the target scene, the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m, n) is the pixel value of the pixel point with coordinates (m, n) in the first photo; The sixth equation is:

[0065]

[0066] Among them, f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the second photo in the target scene and then returns to the first image sensor at time t, h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; According to the fourth equation, the fifth equation and the sixth equation, the seventh equation is obtained. The seventh equation is:

[0067]

[0068] Solve the seventh equation to obtain z(m, n), and determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n), where S b(m, n), S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; based on the determined depth value of each pixel point, a depth image of the target scene is generated.

[0069] In some embodiments, the first light transmission function is related to the coordinates of each pixel point in the first photo, and the second light transmission function is related to the coordinates of each pixel point in the second photo.

[0070] In some embodiments, the dynamic aperture is an image intensifier.

[0071] In some embodiments, the dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

[0072] In some embodiments, the dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of a control device.

[0073] In a third aspect, an embodiment of the present disclosure provides an apparatus for generating a depth image, which is applied to a control device in a system for generating a depth image. The system for generating a depth image includes an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic aperture is configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device. The apparatus includes: an acquisition unit configured to acquire a first photo and a second photo, where the first photo and the second photo are respectively photos obtained by the image sensor photographing the target scene according to the first shooting configuration information and the second shooting configuration information. The first shooting configuration information includes a first light transmission function and a first light intensity function, and the second shooting configuration information includes a second light transmission function and a second light intensity function. The first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the first photo with respect to the image sensor for photographing the first photo and time. The second light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the second photo with respect to the image sensor for photographing the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the second photo and time. The first light transmission function is not a constant and / or the second light transmission function is not a constant, the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function; a depth image generation unit configured to generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0074] In some embodiments, the apparatus further includes: a three-dimensional model generation unit configured to generate a three-dimensional model of a target scene based on a first photo, a second photo, and a depth image.

[0075] In some embodiments, at least one set of a dynamic aperture and a corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor; and the acquisition unit includes: a first control module that simultaneously performs a first control operation, a second control operation, and a third control operation, wherein the first control operation includes controlling the light intensity of first illumination light emitted by an illumination source towards a target scene within a preset exposure duration according to a first light intensity function, wherein the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the second control operation includes controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to a first light transmittance function, the third control operation includes controlling the first image sensor to capture a photo of the target scene; a first determination module configured to determine the photo captured by the first image sensor as a first photo; a second control module configured to simultaneously perform a fourth control operation, a fifth control operation, and a sixth control operation, wherein the fourth control operation includes controlling the light intensity of second illumination light emitted by the illumination source towards the target scene within the preset exposure duration according to a second light intensity function, wherein the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1, the fifth control operation includes controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to a second light transmittance function, the sixth control operation includes controlling the first image sensor to capture a photo of the target scene; a second determination module configured to determine the photo captured by the first image sensor as a second photo.

[0076] In some embodiments, at least one set of dynamic apertures and corresponding image sensors includes a first dynamic aperture and a corresponding first image sensor, and a second dynamic aperture and a corresponding second image sensor; and the acquisition unit includes: a third control module configured to simultaneously perform a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation, wherein the seventh control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function, the eighth control operation includes controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to a second light transmittance function, the ninth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration, wherein the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, the third illumination light includes first illumination light that reaches the first image sensor after being reflected by the target scene and passing through the optical system and the first dynamic aperture, and second illumination light that reaches the second image sensor after being reflected by the target scene and passing through the optical system and the second dynamic aperture, the light intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, the light intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function, and the tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture a photo of the target scene; a third determination module configured to determine the photo captured by the first image sensor as a first photo, and determine the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as a second photo.

[0077] In some embodiments, the optical system includes a filter configured to separate light of at least one wavelength belonging to a first preset wavelength set and light of at least one wavelength belonging to a second preset wavelength set, the third illumination light includes first illumination light of at least one wavelength belonging to the first preset wavelength set and second illumination light of at least one wavelength belonging to the second preset wavelength set, the first illumination light reaches the first image sensor after being reflected by the target scene and passing through the filter and the first dynamic aperture, and the second illumination light reaches the second image sensor after being reflected by the target scene and passing through the filter and the second dynamic aperture; and controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: simultaneously performing an eleventh control operation and a twelfth control operation, wherein the eleventh control operation includes controlling the illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having a light intensity conforming to the first light intensity function to the target scene within the preset exposure duration, and the twelfth control operation includes controlling the illumination source to emit second illumination light of at least one wavelength belonging to the second preset wavelength set and having a light intensity conforming to the second light intensity function to the target scene within the preset exposure duration.

[0078] In some embodiments, the optical system includes a polarizer; the polarizer is configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The third illumination light includes first illumination light with a polarization state being the first preset polarization state and second illumination light with a polarization state being the second preset polarization state. The first illumination light is reflected by the target scene, and the corresponding light reaches the first image sensor through the polarizer and the first dynamic aperture. The second illumination light is reflected by the target scene, and the corresponding light reaches the second image sensor through the polarizer and the second dynamic aperture; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, including: simultaneously performing a thirteenth control operation and a fourteenth control operation, where the thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state being the first preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a first light intensity function, and the fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state being the second preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to a second light intensity function.

[0079] In some embodiments, the optical system includes a first beam splitting component, a first filter component, and a second filter component. The first filter component transmits light with at least one wavelength belonging to a first preset wavelength set, and the second filter component transmits light with at least one wavelength belonging to a second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor are located on a first surface of the first beam splitting component, and the target scene, the second filter component, the second dynamic aperture, and the second image sensor are located on a second surface of the first beam splitting component; and controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, including: simultaneously performing a fifteenth control operation and a sixteenth control operation, where the fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and a light intensity conforming to a first light intensity function to the target scene within the preset exposure duration, and the sixteenth control operation includes controlling the illumination source to emit second illumination light with at least one wavelength belonging to the second preset wavelength set and a light intensity conforming to a second light intensity function to the target scene within the preset exposure duration.

[0080] In some embodiments, the optical system includes a second beam splitting component and a third filter component. The second beam splitting component is configured to split light and transmit light having at least one wavelength belonging to a first preset wavelength set. The third filter component is configured to transmit light having at least one wavelength belonging to a second preset wavelength set. The first dynamic aperture and the first image sensor are located on a first side of the second beam splitting component, and the target scene, the third filter component, the second dynamic aperture, and the second image sensor are located on a second side of the second beam splitting component. And controlling the illumination source to emit third illumination light to the target scene within a preset exposure duration includes: simultaneously performing a seventeenth control operation and an eighteenth control operation, wherein the seventeenth control operation includes controlling the illumination source to emit first illumination light including light having at least one wavelength belonging to the first preset wavelength set and an optical intensity conforming to a first optical intensity function to the target scene within the preset exposure duration, and the eighteenth control operation includes controlling the illumination source to emit second illumination light including light having at least one wavelength belonging to the second preset wavelength set and an optical intensity conforming to a second optical intensity function to the target scene within the preset exposure duration.

[0081] In some embodiments, at least one set of a dynamic aperture and a corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor. The first image sensor is an image sensor array in which first image sensor pixel units and second image sensor pixel units are alternately arranged. The first image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a first preset wavelength set, and the second image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to a second preset wavelength set; and the acquisition unit includes: a fourth control module configured to simultaneously perform a nineteenth control operation, a twentieth control operation, a twenty-first control operation, and a twenty-second control operation. Among them, the nineteenth control operation includes controlling an illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to a first optical intensity function to a target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light including light of at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to a second optical intensity function to the target scene within the preset exposure duration. Among them, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light of a wavelength belonging to the first preset wavelength set within the preset exposure duration according to a first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light of a wavelength belonging to the second preset wavelength set within the preset exposure duration according to a second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; a fourth determination module configured to obtain the photo taken by the first image sensor and determine the obtained photo as a third photo; a first generation module configured to generate a first photo with the pixel values of each pixel point corresponding to the first image sensor pixel units collected in the third photo; a second generation module configured to generate a second photo with the pixel values of each pixel point corresponding to the second image sensor pixel units collected in the third photo.

[0082] In some embodiments, at least one set of a dynamic aperture and a corresponding image sensor includes a first dynamic aperture and a corresponding first image sensor, and a second image sensor. The first image sensor is located on a first side of the first dynamic aperture, and the second image sensor and the target scene are located on a second side of the first dynamic aperture; and the obtaining unit includes: a fifth control module configured to simultaneously perform a twenty-third control operation, a twenty-fourth control operation, and a twenty-fifth control operation. Wherein, the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to a first light transmittance function and a second light transmittance function respectively. The twenty-fourth control operation includes controlling a light source to emit third illumination light to the target scene within the preset exposure duration. Wherein, a first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing corresponding light to reach the first image sensor. A second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function. The pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture a photo of the target scene; a fifth determination module configured to determine the photo captured by the first image sensor as a first photo, and determine the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as a second photo.

[0083] In some embodiments, the above-mentioned depth image generation unit includes: a first equation establishment module configured to establish a first equation and a second equation for each pixel point with coordinates (m, n) in the first photo and the second photo. Wherein, the first equation is:

[0084]

[0085] Wherein, R(m, n) is the reflectance of the area in the target scene corresponding to the pixel point with coordinates (m, n), f1 is the first light intensity function, t is a time variable, is the light intensity of the first illumination light when it reaches the area in the target scene corresponding to the pixel point with coordinates (m, n) in the first photo from the light source and then returns to the first image sensor at time t, t d (m, n) is the duration for the light to reach the area in the target scene corresponding to the pixel point with coordinates (m, n) from the position where the light source is located, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel at coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, and α is the angle between the first side and the second side. Here, the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel at coordinates (m,n) in the target scene, and the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located. h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m,n) is the pixel value of the pixel at coordinates (m,n) in the first photo; The second equation is:

[0086]

[0087] where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel at coordinates (m,n) in the second photo in the target scene and then returns to the first image sensor at time t, h2 is the second light transmission function, and S2(m,n) is the pixel value of the pixel at coordinates (m,n) in the second photo; The second equation establishment module is configured to obtain a third equation according to the first equation and the second equation. The third equation is:

[0088]

[0089] The first equation solving module is configured to solve the third equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; The third generation module ( Figure 3 not shown in the figure) is configured to generate a depth image of the target scene based on the determined depth value of each pixel.

[0090] In some embodiments, the device further includes a partial scene depth image generation unit configured to, for each shooting configuration information in the set of shooting configuration information, obtain a third photo and a fourth photo corresponding to the shooting configuration information based on the shooting configuration information, and generate a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo, where the shooting configuration information includes an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter, and the third photo and the fourth photo corresponding to the shooting configuration information are photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and respectively shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information. The third shooting parameter includes a third light transmission function and a third light intensity function, and the fourth shooting parameter includes a fourth light transmission function and a fourth light intensity function. The third light transmission function is used to characterize the correspondence relationship between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time. The fourth light transmission function is used to characterize the correspondence relationship between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time. The third light intensity function is used to characterize the correspondence relationship between the light intensity of the light emitted by the illumination source to the target scene when shooting the third photo and time. The fourth light intensity function is used to characterize the correspondence relationship between the light intensity of the light emitted by the illumination source to the target scene when shooting the fourth photo and time. The third light transmission function is not a constant and / or the fourth light transmission function is not a constant. The third light transmission function is different from the fourth light transmission function and / or the third light intensity function is different from the fourth light intensity function. The exposure start time parameters in each shooting configuration information in the set of shooting configuration information are different from each other and / or the exposure duration parameters in each shooting configuration information in the set of shooting configuration information are different from each other. The third shooting parameters in each shooting configuration information in the set of shooting configuration information are the same and the fourth shooting parameters in each shooting configuration information in the set of shooting configuration information are the same, or the third shooting parameters in each shooting configuration information in the set of shooting configuration information are different from each other and the fourth shooting parameters in each shooting configuration information in the set of shooting configuration information are different from each other; a target scene depth image generation unit configured to generate a depth image of the target scene based on the partial scene depth images of the target scene corresponding to each shooting configuration information pair in the set of preset shooting configuration information pairs that are generated.

[0091] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the second light transmittance function.

[0092] In some embodiments, the first dynamic aperture and the first image sensor are the same device; and controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to the second light transmittance function includes: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function.

[0093] In some embodiments, the dynamic aperture is further configured to, under the control of a control device, make the wavelength of the light incident on the dynamic aperture different from the wavelength of the light emitted from the dynamic aperture, and the wavelength of the light emitted from the dynamic aperture is related to the preset wavelength sensitive range of the image sensor corresponding to the dynamic aperture.

[0094] In some embodiments, the apparatus further includes: a background light photo determination unit configured to, before acquiring the first photo and the second photo, simultaneously perform a twenty-sixth control operation and a twenty-seventh control operation, the twenty-sixth control operation including controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function, the twenty-seventh control operation including controlling the first image sensor to capture a photo of a target scene, and determining the photo captured by the first image sensor as a background light photo; and the depth image generation unit is further configured to generate a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0095] In some embodiments, the depth image generation unit includes: a third equation establishment module configured to establish a fourth equation, a fifth equation, and a sixth equation for each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo, where the fourth equation is:

[0096]

[0097] where Sb (m, n) is the pixel value of the pixel at coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area in the target scene corresponding to the pixel at coordinates (m, n) under the background light, h1 is the first light transmission function, t is the time variable, t0 is the start time of exposure, and τ is the preset exposure duration; The fifth equation is:

[0098]

[0099] where R(m, n) is the reflectivity of the area in the target scene corresponding to the pixel at coordinates (m, n), and f1 is the first light intensity function. is the light intensity when the first illumination light reaches the area in the target scene corresponding to the pixel at coordinates (m, n) in the first photo from the illumination source at time t and then returns to the first image sensor. t d (m, n) is the duration for the light to travel from the position of the illumination source to the area in the target scene corresponding to the pixel at coordinates (m, n). c is the speed of light, z(m, n) is the distance between the area in the target scene corresponding to the pixel at coordinates (m, n) in the generated depth image and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side. Here, the first side is the line segment connecting the position of the first image sensor and the area in the target scene corresponding to the pixel at coordinates (m, n), and the second side is the line segment connecting the position of the first image sensor and the position of the illumination source. h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m, n) is the pixel value of the pixel at coordinates (m, n) in the first photo; The sixth equation is:

[0100]

[0101] where f2 is the second light intensity function. is the light intensity when the second illumination light reaches the area in the target scene corresponding to the pixel at coordinates (m, n) in the second photo from the illumination source at time t and then returns to the first image sensor. h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel at coordinates (m, n) in the second photo; The fourth equation establishment module ( Figure 3 not shown in the figure) is configured to obtain the seventh equation according to the fourth equation, the fifth equation, and the sixth equation. The seventh equation is:

[0102]

[0103] The second equation solving module is configured to solve the seventh equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image, where S b (m,n), S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; the fourth generation module is configured to generate a depth image of the target scene based on the determined depth value of each pixel.

[0104] In some embodiments, the first light transmission function is related to the coordinates of each pixel in the first photo, and the second light transmission function is related to the coordinates of each pixel in the second photo.

[0105] In some embodiments, the dynamic aperture is an image intensifier.

[0106] In some embodiments, the dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

[0107] In some embodiments, the dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of a control device.

[0108] In a fourth aspect, embodiments of the present disclosure provide a camera, where the camera includes an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors, and the dynamic aperture is configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device.

[0109] In some embodiments, the dynamic aperture is an image intensifier.

[0110] In some embodiments, the dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

[0111] In a fifth aspect, embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method described in any implementation manner of the second aspect.

[0112] In a sixth aspect, embodiments of the present disclosure provide a computer - readable storage medium storing a computer program thereon, where the computer program, when executed by one or more processors, implements the method described in any implementation manner of the second aspect.

[0113] In the prior art, there are the following various methods for measuring the depth of a scene.

[0114] First, stereoscopic vision, binocular imaging, and triangulation: As the distance increases, the distance resolution of this method becomes worse and worse;

[0115] Second, using computer vision based on image and object projection: This method requires a large amount of known object model data, and uses artificial intelligence for image recognition and processing to establish a three-dimensional space model, making it difficult to process previously unknown scenarios;

[0116] Third, illuminating the scene with a pre-determined illumination pattern (structured light), analyzing the scene using the illumination, and finding the depth information using the previously obtained calibration information: This method is only feasible at close range.

[0117] Fourth, using a time-of-flight camera (multi-photodetector) to measure distance: In practice, this method has a short distance and is limited by the propagation distance of light within the modulation signal period.

[0118] Fifth, using a Pockels cell to adjust the transmission of the signal received from the scene to determine the distance of the object in the scene: In this method, the Pockels cell uses high voltage electricity to adjust the polarization of light, and the available field of view angle is relatively small. Although multiple optical compensators can be added, it increases the complexity of the system.

[0119] Among the above various methods for measuring the distance of an object in a scene by imaging, in most imaging systems, the aperture is constant after it is opened, and it cannot reflect the time information of the light arriving at different moments during the exposure. Moreover, most aperture switches are controlled very slowly, at the millisecond level or slower, which does not conform to the characteristics of the high-speed round-trip of light.

[0120] The systems, methods, and devices for generating depth images provided by embodiments of the present disclosure include a lighting source, a control device, and at least one set of dynamic apertures and corresponding image sensors in the system for generating depth images. The dynamic apertures are configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device, and the control device is configured to: obtain a first photo and a second photo, where the first photo and the second photo are respectively the photos obtained by the image sensor capturing a target scene according to first shooting configuration information and second shooting configuration information. The first shooting configuration information includes a first light transmittance function and a first light intensity function, and the second shooting configuration information includes a second light transmittance function and a second light intensity function. The first light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the first photo with respect to the image sensor for capturing the first photo and time. The second light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the second photo with respect to the image sensor for capturing the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the lighting source to the target scene when capturing the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the lighting source to the target scene when capturing the second photo and time. The first light transmittance function is not a constant and / or the second light transmittance function is not a constant, the first light transmittance function is different from the second light transmittance function and / or the first light intensity function is different from the second light intensity function; and generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information. The technical effects of the systems for generating depth images provided by embodiments of the present disclosure may include, but are not limited to, the following:

[0121] First, by introducing a dynamic aperture to dynamically change the light transmittance and a lighting source to dynamically change the light intensity, that is, using different shooting configuration information to capture the first photo and the second photo, and generating a depth image of the target scene according to the obtained first photo, second photo, and corresponding shooting configuration information. First, the cost of the dynamic aperture is not high. Second, any currently commercially available lighting source and image sensor (e.g., currently commercially available cameras) and various lighting sources plus image sensors developed in the future (e.g., cameras developed in the future) can be used in the above systems for generating depth images. Therefore, compared with existing methods for generating depth images, there are no separate requirements for the lighting source and the image sensor, reducing the economic cost of generating depth images.

[0122] Second, the image resolution of any currently commercially available ordinary camera is generally higher than that of various imaging devices used in measuring the distance of a scene by imaging methods. Therefore, compared with existing methods for generating depth images, the image resolution of the generated depth image is improved. Description of the Drawings

[0123] Other features, objects, and advantages of the present disclosure will become more apparent from the following detailed description of non - limiting embodiments read in conjunction with the accompanying drawings:

[0124] Figure 1 It is a system architecture diagram of an embodiment of a system for generating a depth image according to the present disclosure;

[0125] Figure 2A It is a flowchart of an embodiment of a method for generating a depth image according to the present disclosure;

[0126] Figure 2B It is a decomposed flowchart of an embodiment of step 201 according to the present disclosure;

[0127] Figure 2C It is a decomposed flowchart of another embodiment of step 201 according to the present disclosure;

[0128] Figure 2D A schematic structural diagram of an embodiment of an optical system according to the present disclosure;

[0129] Figure 2E A schematic structural diagram of another embodiment of an optical system according to the present disclosure;

[0130] Figure 2F A schematic structural diagram of yet another embodiment of an optical system according to the present disclosure;

[0131] Figure 2G A schematic structural diagram of still another embodiment of an optical system according to the present disclosure;

[0132] Figure 2H It is a decomposed flowchart of yet another embodiment of step 201 according to the present disclosure;

[0133] Figure 2I A schematic structural diagram of an embodiment of at least one set of dynamic apertures and corresponding image sensors according to the present disclosure;

[0134] Figure 2J It is a decomposed flowchart of still another embodiment of step 201 according to the present disclosure;

[0135] Figure 2K A schematic structural diagram of another embodiment of at least one set of dynamic apertures and corresponding image sensors according to the present disclosure;

[0136] Figure 2L It is a decomposed flowchart of an embodiment of step 202 according to the present disclosure;

[0137] Figure 2M It is according to the present disclosure of t d A schematic diagram of an embodiment of the calculation principle of (m,n);

[0138] Figure 2N is a decomposition flowchart of an embodiment of step 202 according to the present disclosure;

[0139] Figure 3 is a schematic structural diagram of an embodiment of an apparatus for generating a depth image according to the present disclosure;

[0140] Figure 4 is a timing diagram of an embodiment of a system for generating a depth image according to the present disclosure;

[0141] Figure 5 is a schematic structural diagram of a camera according to an embodiment of the present disclosure;

[0142] Figure 6 is a schematic structural diagram of a computer system of a control device suitable for implementing an embodiment of the present disclosure. Detailed implementation manners

[0143] The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. In addition, it should be noted that, for the sake of description, only the parts related to the relevant invention are shown in the drawings.

[0144] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the drawings and embodiments.

[0145] Figure 1 Shows a system architecture 100 of an embodiment of a system for generating a depth image according to the present disclosure.

[0146] As Figure 1 shown, the system architecture 100 may include an illumination source 101, an optical system 102, a network 103, a control device 104, a network 105, and at least one group of dynamic apertures and corresponding image sensors 106. Among them, at least one group of dynamic apertures and corresponding image sensors 106 includes: a group 1061 composed of a dynamic aperture 1061A and a corresponding image sensor 1061B, a group 1062 composed of a dynamic aperture 1062A and a corresponding image sensor 1062B,..., and a group 106N composed of a dynamic aperture 106NA and a corresponding image sensor 106NB, where N is a natural number.

[0147] It should be noted that any two of the dynamic apertures 1061A, 1062A, …, 106NA can be two different dynamic apertures or the same one dynamic aperture. Any two of the image sensors 1061B, 1062B, …, 106NB can be two different image sensors or the same one image sensor.

[0148] Here, the illumination source 101 can be various devices with illumination functions, and the present disclosure does not make specific limitations thereto. For example, the illumination source can include gas discharge light sources, semiconductor light sources, and laser light sources, etc., and its products mainly can include gas discharge lamps, semiconductor fluorescent lamps (such as LED light-emitting diodes), semiconductor lasers, solid-state lasers, gas lasers, and so on. It should be noted that the illumination source 101 here can be turned on or off under the control of the control device 104, and dynamically change the wavelength, light intensity, etc. of the emitted light.

[0149] Here, the optical system 102 refers to a system composed of at least one optical element combined in a certain order. For example, the at least one optical element can include but is not limited to lenses, mirrors, prisms, and diaphragms, etc. The optical system 102 is configured such that the reflected light after the light emitted by the illumination source is reflected by the target scene forms an image on the image sensor.

[0150] In some alternative implementation manners of this embodiment, the optical system 102 can include a filter, and here, the filter is configured to only transmit the light emitted by the illumination source 101.

[0151] In some alternative implementation manners of this embodiment, the optical system 102 may also not include a filter.

[0152] In various commercially available cameras currently, the aperture can only control opening and closing, that is, only control the exposure start time and exposure end time of the aperture. Between the aperture opening and the aperture closing, the light transmittance of the aperture remains unchanged. However, the dynamic aperture in the present disclosure can be various aperture devices with the function of dynamically changing the light transmittance, exposure start time, and exposure end time under the control of the control device.

[0153] In some alternative implementation manners of this embodiment, the dynamic aperture can be an image intensifier. An image intensifier is an optoelectronic device that can turn an optical image with very low brightness into an image with sufficient brightness. The control device 104 can control the working start time, termination time, and the overall light transmittance of the image intensifier by controlling the photocathode voltage, microchannel plate voltage, and phosphor sheet voltage of the image intensifier over time.

[0154] In some alternative implementation manners of this embodiment, the dynamic aperture may also be a Fabry - Pérot interferometer containing a nonlinear crystal. The control device 104 can control the light transmittance of the dynamic aperture by controlling the voltage applied to the nonlinear crystal in the Fabry - Pérot interferometer. Here, the surface of the nonlinear crystal contained in the Fabry - Pérot interferometer may be coated with a transparent conductive film, so that the surface of the nonlinear crystal can become a transparent electrode. And the light transmittance of the Fabry - Pérot interferometer can be calculated by the following formula:

[0155]

[0156] where: T is the calculated light transmittance of the Fabry - Pérot interferometer. r is the reflectivity of the two glass plates (for example, mirrors) that make up the Fabry - Pérot interferometer, is the optical phase difference between the light passing through the Fabry - Pérot interferometer and the light returning from the first glass plate of the Fabry - Pérot interferometer to reach the second glass plate.

[0157] When the light incident on the two glass plates of the Fabry - Pérot interferometer is normally incident, the optical phase difference can be calculated according to the following formula:

[0158]

[0159] where: n is the refractive index of the nonlinear crystal in the Fabry - Pérot interferometer, L is the distance between the two glass plates that make up the Fabry - Pérot interferometer, and λ is the wavelength of the light incident on the Fabry - Pérot interferometer.

[0160] When the light incident on the two glass plates of the Fabry - Pérot interferometer is non - normally incident, the optical phase difference can be calculated according to the following formula:

[0161]

[0162] where: θ is the incident angle of the light incident on the two glass plates of the Fabry - Pérot interferometer (measured inside the medium).

[0163] As can be seen from the above formula, when the refractive index n of the nonlinear crystal is changed, the optical phase difference also changes accordingly, and then the light transmittance T of the Fabry - Pérot interferometer changes. Therefore, the control device 104 can change the light transmittance of the dynamic aperture by controlling the refractive index of the nonlinear crystal, that is, by controlling the voltage applied to the nonlinear crystal to change the refractive index of the nonlinear crystal to control the light transmittance of the dynamic aperture.

[0164] For example, an electro-optic crystal (z-cut LiNbO3) is inserted into the Fabry-Perot interferometer as a nonlinear crystal. The control device 104 can obtain a time-varying phase difference by modulating the voltage applied to the electro-optic crystal, thereby achieving the effect of controlling the light transmittance of the dynamic aperture. With an appropriate voltage, the Fabry-Perot interferometer can achieve a light transmittance close to 1 for the normally incident light of a specified wavelength. In some alternative implementation manners of this embodiment, the dynamic aperture can also be a Fabry-Perot interferometer containing a piezoelectric crystal. The control device 104 can change the distance between two parallel glass plates in the Fabry-Perot interferometer by controlling the voltage applied to the piezoelectric crystal in the Fabry-Perot interferometer, so as to control the optical phase and further change the light transmittance of the dynamic aperture.

[0165] The network 103 is a medium for providing a communication link between the illumination source 101 and the control device 104. The network 103 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0166] The user can use the control device 104 to interact with the illumination source 101 through the network 103 to achieve the control of the illumination source 101. For example, controlling the illumination source to turn on or off, controlling the illumination source to emit illumination light at what wavelength and what light intensity during what time period, etc.

[0167] The network 105 is a medium for providing a communication link between the control device 104 and at least one group of dynamic apertures and the corresponding image sensors 106. The network 105 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0168] The user can use the control device 104 to interact with at least one group of dynamic apertures and the corresponding image sensors 106 through the network 105 to achieve the control of each dynamic aperture and the image sensor. For example, controlling the exposure start time, exposure end time, and light transmittance of the dynamic aperture, and controlling the image sensor to take pictures, etc.

[0169] The control device 104 can be either hardware or software. When the control device 104 is hardware, it can be an electronic device with various control functions. For example, the control device 104 can be a separately provided controller, such as a Programmable Logic Controller (PLC), a Field Programmable Gate Array (FPGA), a single-chip microcomputer, an industrial control computer, etc.; it can also be a device composed of other electronic devices with input / output ports and arithmetic control functions; it can also be a computer device installed with control applications having functions of lighting source control, dynamic aperture control, and image sensor control. When the control device 104 is software, it can be installed in the above-listed electronic devices. It can be implemented as multiple software or software modules (for example, used to provide lighting source control function, dynamic aperture control function, and image sensor control function), or it can be implemented as a single software or software module. No specific limitation is made here.

[0170] It should be noted that the method for generating a depth image provided by the embodiments of the present disclosure is generally executed by the control device 104. Correspondingly, the device for generating a depth image is generally provided in the control device 104.

[0171] It should be understood that Figure 1 the numbers of the lighting source, the control device, the network, the dynamic aperture, and the image sensor in

[0172] Continue to refer to Figure 2A which shows a flow 200 of an embodiment of the method for generating a depth image according to the present disclosure, applied to a control device in a system for generating a depth image. The system for generating a depth image includes a lighting source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic aperture can dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device. The method for generating a depth image includes the following steps:

[0173] Step 201, obtain a first photo and a second photo.

[0174] In this embodiment, the execution subject of the method for generating a depth image (such as Figure 1 the control device shown) can obtain the first photo and the second photo in various implementation manners.

[0175] Here, the first photo and the second photo are respectively the photos obtained by an image sensor in a system for generating a depth image by photographing a target scene according to first shooting configuration information and second shooting configuration information. The first shooting configuration information may include a first light transmission function and a first light intensity function. The second shooting configuration information may include a second light transmission function and a second light intensity function. The first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for taking the first photo with respect to the image sensor for taking the first photo and time. The second light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for taking the second photo with respect to the image sensor for taking the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when taking the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when taking the second photo and time.

[0176] In this embodiment, the first light transmission function is not a constant and / or the second light transmission function is not a constant. That the first light transmission function is not a constant means that the light transmittance of the dynamic aperture for taking the first photo during the exposure of taking the first photo is not fixed. That the second light transmission function is not a constant means that the light transmittance of the dynamic aperture for taking the second photo during the exposure of taking the second photo is not fixed. And that the first light transmission function is not a constant and / or the second light transmission function is not a constant means that at least one of the light transmittances, i.e., the light transmittance of the dynamic aperture for taking the first photo during the exposure of taking the first photo and the light transmittance of the dynamic aperture for taking the second photo during the exposure of taking the second photo, is not fixed.

[0177] In this embodiment, the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function. Among them, that the first light transmission function is different from the second light transmission function means that the correspondence between the light transmittance of the dynamic aperture for taking the first photo during the exposure of taking the first photo and time is different from the correspondence between the light transmittance of the dynamic aperture for taking the second photo with respect to the image sensor for taking the second photo and time. That the first light intensity function is different from the second light intensity function means that the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when taking the first photo and time is different from the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when taking the second photo and time.

[0178] In some alternative implementation manners of this embodiment, the difference between the first light transmission function and the second light transmission function means that the absolute value of the difference between the light transmittance of the dynamic aperture for taking the first photo during the exposure period of taking the first photo and the light transmittance of the dynamic aperture for taking the second photo during the exposure period of taking the second photo is integrated during the exposure period of taking the first photo / during the exposure period of taking the second photo and is greater than the fourth preset ratio of the sum of the integral of the absolute value of the light transmittance of the dynamic aperture for taking the first photo during the exposure period of taking the first photo and the integral of the absolute value of the light transmittance of the dynamic aperture for taking the second photo during the exposure period of taking the second photo. In practice, the fourth preset ratio may be 0.1.

[0179] In some alternative implementation manners of this embodiment, the difference between the first light intensity function and the second light intensity function means that the absolute value of the difference between the light intensity of the illumination source emitted from the illumination source in the system for generating the depth image when taking the first photo and the light intensity of the illumination source emitted from the illumination source in the system for generating the depth image when taking the second photo is integrated during the exposure period of taking the first photo / during the exposure period of taking the second photo and is greater than the fifth preset ratio of the sum of the integral of the absolute value of the light intensity of the illumination source emitted from the illumination source in the system for generating the depth image when taking the first photo during the exposure period of taking the first photo and the integral of the absolute value of the light intensity of the illumination source emitted from the illumination source in the system for generating the depth image when taking the second photo during the exposure period of taking the second photo. In practice, the fifth preset ratio may be 0.1. Here, the first photo and the second photo may be taken by the image sensor in the system for generating the depth image, and the image sensor for taking the first photo may be the same as or different from the image sensor for taking the second photo. In this way, the above-mentioned execution entity can obtain the first photo from the image sensor for taking the first photo and obtain the second photo from the image sensor for taking the second photo.

[0180] Step 202: Generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0181] Since the first photo is a photo of the target scene taken using the first configuration information, and the second photo is a photo of the target scene taken using the second configuration information, in order to obtain the depth image of the target scene, the above-mentioned execution entity (such as Figure 1The control device shown can generate a depth image of the target scene in various implementation manners based on the first photo and the second photo obtained in step 201, as well as the first configuration information used for taking the first photo and the second configuration information used for taking the second photo. That is, determine the distance between the region corresponding to each pixel point in the depth image in the target scene and the target image sensor. Here, the target image sensor can be the image sensor that took the first photo, or the image sensor that took the second photo, or a hypothetical virtual image sensor that took the depth image.

[0182] In some alternative implementation manners of this embodiment, the depth image can be an image whose pixel values of each pixel point include only one depth value channel. Among them, the pixel value of the depth value channel of each pixel point is used to represent the distance between the region corresponding to this pixel point in the target scene and the target image sensor.

[0183] In some alternative implementation manners of this embodiment, the depth image can also be an image whose pixel values of each pixel point include both pixel values of a color value channel and pixel values of a depth value channel. Among them, the pixel value of the color value channel of each pixel point is used to represent the color of the region corresponding to this pixel point in the target scene. Here, the color value channel can be one channel or more than one channel. For example, the color value channel can include an R channel for representing the red color value, a G channel for representing the green color value, and a B channel for representing the blue color value. And the pixel value of the depth value channel of each pixel point is used to represent the distance between the region corresponding to this pixel point in the target scene and the target image sensor.

[0184] In some cases, this embodiment can also have the following alternative implementation manners:

[0185] Alternative implementation manner (1): At least one set of dynamic aperture and the corresponding image sensor in the system for generating the depth image can include a first dynamic aperture and the corresponding first image sensor. In this way, step 201 can include sub-steps 2011A to 2014A as Figure 2B shown. Please refer to Figure 2B , which shows a decomposed flowchart of an embodiment of step 201 according to the present disclosure.

[0186] Sub-step 2011A, perform the first control operation, the second control operation, and the third control operation simultaneously.

[0187] Among them:

[0188] The first control operation includes controlling the intensity of the first illumination light emitted by the illumination source towards the target scene within a preset exposure duration according to a first light intensity function. Among them, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, and the first preset ratio is greater than zero and less than or equal to 1. For example, the first preset ratio can be one-tenth.

[0189] The second control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a first light transmittance function.

[0190] The third control operation includes controlling the first image sensor to take a photo of the target scene.

[0191] Sub-step 2012A: Determine the photo taken by the first image sensor as the first photo.

[0192] That is, obtain the photo taken by the first image sensor in sub-step 2011A and determine the obtained photo as the first photo.

[0193] Sub-step 2013A: Simultaneously perform the fourth control operation, the fifth control operation, and the sixth control operation.

[0194] Among them:

[0195] The fourth control operation includes controlling the intensity of the second illumination light emitted by the illumination source towards the target scene within a preset exposure duration according to a second light intensity function. Among them, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, and the second preset ratio is greater than zero and less than or equal to 1. Here, the second preset ratio can be the same as or different from the first preset ratio. For example, the second preset ratio can be one-tenth.

[0196] The fifth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to a second light transmittance function.

[0197] The sixth control operation includes controlling the first image sensor to take a photo of the target scene.

[0198] Sub-step 2014A: Determine the photo taken by the first image sensor as the second photo.

[0199] From the above description, it can be seen that the first photo and the second photo are two photos taken by the same image sensor (i.e., the first image sensor) at different times. The first light intensity function and the first light transmittance function are used during the process of taking the first photo, and the second light intensity function and the second light transmittance function are used during the process of taking the second photo. The first light transmittance function is not a constant and / or the second light transmittance function is not a constant, and the first light transmittance function is different from the second light transmittance function and / or the first light intensity function is different from the second light intensity function.

[0200] It should be noted that in the above optional implementation (i), in addition to including the first dynamic aperture and the corresponding first image sensor, other dynamic apertures and corresponding image sensors may also be included, and the first dynamic aperture and the corresponding first image sensor may be any dynamic aperture and corresponding image sensor in the system for generating depth images. Here, only the first image sensor is used as an example for illustration.

[0201] Optional implementation (ii): At least one group of dynamic apertures and corresponding image sensors in the system for generating depth images may include the first dynamic aperture and the corresponding first image sensor, and the second dynamic aperture and the corresponding second image sensor. Among them, the first image sensor and the second image sensor may be two independent image sensor chips, or the first image sensor and the second image sensor may also be two non-overlapping parts of the same image sensor chip. In this way, step 201 may further include sub-steps 2011B to 2012B as Figure 2C shown. Please refer to Figure 2C , which shows a decomposed flowchart of another embodiment of step 201 according to the present disclosure.

[0202] Sub-step 2011B, simultaneously perform the seventh control operation, the eighth control operation, the ninth control operation, and the tenth control operation.

[0203] Among them:

[0204] The seventh control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function.

[0205] The eighth control operation includes controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function.

[0206] The ninth control operation includes controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration. Among them, the pulse width of the third illumination light is less than the third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. And the first illumination light included in the third illumination light reaches the first image sensor after being reflected by the target scene and passing through the optical system and the first dynamic aperture, and the second illumination light included in the third illumination light reaches the second image sensor after being reflected by the target scene and passing through the optical system and the second dynamic aperture. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function.

[0207] The tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0208] Sub-step 2012B: Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0209] As can be seen from the above description, in the optional implementation (ii), by introducing an optical system, and using the first image sensor and the second image sensor to respectively capture the target scene at the same time to obtain the first photo and the second photo. The first light transmission function is used to control the light transmittance of the first dynamic aperture corresponding to the first image sensor that captures the first photo during the process of capturing the first photo. The second light transmission function is used to control the light transmittance of the second dynamic aperture corresponding to the second image sensor that captures the second photo during the process of capturing the second photo. During the process of capturing the first photo and the second photo, the first illumination light included in the third illumination light emitted by the illumination source reaches the first image sensor after being reflected by the target scene and passing through the optical system and the first dynamic aperture, and the second illumination light included in the third illumination light reaches the second image sensor after being reflected by the target scene and passing through the optical system and the second dynamic aperture. Here, it is also necessary to ensure that the first light transmission function is not a constant and / or the second light transmission function is not a constant, and the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function.

[0210] It should be noted that in addition to including the first dynamic aperture and the corresponding first image sensor, and the second dynamic aperture and the corresponding second image sensor in the above optional implementation (ii), other dynamic apertures and corresponding image sensors may also be included. Moreover, the first dynamic aperture and the corresponding first image sensor, and the second dynamic aperture and the corresponding second image sensor can be any two different sets of dynamic apertures and corresponding image sensors in the system for generating depth images. Here, only the first dynamic aperture and the corresponding first image sensor, and the second dynamic aperture and the corresponding second image sensor are used as examples for illustration.

[0211] Alternative implementation (3): Based on the above alternative implementation (2), the optical system in the system for generating a depth image may include a filter. Here, the number of narrowband green filters included in the optical system may be one or more than one. The filter may separate light with at least one wavelength belonging to a first preset wavelength set and light with at least one wavelength belonging to a second preset wavelength set. The third illumination light in the above alternative implementation (2) may include a first illumination light including light with at least one wavelength belonging to the first preset wavelength set and a second illumination light including light with at least one wavelength belonging to the second preset wavelength set. The first illumination light reaches the first image sensor after being reflected by the target scene and passing through the above filter and the first dynamic aperture, and the second illumination light reaches the second image sensor after being reflected by the target scene and passing through the above filter and the second dynamic aperture. Thus, the ninth control operation in the above alternative implementation (2), that is, controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, may be performed as follows:

[0212] Simultaneously perform an eleventh control operation and a twelfth control operation. Among them:

[0213] The eleventh control operation includes controlling the illumination source to emit a first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having a light intensity conforming to a first light intensity function to the target scene within a preset exposure duration.

[0214] The twelfth control operation includes controlling the illumination source to emit a second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having a light intensity conforming to a second light intensity function to the target scene within a preset exposure duration.

[0215] For ease of understanding alternative implementation (3), please refer to Figure 2D . Figure 2D shows a schematic structural diagram of an embodiment of the optical system according to the present disclosure. In Figure 2D , the optical system may include at least one filter, and the at least one filter may separate light with at least one wavelength belonging to the first preset wavelength set and light with at least one wavelength belonging to the second preset wavelength set. According to practical needs, the illumination source, the second dynamic aperture, and the second image sensor may be located on one side of the filter, while the first dynamic aperture and the first image sensor may be located on the other side of the filter. Of course, according to practical needs, the illumination source, the first dynamic aperture, the first image sensor, the second dynamic aperture, and the second image sensor may also be all located on the same side of the filter. Thus, step 201 may be to simultaneously perform the following operations to execute sub-step 2011B:

[0216] The seventh control operation: controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmission function h1.

[0217] The eighth control operation: control the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function h2.

[0218] The eleventh control operation: control the illumination source to emit first illumination light including at least one wavelength belonging to the first preset wavelength set λ1 and with an optical intensity conforming to the first optical intensity function f1 to the target scene within a preset exposure duration.

[0219] The twelfth control operation: control the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set λ2 and with an optical intensity conforming to the second optical intensity function f2 to the target scene within a preset exposure duration.

[0220] The tenth control operation: control the first image sensor and the second image sensor to simultaneously capture photos of the target scene.

[0221] After performing sub-step 2011B, perform sub-step 2012B: determine the photo captured by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0222] Since the filter can transmit light with at least one wavelength belonging to the first preset wavelength set λ1. Therefore, the first illumination light including at least one wavelength belonging to the first preset wavelength set λ1 and with an optical intensity conforming to the first optical intensity function f1 is reflected by the target scene, passes through the filter and the first dynamic aperture, causing the corresponding light to reach the first image sensor. And the second illumination light including at least one wavelength belonging to the second preset wavelength set λ2 and with an optical intensity conforming to the second optical intensity function f2 is reflected by the target scene, passes through the filter and the second dynamic aperture, causing the corresponding light to reach the second image sensor. The light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration conforms to the first light transmittance function h1, and the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration conforms to the second light transmittance function h2.

[0223] Alternative implementation (4): Based on the above alternative implementation (2), the optical system in the system for generating a depth image may include a polarizer. Here, the number of polarizers included in the optical system may be one or more than one. Among them, the polarizer is configured to separate light with polarization states of a first preset polarization state and a second preset polarization state. The third illumination light includes first illumination light with a polarization state of the first preset polarization state and second illumination light with a polarization state of the second preset polarization state. The first illumination light is reflected by the target scene, and through the polarizer and the first dynamic aperture, the corresponding light reaches the first image sensor. The second illumination light is reflected by the target scene, and through the polarizer and the second dynamic aperture, the corresponding light reaches the second image sensor. Thus, the ninth control operation of the above alternative implementation (2), that is, controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, can be performed as follows:

[0224] Simultaneously perform the thirteenth control operation and the fourteenth control operation, where:

[0225] The thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state of the first preset polarization state emitted by the illumination source to the target scene within a preset exposure duration according to the first light intensity function.

[0226] The fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state of the second preset polarization state emitted by the illumination source to the target scene within a preset exposure duration according to the second light intensity function.

[0227] In some implementations, the first preset polarization state and the second preset polarization state may be two mutually perpendicular polarization states. In this way, it is easier for the polarizer to separate the light of the first preset polarization state and the second preset polarization state, thereby improving the utilization rate of the third illumination light emitted by the illumination source.

[0228] For ease of understanding of alternative implementation (4), please refer to Figure 2E 。 Figure 2E is a schematic structural diagram of another embodiment of the optical system according to the present disclosure. In Figure 2E the optical system may include at least one polarizer. Among them, the at least one polarizer is configured to separate light with polarization states of a first preset polarization state P1 and a second preset polarization state P2. According to practical needs, the illumination source, the first dynamic aperture, and the first image sensor may be located on one side of the polarizer, while the second dynamic aperture and the second image sensor may be located on the other side of the polarizer. Of course, according to practical needs, the illumination source, the first dynamic aperture, the first image sensor, the second dynamic aperture, and the second image sensor may also be all located on the same side of the polarizer. In this way, step 201 may be to simultaneously perform the following operations to execute sub-step 2011B:

[0229] The seventh control operation: Control the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function h1.

[0230] The eighth control operation: Control the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function h2.

[0231] The thirteenth control operation: Control the light intensity of the first illumination light with a first preset polarization state P1 emitted by the illumination source to the target scene within a preset exposure duration according to the first light intensity function f1. Here, the first illumination light reaches the first dynamic aperture after being reflected by the target scene and passing through the polarizer and the first dynamic aperture.

[0232] The fourteenth control operation: Control the light intensity of the second illumination light with a second preset polarization state P2 emitted by the illumination source to the target scene within a preset exposure duration according to the second light intensity function f2. Here, the second illumination light reaches the second dynamic aperture after being reflected by the target scene and passing through the polarizer and the second dynamic aperture.

[0233] The tenth control operation: Control the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0234] And after performing sub-step 2011B, perform sub-step 2012B: Determine the picture taken by the first image sensor as the first picture, and determine the picture obtained by calibrating the spatial position of the picture taken by the second image sensor to the coordinate system of the first image sensor as the second picture.

[0235] Alternative implementation (5): Based on the above alternative implementation (2), the optical system in the system for generating a depth image may include a first beam splitting component, a first filter component, and a second filter component. The first filter component can transmit light with at least one wavelength belonging to the first preset wavelength set, and the second filter component can transmit light with at least one wavelength belonging to the second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component and the first image sensor are located on the first side of the first beam splitting component, and the second filter component and the second image sensor are located on the second side of the first beam splitting component. The third illumination light may include a first illumination light with at least one wavelength belonging to the first preset wavelength set and the light intensity conforming to the first light intensity function, and a second illumination light with at least one wavelength belonging to the second preset wavelength set and the light intensity conforming to the second light intensity function. The first illumination light is reflected by the target scene and the corresponding light reaches the first image sensor through the first beam splitting component, the first filter component, and the first dynamic aperture. The second illumination light is reflected by the target scene and the corresponding light reaches the second image sensor through the first beam splitting component, the second filter component, and the second dynamic aperture. Thus, the ninth control operation of the above alternative implementation (2), that is, controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, can be performed as follows:

[0236] Simultaneously perform the fifteenth control operation and the sixteenth control operation. Among them:

[0237] The fifteenth control operation includes controlling the illumination source to emit a first illumination light with at least one wavelength belonging to the first preset wavelength set and the light intensity conforming to the first light intensity function to the target scene within a preset exposure duration.

[0238] The sixteenth control operation includes controlling the illumination source to emit a second illumination light with at least one wavelength belonging to the second preset wavelength set and the light intensity conforming to the second light intensity function to the target scene within a preset exposure duration.

[0239] For ease of understanding of alternative implementation (5), please refer to Figure 2F . Figure 2F A schematic structural diagram of another embodiment of the optical system according to the present disclosure. In Figure 2F , the first filter component, the first dynamic aperture, and the first image sensor are located on one side of the first beam splitting component, and the target scene, the second dynamic aperture, the second filter component, and the second image sensor are located on the other side of the first beam splitting component. The first beam splitting component does not have a filtering function. The first filter component can transmit light with at least one wavelength belonging to the first preset wavelength set λ1, and the second filter component can transmit light with at least one wavelength belonging to the second preset wavelength set λ2. Thus, step 201 may be to simultaneously perform the following operations to execute sub-step 2011B:

[0240] Seventh control operation: Control the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function h1.

[0241] Eighth control operation: Control the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function h2.

[0242] Fifteenth control operation: Control the illumination source to emit first illumination light including at least one wavelength belonging to the first preset wavelength set λ1 and having an optical intensity conforming to the first optical intensity function f1 to the target scene within a preset exposure duration.

[0243] Sixteenth control operation: Control the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set λ2 and having an optical intensity conforming to the second optical intensity function f2 to the target scene within a preset exposure duration.

[0244] Tenth control operation: Control the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0245] And after performing sub-step 2011B, perform sub-step 2012B: Determine the picture taken by the first image sensor as the first picture, and determine the picture obtained by calibrating the spatial position of the picture taken by the second image sensor to the coordinate system of the first image sensor as the second picture.

[0246] It should be noted that the first dynamic aperture and the second dynamic aperture here can be two identical dynamic apertures or the same dynamic aperture. The first light transmittance function h1 can be the same as or different from the second light transmittance function h2. When the first dynamic aperture and the second dynamic aperture are the same dynamic aperture, and when the first light transmittance function h1 and the second light transmittance function h2 are the same, only one of the seventh control operation and the eighth control operation needs to be executed. When the first light transmittance function h1 and the second light transmittance function h2 are different, although there is only one dynamic aperture, the first dynamic aperture (i.e., the second dynamic aperture), but since light can be composed of light of multiple wavelengths, different light transmittance functions can be applied to light of different wavelengths here. For example, it can be controlled that the first dynamic aperture simultaneously has a light transmittance conforming to the first light transmittance function h1 for light of at least one wavelength belonging to the first preset wavelength set λ1 and a light transmittance conforming to the second light transmittance function h2 for light of at least one wavelength belonging to the second preset wavelength set λ2 within a preset exposure time. Of course, this requires that the hardware device of the first dynamic aperture can support such an operation. For example, a Fabry - Perot interferometer (F - P etalon) containing a nonlinear crystal or a Pockels cell can support the above operation.

[0247] In addition, it should be noted that the first illumination light with a wavelength belonging to the first preset wavelength set λ1 and an optical intensity conforming to f1 and the second illumination light with a wavelength belonging to the second preset wavelength set λ2 and an optical intensity conforming to f2 can pass through the first beam splitting component and be irradiated onto the first filter component and the second filter component respectively according to the first preset beam splitting ratio R1 and the second preset beam splitting ratio R2. Similarly, the second illumination light with a wavelength belonging to the second preset wavelength set λ2 and an optical intensity conforming to f2 can pass through the first beam splitting component and also be irradiated onto the first filter component and the second filter component respectively according to the first preset beam splitting ratio R1 and the second preset beam splitting ratio R1. Since the first filter component can transmit light with a wavelength belonging to the first preset wavelength set λ1 and the second filter component can transmit light with a wavelength belonging to the second preset wavelength set λ2, the light irradiated onto the first image sensor is light with a wavelength belonging to the first preset wavelength set λ1 and an optical intensity conforming to (R1×f1), and the light irradiated onto the second image sensor is light with a wavelength belonging to the second preset wavelength set λ2 and an optical intensity conforming to (R2×f2). In practice, the sum of the first preset beam splitting ratio and the second preset beam splitting ratio is any positive number greater than 0 and less than or equal to 1.

[0248] Optional implementation method (six): The optical system in the system for generating a depth image may include a second beam splitting component and a third filter component. Among them, the second beam splitting component may be configured to split light and transmit at least one wavelength belonging to the first preset wavelength set. The third filter component may be configured to transmit at least one wavelength belonging to the second preset wavelength set. Here, the first dynamic aperture and the first image sensor may be located on the first surface of the second beam splitting component, and the target scene, the third filter component, the second dynamic aperture, and the second image sensor may be located on the second surface of the second beam splitting component. The third illumination light may include the first illumination light with at least one wavelength belonging to the first preset wavelength set and an optical intensity conforming to the first optical intensity function and the second illumination light with at least one wavelength belonging to the second preset wavelength set and an optical intensity conforming to the second optical intensity function. The first illumination light is reflected by the target scene and passes through the second beam splitting component and the first dynamic aperture, causing the corresponding light to reach the first image sensor. The second illumination light is reflected by the target scene and passes through the second beam splitting component, the third filter component, and the second dynamic aperture, causing the corresponding light to reach the second image sensor. In this way, the ninth control operation of the above optional implementation method (two), that is, controlling the illumination source to emit the third illumination light to the target scene within a preset exposure duration, can be performed as follows:

[0249] Simultaneously execute the seventeenth control operation and the eighteenth control operation. Among them:

[0250] The seventeenth control operation includes controlling the illumination source to emit the first illumination light with at least one wavelength belonging to the first preset wavelength set and an optical intensity conforming to the first optical intensity function to the target scene within a preset exposure duration.

[0251] The eighteenth control operation includes controlling the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set and having a light intensity conforming to the second light intensity function to the target scene within a preset exposure duration.

[0252] For easy understanding of the optional implementation (6), please refer to Figure 2G . Figure 2G Schematic structural diagram of another embodiment of the optical system according to the present disclosure. In Figure 2G , the first dynamic aperture and the first image sensor are located on one side of the second beam splitting component, and the target scene, the third filter component, the second dynamic aperture and the second image sensor are located on the other side of the second beam splitting component. The second beam splitting component can split light and transmit at least one wavelength belonging to the first preset wavelength set λ1, and the third filter component can transmit at least one wavelength belonging to the second preset wavelength set λ2. Step 201 may be to simultaneously perform the following operations to execute sub-step 2011B:

[0253] The seventh control operation: controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function h1.

[0254] The eighth control operation: controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function h2.

[0255] The seventeenth control operation: controlling the illumination source to emit first illumination light including at least one wavelength belonging to the first preset wavelength set λ1 and having a light intensity conforming to the first light intensity function f1 to the target scene within a preset exposure duration.

[0256] The eighteenth control operation: controlling the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set λ2 and having a light intensity conforming to the second light intensity function f2 to the target scene within a preset exposure duration.

[0257] The tenth control operation: controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0258] And after executing sub-step 2011B, execute sub-step 2012B: determining the picture taken by the first image sensor as the first picture, and determining the picture obtained by calibrating the spatial position of the picture taken by the second image sensor to the coordinate system of the first image sensor as the second picture.

[0259] It should be noted that the first dynamic aperture and the second dynamic aperture can be two identical dynamic apertures or the same dynamic aperture. The first light transmission function h1 can be the same as or different from the second light transmission function h2. When the first dynamic aperture and the second dynamic aperture are the same dynamic aperture, and when the first light transmission function h1 and the second light transmission function h2 are the same, only one of the seventh control operation and the eighth control operation needs to be executed. When the first light transmission function h1 and the second light transmission function h2 are different, although there is only one dynamic aperture, the first dynamic aperture (i.e., the second dynamic aperture), since light can be composed of light of multiple wavelengths, different light transmission functions can be applied to light of different wavelengths here. For example, it can be controlled that the first dynamic aperture, within a preset exposure time, has a light transmittance that conforms to the first light transmission function h1 for light of at least one wavelength belonging to the first preset wavelength set λ1, and a light transmittance that conforms to the second light transmission function h2 for light of at least one wavelength belonging to the first preset wavelength set λ2. Of course, this requires that the hardware device of the first dynamic aperture can support such operations. For example, a Fabry - Perot interferometer (F - P etalon) or a Pockels cell containing a nonlinear crystal can support the above operations.

[0260] In addition, it should be noted that since the second beam - splitting component has both a beam - splitting function and a filtering function, therefore, the first illumination light including light with an intensity conforming to f1 and at least one wavelength belonging to the first preset wavelength set λ1 can pass through the second filtering component and irradiate the first dynamic aperture at a first preset beam - splitting ratio R1. Then, the light irradiating the first image sensor is light with an intensity conforming to (R1×f1) and a wavelength belonging to the first preset wavelength set λ1. And the first illumination light including light with an intensity conforming to f1 and at least one wavelength belonging to the first preset wavelength set λ1 can pass through the second beam - splitting component and irradiate the third filtering component at a second preset beam - splitting ratio R2, and the second illumination light including light with an intensity conforming to f2 and at least one wavelength belonging to the second preset wavelength set λ2 also passes through the second beam - splitting component and irradiates the third filtering component at the second preset beam - splitting ratio R2. And since the third filtering component can transmit light with a wavelength belonging to the second preset wavelength set λ2, the light irradiating the second image sensor is light with an intensity conforming to (R2×f2) and a wavelength belonging to the second preset wavelength set λ2. In practice, the sum of the first preset beam - splitting ratio and the second preset beam - splitting ratio is any positive number greater than 0 and less than or equal to 1.

[0261] Alternative Implementation (VII): At least one set of dynamic apertures and corresponding image sensors in a system for generating depth images may include a first dynamic aperture and a corresponding first image sensor. Here, the first image sensor may be an image sensor array alternately provided with first image sensor pixel units and second image sensor pixel units. The first image sensor pixel units may be provided with filters that transmit light of at least one wavelength belonging to a first preset wavelength set, while the second image sensor pixel units may be provided with filters that transmit light of at least one wavelength belonging to a second preset wavelength set. Thus, step 201 may include sub-steps 2011C to 2014C as Figure 2H shown. Please refer to Figure 2H , which shows a decomposed flowchart of another embodiment of step 201 according to the present disclosure.

[0262] Sub-step 2011C, simultaneously perform the nineteenth control operation, the twentieth control operation, the twenty-first control operation, and the twenty-second control operation.

[0263] Wherein:

[0264] The nineteenth control operation includes controlling the illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to a first optical intensity function to the target scene within a preset exposure duration.

[0265] The twentieth control operation includes controlling the illumination source to emit second illumination light including light of at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to a second optical intensity function to the target scene within a preset exposure duration.

[0266] Here, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, and the second preset ratio is greater than zero and less than or equal to 1.

[0267] The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light of at least one wavelength belonging to the first preset wavelength set within the preset exposure duration according to a first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light of at least one wavelength belonging to the second preset wavelength set within the preset exposure duration according to a second light transmittance function.

[0268] The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene.

[0269] Sub-step 2012C, obtain the photo taken by the first image sensor and determine the obtained photo as the third photo.

[0270] Sub-step 2013C: Generate a first photo using the pixel values corresponding to the respective pixel points collected by the first image sensor pixel units in the third photo.

[0271] Sub-step 2014C: Generate a second photo using the pixel values corresponding to the respective pixel points collected by the second image sensor pixel units in the third photo.

[0272] As can be seen from the above description, in the optional implementation (VII), by alternately arranging the first image sensor pixel units and the second image sensor pixel units in the first image sensor, and the first image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to the first preset wavelength set, while the second image sensor pixel units are provided with filters that transmit light of at least one wavelength belonging to the second preset wavelength set. Then, only one image sensor needs to take a single shot to obtain the first photo and the second photo.

[0273] For ease of understanding the optional implementation (VI), please refer to Figure 2I . Figure 2I is a schematic structural diagram of an embodiment of at least one set of dynamic apertures and corresponding image sensors according to the present disclosure. In Figure 2I , an illumination source, a first dynamic aperture, and a first image sensor are shown. Among them, Figure 2I schematically shows a first image sensor including a 5×5 image sensor array. Among them, in the first image sensor pixel units indicated by shading, filters that transmit light of at least one wavelength belonging to the first preset wavelength set λ1 are provided, and in the second image sensor pixel units indicated by non-shading, filters that transmit light of at least one wavelength belonging to the second preset wavelength set λ2 are provided. Thus, step 201 may include performing the following sub-steps:

[0274] Sub-step 2011C: Simultaneously perform the nineteenth control operation, the twentieth control operation, the twenty-first control operation, the twenty-second control operation, and the twenty-third control operation.

[0275] Among them:

[0276] The nineteenth control operation: Control the illumination source to emit first illumination light including light of at least one wavelength belonging to the first preset wavelength set λ1 and with an optical intensity conforming to the first optical intensity function f1 to the target scene within a preset exposure duration.

[0277] The twentieth control operation: Control the illumination source to emit second illumination light of at least one wavelength belonging to the second preset wavelength set λ2 and with an optical intensity conforming to the second optical intensity function f2 to the target scene within a preset exposure duration.

[0278] The twenty - first control operation: controlling the light transmittance of the first dynamic aperture for light with at least one wavelength belonging to the first preset wavelength set within a preset exposure duration according to the first light transmittance function h1 and / or controlling the light transmittance of the first dynamic aperture for light with at least one wavelength belonging to the second preset wavelength set within a preset exposure duration according to the second light transmittance function h2.

[0279] The twenty - second control operation: controlling the first image sensor to take a photo of the target scene.

[0280] Sub - step 2012C: obtaining the photo taken by the first image sensor and determining the obtained photo as the third photo.

[0281] Sub - step 2013C: generating the first photo with the pixel values of each pixel point corresponding to the pixel units of the first image sensor in the third photo.

[0282] The pixel values of the pixel points corresponding to the pixel units of the first image sensor in the third photo represent the color values of the light with a wavelength belonging to the first preset wavelength set λ1 and a light intensity conforming to the first light intensity function in the area corresponding to the pixel point in the target scene. Therefore, when generating the first photo with the pixel values of each pixel point corresponding to the pixel units of the first image sensor in the third photo, the pixel values of each pixel point in the first photo are the color values of the light with a wavelength of λ1 and a light intensity conforming to the first light intensity function in the area corresponding to the pixel point in the target scene.

[0283] Sub - step 2014C: generating the second photo with the pixel values of each pixel point corresponding to the pixel units of the second image sensor in the third photo.

[0284] The pixel values of the pixel points corresponding to the pixel units of the second image sensor in the third photo represent the color values of the light with a wavelength belonging to the second preset wavelength set λ2 and a light intensity conforming to the second light intensity function in the area corresponding to the pixel point in the target scene. Therefore, when generating the second photo with the pixel values of each pixel point corresponding to the pixel units of the second image sensor in the third photo, the pixel values of each pixel point in the second photo are the color values of the light with a wavelength of λ2 and a light intensity conforming to the second light intensity function in the area corresponding to the pixel point in the target scene.

[0285] It should be noted that here the first light transmittance function is the same as the second light transmittance function, while the first light intensity function f1 is different from the second light intensity function f2.

[0286] Alternative implementation (VIII): At least one set of dynamic apertures and corresponding image sensors in a system for generating depth images may include a first dynamic aperture and a corresponding first image sensor, as well as a second image sensor. The first image sensor and the illumination source are located on a first side of the first dynamic aperture, and the second image sensor is located on a second side of the first dynamic aperture. Thus, step 201 may include sub-steps 2011D and 2012D as shown in Figure 2J :

[0287] Sub-step 2011D, performing the twenty-third control operation, the twenty-fourth control operation, and the twenty-fifth control operation simultaneously.

[0288] Wherein:

[0289] The twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to a first light transmittance function and a second light transmittance function, respectively.

[0290] The twenty-fourth control operation includes controlling the illumination source to emit third illumination light towards the target scene within a preset exposure duration. Among them, the first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing the corresponding light to reach the first image sensor. The second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing the corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to a first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to a second light intensity function. The pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1.

[0291] The twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0292] Sub-step 2012D, determining the picture taken by the first image sensor as the first picture, and determining the picture obtained by calibrating the spatial position of the picture taken by the second image sensor to the coordinate system of the first image sensor as the second picture.

[0293] For ease of understanding of alternative implementation (VIII), please refer to Figure 2K . Figure 2K is a schematic structural diagram of another embodiment of at least one set of dynamic apertures and corresponding image sensors according to the present disclosure. It can be seen from Figure 2K that the first image sensor is located on one side of the first dynamic aperture, and the second image sensor and the target scene are located on the other side of the first dynamic aperture. Thus, step 201 may include performing the following sub-steps:

[0294] Sub-step 2011D: Simultaneously perform the twenty-third control operation, the twenty-fourth control operation, and the twenty-fifth control operation.

[0295] The twenty-third control operation: Control the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to the first light transmittance function h1 and the second light transmittance function h2 respectively. Here, the light transmittance represents the ratio of the light flux of the light transmitted from one side of the first dynamic aperture to the other side of the first dynamic aperture to the light flux of the light irradiated on one side of the first dynamic aperture, and the reflectance represents the ratio of the light flux of the light reflected from one side of the first dynamic aperture to the light flux of the light irradiated on one side of the first dynamic aperture. Therefore, the light transmittance of the light transmitted to the first image sensor relative to the light irradiated on the first dynamic aperture within the preset exposure duration conforms to the first light transmittance function h1, and the light transmittance of the light reflected to the second image sensor relative to the light irradiated on the first dynamic aperture within the preset exposure duration conforms to the second light transmittance function h2.

[0296] The twenty-fourth control operation: Control the third illumination light emitted by the illumination source to the target scene within a preset exposure duration. Among them, the first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, resulting in the corresponding light reaching the first image sensor. The second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, resulting in the corresponding light reaching the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function f1, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function f2.

[0297] The twenty-fifth control operation: Control the first image sensor and the second image sensor to simultaneously take pictures of the target scene.

[0298] Sub-step 2012D: Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0299] As can be seen from the above description, the first photo and the second photo are different photos obtained by two different image sensors taking pictures of the target scene at the same time. Moreover, different light intensity functions and / or different light transmittance functions are adopted during the process of taking the first photo and the second photo, that is, the first light intensity function is different from the second light intensity function and / or the first light transmittance function is different from the second light transmittance function.

[0300] The above optional implementation methods (one) to (eight) give specific implementation methods of different step 201. Which implementation method is specifically adopted is not specifically limited in the present disclosure.

[0301] Alternative implementation (IX): Based on any one of the above alternative implementations (I) to (VIII), step 202 may include sub-steps 2021A to 2023A as shown in Figure 2L . Please refer to Figure 2L , which shows a decomposed flowchart of step 202 according to an embodiment of the present disclosure.

[0302] Sub-step 2021A: For each pixel point with coordinates (m, n) in the first photo and the second photo, establish a first equation and a second equation.

[0303] Among them, the first equation is:

[0304]

[0305] Among them:

[0306] R(m, n) is the reflectivity of the area in the target scene corresponding to the pixel point with coordinates (m, n).

[0307] f1 is the first light intensity function.

[0308] t is the time variable.

[0309] is the light intensity when the first illumination light reaches the area in the target scene corresponding to the pixel point with coordinates (m, n) in the first photo from the illumination source at time t and then returns to the first image sensor.

[0310] t d (m, n) is the duration for the light to reach the area in the target scene corresponding to the pixel point with coordinates (m, n) from the position of the illumination source.

[0311]

[0312] c is the speed of light.

[0313] z(m, n) is the distance between the area in the target scene corresponding to the pixel point with coordinates (m, n) in the generated depth image and the first image sensor, that is, the unknown quantity to be solved finally for generating the depth image.

[0314] L is the distance between the first image sensor and the illumination source.

[0315] α is the angle between the first side and the second side. Among them, the first side is the line segment connecting the position of the first image sensor and the area in the target scene corresponding to the pixel point with coordinates (m, n). The second side is the line segment connecting the position of the first image sensor and the position of the illumination source. It can be understood that L is also the length of the second side.

[0316] For ease of understanding, please refer to Figure 2M . Figure 2M Shows an embodiment of the calculation principle of t d (m,n). As Figure 2M shown, point A is the location of the illumination source, point B is the area in the target scene corresponding to the pixel point with coordinates (m,n), and point C is the location of the first image sensor. Then the first side is side CB, the second side is side CA, and L is the length of side CA. α is the apex angle corresponding to vertex C of triangle ABC and is also the included angle between side CB and side CA. The length of side AB is t d (m,n)·c, that is, the distance between the illumination source and the area in the target scene corresponding to the pixel point with coordinates (m,n). The length of side BC is z(m,n), and the length of side AC is L. From triangle knowledge, the following formula is obtained:

[0317] z(m,n) 2 +L 2 =(t d (m,n)·c) 2 +2·L·z(m,n)·cosα (Formula 6)

[0318] From the above equation, it can be obtained that: It can be understood that when the locations of the illumination source and the first image sensor are the same, then the first equation can be expressed as follows:

[0319]

[0320] Next, continue to explain other parameters in the first equation:

[0321] h1 is the first light transmission function, t0 is the start time of exposure, and τ is the preset exposure duration.

[0322] S1(m,n) is the pixel value of the pixel point with coordinates (m,n) in the first photo.

[0323] The second equation is:

[0324]

[0325] Where:

[0326] f2 is the second light intensity function.

[0327] is the light intensity when the second illumination light reaches the area in the target scene corresponding to the pixel point with coordinates (m,n) in the second photo from the illumination source at time t and then returns to the first image sensor.

[0328] h2 is the second light transmission function.

[0329] S2(m, n) is the pixel value of the pixel at coordinates (m, n) in the second photo.

[0330] Then, a third equation is obtained from the first equation and the second equation, and the third equation is:

[0331]

[0332] It should be noted that the pixel distributions of the first photo and the second photo can be the same. It can be understood that if the pixel distributions of the first photo and the second photo are different, the pixel distributions of the first photo and the second photo can be made the same by upsampling (e.g., interpolation) or downsampling methods. Therefore, for each pixel at coordinates (m, n), the first equation and the second equation can be established.

[0333] Sub-step 2022A: Solve the third equation to obtain z(m, n), and determine the depth value of the pixel at coordinates (m, n) in the generated depth image according to z(m, n).

[0334] Here, S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known.

[0335] Here, when generating the depth image, first, the pixel distribution of the generated depth image can be determined according to the pixel distributions of the first photo and the second photo. For example, when the pixel distributions of the first photo and the second photo are M×N, that is, both the first photo and the second photo have M pixel points horizontally and N pixel points vertically.

[0336] Then, for each pixel at coordinates (m, n), the depth value of the pixel is determined according to z(m, n). For example, z(m, n) can be directly determined as the depth value of the pixel. Another example is that z(m, n) can also be normalized to a preset value range, and the value obtained after normalization can be determined as the depth value of the pixel. For example, it can be normalized to an integer between 0 and 255.

[0337] Sub-step 2023A: Generate a depth image of the target scene based on the determined depth value of each pixel.

[0338] The pixel distribution of the depth image is determined, and the depth value of each pixel is determined, that is, a depth image of the target scene can be generated.

[0339] It should be noted that the specific implementation method of generating the depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information is shown in the optional implementation method (IX) when only two photos, that is, the first photo and the second photo, are taken. In practice, at least one set of dynamic apertures and the corresponding image sensors can be used to take more photos of the target scene, and more equations can be established. As long as the unknown z(m,n) is introduced into the established equations, more z(m,n) can be obtained by solving the equations, and the final z(m,n) can be determined through the multiple z(m,n) obtained by the solution. For example, the mean value of multiple z(m,n) can be taken as the final z(m,n). It can be understood that the more photos are collected, the more equations are established, and the more accurate the finally obtained z(m,n) is.

[0340] Optional implementation method (X), based on any one of the above optional implementation methods (I) to (VIII), before the execution subject executes step 201, the twenty-sixth control operation and the twenty-seventh control operation are simultaneously executed, where:

[0341] The twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function.

[0342] The twenty-seventh control operation includes controlling the first image sensor to take a photo of the target scene and determining the photo taken by the first image sensor as the background light photo.

[0343] From the above description of any one of the optional implementation methods (I) to (VIII) and the above description of the twenty-sixth control operation and the twenty-seventh control operation, it can be seen that both the background light photo and the first photo are taken by the first image sensor, and during the process of taking the background light photo and the first photo, the light transmittance of the first dynamic aperture for the first image sensor is controlled within a preset exposure duration according to the first light transmittance function. The difference between the shooting processes of the background light photo and the first photo is that there is no illumination source irradiating light on the target scene in the background light photo, while there is an illumination source irradiating the target scene within a preset exposure duration according to the first light intensity function during the shooting process of the first photo.

[0344] Based on the above operation of determining the background light photo performed before step 201, step 202 can be carried out as follows: Generate the depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0345] It can be understood that with the background light photo, the first photo, and the second photo, at least three equations can be established, and unknowns can be introduced into the three established equations, that is, the distance between the region corresponding to each pixel point in the generated depth image in the target scene and the image sensor. By using various implementation methods to solve the equations, the above distance can be obtained, and a depth image of the target scene can be generated based on the distance obtained by the above solution.

[0346] Optional implementation method (eleven), based on the above optional implementation method (ten), step 202, that is, generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information, may include sub-steps 2021B to 2024B as Figure 2N shown. Please refer to Figure 2N , which shows a decomposition flowchart of step 202 according to an embodiment of the present disclosure:

[0347] Sub-step 2021B, for each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo, establish a fourth equation, a fifth equation, and a sixth equation.

[0348] The fourth equation is:

[0349]

[0350] Where:

[0351] S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo.

[0352] P0(m, n) is the light intensity of the region corresponding to the pixel point with coordinates (m, n) in the target scene under the background light.

[0353] h1 is the first light transmission function.

[0354] t is the time variable, t0 is the start time of exposure, and τ is the preset exposure duration;

[0355] The fifth equation is:

[0356]

[0357] Where:

[0358] R(m, n) is the reflectivity of the region corresponding to the pixel point with coordinates (m, n) in the target scene.

[0359] f1 is the first light intensity function.

[0360] It is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor.

[0361] t d (m, n) is the duration for the light to travel from the position of the illumination source to the area corresponding to the pixel point with coordinates (m, n) in the target scene.

[0362]

[0363] c is the speed of light.

[0364] z(m, n) is the distance between the area corresponding to the pixel point with coordinates (m, n) in the generated depth image in the target scene and the first image sensor, which is the unknown quantity to be solved ultimately for generating the depth image.

[0365] L is the distance between the first image sensor and the illumination source, and α is the angle between the first side and the second side. Here, the first side is the line segment connecting the position of the first image sensor and the area corresponding to the pixel point with coordinates (m, n) in the target scene, and the second side is the line segment connecting the position of the first image sensor and the position of the illumination source. It can be understood that L is also the length of the second side.

[0366] For the specific understanding of the above description, reference can be made to the relevant description in the optional implementation (nine) regarding Figure 2M and will not be elaborated here.

[0367] h1 is the first light transmission function, t0 is the start time of exposure, and τ is the preset exposure duration.

[0368] S1(m, n) is the pixel value of the pixel point with coordinates (m, n) in the first photo.

[0369] The sixth equation is:

[0370]

[0371] Where:

[0372] f2 is the second light intensity function.

[0373] It is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor. It can be understood that it is assumed here that the image sensor for capturing the first illumination and the image sensor for capturing the second photo are located at the same position. In practice, when both the image sensors for capturing the first photo and the second photo are the first image sensor, the above assumption holds. When the image sensor for capturing the first photo is the first image sensor, and the image sensor for capturing the second photo is the second image sensor, and the first image sensor and the second image sensor are located at different positions, then the photo captured by the second image sensor can be spatially calibrated to the coordinates of the first image sensor to achieve the assumption that the image sensor for capturing the first illumination and the image sensor for capturing the second photo are located at the same position.

[0374] h2 is the second light transmission function.

[0375] S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo.

[0376] Sub-step 2022B, obtain the seventh equation according to the fourth equation, the fifth equation and the sixth equation.

[0377] The seventh equation is:

[0378]

[0379] Sub-step 2023B, solve the seventh equation to obtain z(m, n), and determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n).

[0380] Here, S b (m, n), S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known.

[0381] Here, regarding how to determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n), reference can be made to the relevant description in sub-step 2022A of the above optional implementation (nine), which will not be elaborated here.

[0382] Sub-step 2024B, generate the depth image of the target scene based on the determined depth value of each pixel point.

[0383] The pixel distribution of the depth image is determined, and the depth value of each pixel point is determined, that is, the depth image of the target scene can be generated.

[0384] It should be noted that although the first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for taking the first photo with respect to the image sensor for taking the first photo and time, in practice, the first light transmission function can be related to the coordinates of each pixel point in the first photo. In other words, for each pixel point in the first photo, there is a corresponding first light transmission function, and the first light transmission function corresponding to this pixel point is used to characterize the correspondence between the light transmittance of the dynamic aperture for taking the first photo with respect to the pixel unit of the image sensor for taking this pixel point and time. For the sake of simplicity in description, only the above description mentions that the first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for taking the first photo with respect to the image sensor for taking the first photo and time. Similarly, the above description also applies to the second light transmission function.

[0385] Similarly, although the first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when taking the first photo and time. In practice, the first light intensity function can be related to the area in the target scene corresponding to each pixel point in the first photo. In other words, for each pixel point in the first photo, there is a corresponding first light intensity function, and the first light intensity function corresponding to this pixel point is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the area in the target scene corresponding to this pixel point when taking the first photo and time. For the sake of simplicity in description, only the above description mentions that the first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when taking the first photo and time. Similarly, the above description also applies to the second light intensity function.

[0386] Optional implementation method (twelve), the dynamic aperture in the system for generating a depth image can be an image intensifier. In practice, the light transmittance, exposure start time, and exposure end time of the dynamic aperture can be controlled according to different light transmission functions (such as the first light transmission function, the second light transmission function) by adjusting the voltage of the image intensifier.

[0387] Optional implementation method (thirteen), the dynamic aperture in the system for generating a depth image can be a Fabry - Perot interferometer containing a nonlinear crystal. In practice, the light transmittance, exposure start time, and exposure end time of the dynamic aperture can be controlled according to different light transmission functions (such as the first light transmission function, the second light transmission function) by adjusting the voltage of the Fabry - Perot interferometer containing a nonlinear crystal.

[0388] Optional implementation method (fourteen), the above execution entity can also, after executing step 202, execute the following step 203:

[0389] Step 203, generate a three - dimensional model of the target scene according to the first photo, the second photo, and the depth image.

[0390] Here, the above-mentioned execution entity can adopt various implementation manners to generate a three-dimensional model of the target scene based on the first photo and the second photo obtained in step 201 and the depth image generated in step 202.

[0391] When the depth image is an image in which each pixel point includes only a pixel value of a depth value channel, the above-mentioned execution entity can generate a three-dimensional model of the target scene based on the first photo and the depth image; the above-mentioned execution entity can also generate a three-dimensional model of the target scene based on the second photo and the depth image; the above-mentioned execution entity can also generate a three-dimensional model of the target scene based on the average image of the first photo and the second photo and the depth image; the above-mentioned execution entity can also first calculate the average pixel value of the first photo and the second photo, and generate a three-dimensional model of the target scene based on the photo with the higher average pixel value among the first photo and the second photo and the depth image. No matter which implementation manner above, a three-dimensional model is generated based on an image with a color value channel and an image with a depth value channel.

[0392] When the depth image is an image in which each pixel point includes both a pixel value of a color value channel and a pixel value of a depth value channel, the above-mentioned execution entity can directly use the depth image to generate a three-dimensional model of the target scene, because the depth image itself already includes a color value channel and a depth value channel.

[0393] In practice, the pixel distributions of the first photo and the second photo obtained in step 201 and the depth image generated in step 202 can be the same, that is, there is a one-to-one correspondence between the pixel points in the first photo, the second photo obtained in step 201 and the depth image generated in step 202. In other words, if the pixel distributions of the first photo, the second photo obtained in step 201 and the depth image generated in step 202 are different, the above-mentioned execution entity can also register the first photo, the second photo and the depth image to the same pixel distribution by upsampling or downsampling, and obtain the first photo, the second photo and the depth image with the same pixel distribution.

[0394] It should be noted that generating a three-dimensional model based on an image with a color value channel and a depth value channel having the same pixel distribution is an existing technology that has been widely studied and applied at present, and will not be elaborated here. For example, the surface representation method, the volume representation method, the implicit representation method, the hierarchical model method, etc. can be adopted.

[0395] It should be noted that here, the dynamic aperture can dynamically change the light transmittance under the control of a device in a system for generating a depth image. Here, the light transmittance of the dynamic aperture can be dynamically changed to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1. That is, the dynamic aperture can not only achieve the restoration and reduction of light, but also achieve the amplification of light.

[0396] Optional implementation method (fifteen): The above-mentioned execution entity can also execute the following steps:

[0397] First, for each shooting configuration information in the set of shooting configuration information, based on the shooting configuration information, obtain the third photo and the fourth photo corresponding to the shooting configuration information, and generate a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo.

[0398] Here, the set of shooting configuration information can include at least one shooting configuration information.

[0399] Among them, the shooting configuration information can include an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter. The third photo and the fourth photo corresponding to the shooting configuration information can be the photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information respectively. The third shooting parameter can include a third light transmittance function and a third light intensity function, and the fourth shooting parameter can include a fourth light transmittance function and a fourth light intensity function. The third light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time. The fourth light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time. The third light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the third photo and time. The fourth light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the fourth photo and time. The third light transmittance function is not a constant and / or the fourth light transmittance function is not a constant. The third light transmittance function is different from the fourth light transmittance function and / or the third light intensity function is different from the fourth light intensity function. The exposure start time parameters in the shooting configuration information of the set of shooting configuration information are different from each other and / or the exposure duration parameters in the shooting configuration information of the set of shooting configuration information are different from each other. The third shooting parameters in the shooting configuration information of the set of shooting configuration information are all the same and the fourth shooting parameters in the shooting configuration information of the set of shooting configuration information are all the same, or the third shooting parameters in the shooting configuration information of the set of shooting configuration information are different from each other and the fourth shooting parameters in the shooting configuration information of the set of shooting configuration information are different from each other.

[0400] Then, based on the generated depth images of partial scenes corresponding to each shooting configuration information pair in the set of shooting configuration information of the target scene and the preset shooting configuration information, a depth image of the target scene is generated.

[0401] Optional implementation (sixteen): The first dynamic aperture and the first image sensor can be the same device. In this way, the above-mentioned execution entity controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function can include: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function. Moreover, the above-mentioned execution entity controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the second light transmittance function can include: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the second light transmittance function.

[0402] Optional implementation (seventeen): The first dynamic aperture and the first image sensor can be the same device. In this way, the above-mentioned execution entity controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function can include: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function; and the above-mentioned execution entity controlling the light transmittance of the second dynamic aperture for the second image sensor within a preset exposure duration according to the second light transmittance function can include: controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmittance function.

[0403] Optional implementation (eighteen): The dynamic aperture can also be configured such that, under the control of the control device, the wavelength of the light incident on the dynamic aperture is different from the wavelength of the light emitted from the dynamic aperture, and the wavelength of the light emitted from the dynamic aperture is related to the preset wavelength sensitive range of the image sensor corresponding to the dynamic aperture.

[0404] The method provided by the above embodiments of the present disclosure generates a depth image of a target scene by first obtaining a first photo and a second photo, and then generating a depth image of the target scene based on the obtained first photo, second photo, and corresponding shooting configuration information. The first photo and the second photo are photos taken with different configuration information obtained by dynamically changing the light intensity using an illumination source and dynamically changing the light transmittance using a dynamic aperture. First, the cost of the dynamic aperture is not high. Second, any currently commercially available illumination source and image sensor (e.g., currently commercially available cameras) and various illumination sources and image sensors developed in the future (e.g., cameras developed in the future) can be used to implement the generation of the depth image. Therefore, compared with the existing methods for generating depth images, the economic cost is reduced. In addition, the image resolution of any currently commercially available ordinary camera is generally higher than that of various imaging devices used in measuring the scene distance by imaging methods. Therefore, compared with the existing methods for generating depth images, the image resolution of the generated depth image is improved.

[0405] Further referring to Figure 3 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an apparatus for generating a depth image. This apparatus embodiment corresponds to the Figure 2A method embodiment shown. The apparatus can be specifically applied to a control device in a system for generating a depth image. The system for generating a depth image includes an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors. The dynamic apertures are configured to dynamically change the light transmittance, exposure start time, and exposure end time under the control of the control device.

[0406] As shown in Figure 3As shown in the figure, the device 300 for generating a depth image in this embodiment includes: an acquisition unit 301 and a depth image generation unit 302. Among them, the acquisition unit 301 is configured to acquire a first photo and a second photo. The first photo and the second photo are respectively photos obtained by the image sensor shooting a target scene according to first shooting configuration information and second shooting configuration information. The first shooting configuration information includes a first light transmission function and a first light intensity function, and the second shooting configuration information includes a second light transmission function and a second light intensity function. The first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the first photo with respect to the image sensor for shooting the first photo and time. The second light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the second photo with respect to the image sensor for shooting the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the second photo and time. The first light transmission function is not a constant and / or the second light transmission function is not a constant. The first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function. The depth image generation unit 302 is configured to generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0407] In this embodiment, for the specific processing of the acquisition unit 301 and the depth image generation unit 302 of the device 300 for generating a depth image and the technical effects brought by them, reference can be made to Figure 2A the relevant descriptions of steps 201 and 202 in the corresponding embodiments, which will not be elaborated here.

[0408] In some optional implementation manners of this embodiment, the device 300 may further include: a three-dimensional model generation unit 303, configured to generate a three-dimensional model of the target scene according to the first photo, the second photo, and the depth image.

[0409] In some optional implementation manners of this embodiment, the at least one group of dynamic apertures and the corresponding image sensors may include a first dynamic aperture and the corresponding first image sensor; and the acquisition unit 301 may include: a first control module ( Figure 3(not shown in the figure), while performing a first control operation, a second control operation, and a third control operation, wherein the first control operation includes controlling the intensity of the first illumination light emitted by the illumination source to the target scene within a preset exposure duration according to the first light intensity function, wherein the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the second control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the first light transmittance function, and the third control operation includes controlling the first image sensor to take a photo of the target scene; a first determination module( Figure 3 (not shown in the figure), configured to determine the photo taken by the first image sensor as the first photo; a second control module( Figure 3 (not shown in the figure), configured to simultaneously perform a fourth control operation, a fifth control operation, and a sixth control operation, wherein the fourth control operation includes controlling the intensity of the second illumination light emitted by the illumination source to the target scene within the preset exposure duration according to the second light intensity function, wherein the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1, the fifth control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the second light transmittance function, and the sixth control operation includes controlling the first image sensor to take a photo of the target scene; a second determination module( Figure 3 (not shown in the figure), configured to determine the photo taken by the first image sensor as the second photo.

[0410] In some alternative implementation manners of this embodiment, the at least one group of dynamic apertures and the corresponding image sensors may include a first dynamic aperture and the corresponding first image sensor, and a second dynamic aperture and the corresponding second image sensor; and the obtaining unit 301 may include: a third control module( Figure 3(not shown in the figure) is configured to perform a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation simultaneously, wherein the seventh control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmittance function, the eighth control operation includes controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to the second light transmittance function, the ninth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration, wherein the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, the third preset ratio is greater than zero and less than or equal to 1, the first illumination light included in the third illumination light is reflected by the target scene and the corresponding light reaches the first image sensor due to the optical system and the first dynamic aperture, the second illumination light included in the third illumination light is reflected by the target scene and the corresponding light reaches the second image sensor due to the optical system and the second dynamic aperture, the light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function, and the tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously capture a photo of the target scene; a third determination module( Figure 3 (not shown in the figure) is configured to determine the photo captured by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo captured by the second image sensor to the coordinate system of the first image sensor as the second photo.

[0411] In some alternative implementation manners of this embodiment, the above optical system may include a filter. The above filter may be configured to separate light with at least one wavelength belonging to a first preset wavelength set and light with at least one wavelength belonging to a second preset wavelength set. The above third illumination light may include first illumination light with at least one wavelength belonging to the above first preset wavelength set and second illumination light with at least one wavelength belonging to the above second preset wavelength set. After the first illumination light is reflected by the above target scene and passes through the above filter and the above first dynamic aperture, the corresponding light can reach the above first image sensor. After the second illumination light is reflected by the above target scene and passes through the above filter and the above second dynamic aperture, the corresponding light can reach the above second image sensor. And the control of the illumination source to emit the third illumination light to the above target scene within the above preset exposure duration may include: simultaneously performing an eleventh control operation and a twelfth control operation, where the above eleventh control operation includes controlling the illumination source to emit first illumination light with at least one wavelength belonging to the above first preset wavelength set and the light intensity conforming to the above first light intensity function to the above target scene within the above preset exposure duration, and the above twelfth control operation includes controlling the illumination source to emit second illumination light with at least one wavelength belonging to the above second preset wavelength set and the light intensity conforming to the above second light intensity function to the above target scene within the above preset exposure duration.

[0412] In some alternative implementation manners of this embodiment, the above optical system may include a polarizer. The above polarizer may be configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The above third illumination light may include first illumination light with a polarization state being the above first preset polarization state and second illumination light with a polarization state being the above second preset polarization state. After the first illumination light is reflected by the above target scene and passes through the above polarizer and the above first dynamic aperture, the corresponding light can reach the above first image sensor. After the second illumination light is reflected by the above target scene and passes through the above polarizer and the above second dynamic aperture, the corresponding light can reach the above second image sensor. And the control of the illumination source to emit the third illumination light to the above target scene within the above preset exposure duration may include: simultaneously performing a thirteenth control operation and a fourteenth control operation, where the above thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state being the above first preset polarization state emitted by the illumination source to the above target scene within the above preset exposure duration according to the above first light intensity function, and the above fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state being the above second preset polarization state emitted by the illumination source to the above target scene within the above preset exposure duration according to the above second light intensity function.

[0413] In some alternative implementation manners of this embodiment, the above optical system may include a first beam splitting component, a first filter component, and a second filter component. The first filter component may transmit light with at least one wavelength belonging to a first preset wavelength set. The second filter component may transmit light with at least one wavelength belonging to a second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor may be located on a first surface of the first beam splitting component. The target scene, the second filter component, the second dynamic aperture, and the second image sensor may be located on a second surface of the first beam splitting component. And controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration may include: simultaneously performing a fifteenth control operation and a sixteenth control operation. Among them, the fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within the preset exposure duration. The sixteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration.

[0414] In some alternative implementation manners of this embodiment, the above optical system may include a second beam splitting component and a third filter component. The second beam splitting component may be configured to split light and transmit light with at least one wavelength belonging to a first preset wavelength set. The third filter component may be configured to transmit light with at least one wavelength belonging to a second preset wavelength set. The first dynamic aperture and the first image sensor may be located on a first surface of the second beam splitting component. The target scene, the third filter component, the second dynamic aperture, and the second image sensor may be located on a second surface of the second beam splitting component. And controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration may include: simultaneously performing a seventeenth control operation and an eighteenth control operation. Among them, the seventeenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within the preset exposure duration. The eighteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration.

[0415] In some alternative implementation manners of this embodiment, the at least one set of dynamic apertures and corresponding image sensors may include a first dynamic aperture and a corresponding first image sensor. The first image sensor may be an image sensor array in which first image sensor pixel units and second image sensor pixel units are alternately arranged. The first image sensor pixel units may be provided with filters that transmit light with at least one wavelength belonging to a first preset wavelength set, and the second image sensor pixel units may be provided with filters that transmit light with at least one wavelength belonging to a second preset wavelength set; and the obtaining unit 301 may include: a fourth control module ( Figure 3 not shown in the figure), configured to perform a nineteenth control operation, a twentieth control operation, a twenty-first control operation, and a twenty-second control operation simultaneously. Among them, the nineteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration. Among them, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the first preset wavelength set within the preset exposure duration according to the first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the second preset wavelength set within the preset exposure duration according to the second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; a fourth determination module ( Figure 3 not shown in the figure), obtaining the photo taken by the first image sensor and determining the obtained photo as a third photo; a first generation module ( Figure 3 not shown in the figure), configured to generate a first photo with the pixel values of each pixel point corresponding to the first image sensor pixel units in the third photo; a second generation module ( Figure 3 not shown in the figure), configured to generate a second photo with the pixel values of each pixel point corresponding to the second image sensor pixel units in the third photo.

[0416] In some alternative implementation manners of this embodiment, the at least one group of dynamic apertures and the corresponding image sensors may include a first dynamic aperture, a corresponding first image sensor, and a second image sensor. The first image sensor may be located on a first surface of the first dynamic aperture, and the second image sensor and the target scene may be located on a second surface of the first dynamic aperture. And the obtaining unit 301 may include: a fifth control module ( Figure 3 , not shown in the figure), configured to perform a twenty-third control operation, a twenty-fourth control operation, and a twenty-fifth control operation simultaneously, where the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within the preset exposure duration according to the first light transmittance function and the second light transmittance function respectively. The twenty-fourth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration. The first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing the corresponding light to reach the first image sensor. The second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing the corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function. The pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene; a fifth determination module ( Figure 3 , not shown in the figure), configured to determine the picture taken by the first image sensor as the first picture, and determine the picture obtained by calibrating the spatial position of the picture taken by the second image sensor to the coordinate system of the first image sensor as the second picture.

[0417] In some alternative implementation manners of this embodiment, the depth image generation unit 302 may include: a first equation establishment module ( Figure 3 , not shown in the figure), configured to establish a first equation and a second equation for each pixel point with coordinates (m, n) in the first picture and the second picture, where the first equation is:

[0418]

[0419] where R(m, n) is the reflectance of the area corresponding to the pixel point with coordinates (m, n) in the target scene, f1 is the first light intensity function, and t is a time variable. is the light intensity when the above-mentioned first illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the above-mentioned target scene from the above-mentioned illumination source at time t and then returns to the above-mentioned first image sensor, t d (m, n) is the duration for the light to reach the area corresponding to the pixel point with coordinates (m, n) in the above-mentioned target scene from the position where the above-mentioned illumination source is located, c is the speed of light, z(m, n) is the distance between the area corresponding to the pixel point with coordinates (m, n) in the generated depth image in the above-mentioned target scene and the above-mentioned first image sensor, L is the distance between the above-mentioned first image sensor and the above-mentioned illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the above-mentioned first image sensor is located and the area corresponding to the pixel point with coordinates (m, n) in the above-mentioned target scene, and the second side is the line segment connecting the position where the above-mentioned first image sensor is located and the position where the above-mentioned illumination source is located, h1 is the above-mentioned first light transmission function, t0 is the start time of exposure, τ is the above-mentioned preset exposure duration, S1(m, n) is the pixel value of the pixel point with coordinates (m, n) in the above-mentioned first photo; the first equation is:

[0420]

[0421] where f2 is the above-mentioned second light intensity function, is the light intensity when the above-mentioned second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the above-mentioned target scene from the above-mentioned illumination source at time t and then returns to the above-mentioned first image sensor, h2 is the above-mentioned second light transmission function, S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the above-mentioned second photo; the second equation establishing module ( Figure 3 not shown in the figure) is configured to obtain a third equation according to the above-mentioned first equation and the above-mentioned second equation, and the third equation is:

[0422]

[0423] The first equation solving module ( Figure 3 not shown in the figure) is configured to solve the above-mentioned third equation to obtain z(m, n), and determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n), where S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L and α are all known; the third generating module ( Figure 3 not shown in the figure) is configured to generate the depth image of the above-mentioned target scene based on the determined depth value of each pixel point.

[0424] In some alternative implementation manners of this embodiment, the above device 300 may further include: a background light photo determination unit 304, configured to perform a twenty-sixth control operation and a twenty-seventh control operation simultaneously before acquiring the first photo and the second photo. The twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the first light transmission function. The twenty-seventh control operation includes controlling the first image sensor to capture a photo of the target scene, and determining the photo captured by the first image sensor as the background light photo; and the depth image generation unit may be further configured to generate a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0425] In some alternative implementation manners of this embodiment, the above depth image generation unit 302 may include: a third equation establishment module ( Figure 3 not shown in the figure), configured to establish a fourth equation, a fifth equation, and a sixth equation for each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo. Among them, the fourth equation is:

[0426]

[0427] where S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area corresponding to the pixel point with coordinates (m, n) in the target scene under the background light, h1 is the first light transmission function, t is a time variable, t0 is the start time of exposure, and τ is the preset exposure duration; the fifth equation is:

[0428]

[0429] where R(m, n) is the reflectivity of the area corresponding to the pixel point with coordinates (m, n) in the target scene, f1 is the first light intensity function, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor, t d (m, n) is the duration for the light to reach the area corresponding to the pixel point with coordinates (m, n) in the target scene from the position where the illumination source is located, c is the speed of light, z(m,n) is the distance between the region corresponding to the pixel at coordinates (m,n) in the generated depth image in the above target scene and the above first image sensor, L is the distance between the above first image sensor and the above illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the above first image sensor is located and the region corresponding to the pixel at coordinates (m,n) in the above target scene, and the second side is the line segment connecting the position where the above first image sensor is located and the position where the above illumination source is located, h1 is the above first light transmission function, t0 is the start time of exposure, τ is the above preset exposure duration, S1(m,n) is the pixel value of the pixel at coordinates (m,n) in the above first photo; the above sixth equation is:

[0430]

[0431] where f2 is the above second light intensity function, is the light intensity when the above second illumination light reaches the region corresponding to the pixel at coordinates (m,n) in the above second photo in the above target scene and then returns to the above first image sensor at time t, h2 is the above second light transmission function, S2(m,n) is the pixel value of the pixel at coordinates (m,n) in the above second photo; the fourth equation establishing module ( Figure 3 not shown in the figure) is configured to obtain a seventh equation according to the above fourth equation, the above fifth equation, and the above sixth equation, and the above seventh equation is:

[0432]

[0433] The second equation solving module ( Figure 3 not shown in the figure) is configured to solve the above seventh equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S b (m,n), S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; the fourth generation module ( Figure 3 not shown in the figure) is configured to generate the depth image of the above target scene based on the determined depth value of each pixel.

[0434] In some alternative implementation manners of this embodiment, the above first light transmission function is related to the coordinates of each pixel in the above first photo, and the above second light transmission function is related to the coordinates of each pixel in the above second photo.

[0435] In some alternative implementation manners of this embodiment, the above dynamic aperture is an image intensifier.

[0436] In some alternative implementation manners of this embodiment, the above dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

[0437] In some alternative implementation manners of this embodiment, the above dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of the above control device.

[0438] It should be noted that the implementation details and technical effects of each unit in the device for generating a depth image provided in the embodiments of the present disclosure can refer to the descriptions of other embodiments in the present disclosure, and will not be elaborated here.

[0439] Continue to refer to Figure 4 , Figure 4 which is a timing sequence 400 of an embodiment of a system for generating a depth image according to the present disclosure.

[0440] The system for generating a depth image in the embodiments of the present disclosure may include an illumination source, a control device, and at least one set of dynamic apertures and corresponding image sensors.

[0441] As Figure 4 shown, the timing sequence 400 of an embodiment of a system for generating a depth image according to the present disclosure includes the following steps:

[0442] Step 401, the control device acquires a first photo and a second photo.

[0443] In this embodiment, the control device in the system for generating a depth image can acquire the first photo and the second photo in various implementation manners.

[0444] Here, the first photo and the second photo are respectively photos obtained by the image sensors in the system for generating a depth image shooting a target scene according to the first shooting configuration information and the second shooting configuration information. The first shooting configuration information may include a first light transmittance function and a first light intensity function. The second shooting configuration information may include a second light transmittance function and a second light intensity function. The first light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the first photo with respect to the image sensor for shooting the first photo and time. The second light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the second photo with respect to the image sensor for shooting the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when shooting the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source in the system for generating a depth image to the target scene when shooting the second photo and time.

[0445] In this embodiment, the first light transmission function is not a constant and / or the second light transmission function is not a constant. The first light transmission function not being a constant means that the light transmittance of the dynamic aperture for taking the first photo during the exposure for taking the first photo is not fixed. The second light transmission function not being a constant means that the light transmittance of the dynamic aperture for taking the second photo during the exposure for taking the second photo is not fixed. And the first light transmission function not being a constant and / or the second light transmission function not being a constant means that at least one of the light transmittances, namely the light transmittance of the dynamic aperture for taking the first photo during the exposure for taking the first photo and the light transmittance of the dynamic aperture for taking the second photo during the exposure for taking the second photo, is not fixed.

[0446] In this embodiment, the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function. Among them, the first light transmission function being different from the second light transmission function means that the correspondence relationship between the light transmittance of the dynamic aperture for taking the first photo during the exposure for taking the first photo and time is different from the correspondence relationship between the light transmittance of the dynamic aperture for taking the second photo for the image sensor for taking the second photo and time. The first light intensity function being different from the second light intensity function means that the correspondence relationship between the light intensity of the light emitted by the illumination source in the system for generating the depth image when taking the first photo to the target scene and time is different from the correspondence relationship between the light intensity of the light emitted by the illumination source in the system for generating the depth image when taking the second photo to the target scene and time.

[0447] Here, the first photo and the second photo can be taken by the image sensor in the system for generating the depth image, and the image sensor for taking the first photo can be the same as or different from the image sensor for taking the second photo. In this way, the control device can obtain the first photo from the image sensor for taking the first photo and obtain the second photo from the image sensor for taking the first photo.

[0448] It should be noted that the control device can also adopt Figure 2A any one of the optional implementation manners (1) to (8) in the shown embodiment to execute step 401. For the specific operations and the resulting technical effects, please refer to Figure 2A the relevant specific descriptions therein, which will not be elaborated here.

[0449] Step 402, the control device generates a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0450] Since the first photo is a photo of the target scene taken using the first configuration information, and the second photo is a photo of the target scene taken using the second configuration information, in order to obtain the depth image of the target scene, the control device can adopt various implementation manners to generate the depth image of the target scene according to the first photo and the second photo obtained in step 301, as well as the first configuration information used for taking the first photo and the second configuration information used for taking the second photo. That is, determine the distance between the region corresponding to each pixel point in the depth image in the target scene and the target image sensor. Here, the target image sensor can be the image sensor that takes the first photo, or the image sensor that takes the second photo, or a hypothetical virtual image sensor that takes the depth image.

[0451] It should be noted that the control device can also execute step 402 correspondingly according to the specific implementation manner adopted in step 401. Specifically, when step 401 is executed by adopting any one of the optional implementation manners (one) to (eight) in the embodiment shown as Figure 2A below, the control device can also adopt the optional implementation manner (ten) in the embodiment shown as Figure 2A below to execute step 402. The specific operations and the technical effects brought by them can refer to the relevant specific descriptions in Figure 2A and will not be elaborated here.

[0452] In addition, when step 401 is executed by adopting any one of the optional implementation manners (one) to (eight) in the embodiment shown as Figure 2A below, in the above-mentioned timing 400, before step 401, the following step 403 can also be included:

[0453] Step 403, the control device executes the twenty-sixth control operation and the twenty-seventh control operation.

[0454] Wherein:

[0455] The twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within a preset exposure duration according to the first light transmission function.

[0456] The twenty-seventh control operation includes controlling the first image sensor to take a photo of the target scene, and determining the photo taken by the first image sensor as the background light photo.

[0457] By Figure 2AFrom the description of any one of the optional implementation manners (one) to (eight) in the illustrated embodiment, and the above descriptions of the twenty-sixth control operation and the twenty-seventh control operation, it can be seen that both the background light photo and the first photo are taken by the first image sensor, and during the process of taking the background light photo and the first photo, the light transmittance of the first dynamic aperture for the first image sensor is controlled according to the first light transmittance function within the preset exposure duration. The difference between the process of taking the background light photo and the first photo is that there is no illumination source irradiating light on the target scene in the background light photo, while there is an illumination source irradiating the target scene within the preset exposure duration according to the first light intensity function during the process of taking the first photo.

[0458] Based on the above, in step 403, the control device determines the background light photo, then step 402 can be performed as follows: The control device generates a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

[0459] It can be understood that with the background light photo, the first photo, and the second photo, at least three equations can be established, and unknowns can be introduced into the three established equations, that is, the distance between the area corresponding to each pixel point in the generated depth image in the target scene and the image sensor. By using various implementation manners to solve the equations, the above distance can be obtained, and a depth image of the target scene can be generated based on the distance obtained by the above solution.

[0460] In some implementation manners, the specific operations of the control device for generating a depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information and the technical effects brought thereby can be referred to Figure 2A the relevant descriptions in the optional implementation manner (eleven) in the illustrated embodiment, which will not be elaborated here.

[0461] The system provided by the above embodiments of the present disclosure controls the lighting source to dynamically change the light intensity through a control device, and controls the dynamic aperture to dynamically change the light transmittance, thereby obtaining a first photo and a second photo taken with different configuration information, and generating a depth image of the target scene based on the obtained first photo, second photo, and the corresponding shooting configuration information. First, the cost of the dynamic aperture is not high. Second, any currently commercially available lighting source and image sensor (e.g., currently commercially available cameras) and various lighting sources plus image sensors developed in the future (e.g., cameras developed in the future) can be used to generate a depth image. Therefore, compared with the existing methods for generating depth images, the economic cost is reduced. In addition, the image resolution of any currently commercially available ordinary camera is generally higher than that of various imaging devices used in measuring the scene distance by the imaging method. Therefore, compared with the existing methods for generating depth images, the image resolution of the generated depth image is improved.

[0462] Further referring to Figure 5 , which shows a schematic structural diagram of a camera 500 according to an embodiment of the present disclosure. Figure 5 The shown camera is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0463] As Figure 5 shown, the camera 500 includes a lighting source 501, an optical system 502, a control device 503, and at least one group of dynamic apertures and corresponding image sensors 504. The at least one group of dynamic apertures and corresponding image sensors 504 may include: a group 5041 composed of a dynamic aperture 5031A and a corresponding image sensor 5041B, a group 5042 composed of a dynamic aperture 5042A and a corresponding image sensor 5042B,..., and a group 504N composed of a dynamic aperture 504NA and a corresponding image sensor 504NB, where N is a natural number.

[0464] It should be noted that any two of the dynamic apertures 5041A, 5042A,..., 504NA may be two different dynamic apertures or the same dynamic aperture. Any two of the image sensors 5041B, 5042B,..., 504NB may be two different image sensors or the same image sensor.

[0465] Here, each dynamic aperture can dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device 503.

[0466] In some alternative implementation manners of this embodiment, the dynamic aperture may be an image intensifier. The control device 503 may control the exposure start time, exposure end time, and light transmittance of the image intensifier by controlling the voltage of the image intensifier, that is, control the exposure start time, exposure end time, and light transmittance of the dynamic aperture.

[0467] In some alternative implementation manners of this embodiment, the dynamic aperture may be a Fabry - Perot interferometer containing a nonlinear crystal. The control device 503 may control the exposure start time, exposure end time, and light transmittance of the Fabry - Perot interferometer containing a nonlinear crystal by controlling the voltage of the Fabry - Perot interferometer containing a nonlinear crystal, that is, control the exposure start time, exposure end time, and light transmittance of the dynamic aperture.

[0468] Reference is made below to Figure 6 , which shows a schematic structural diagram of a computer system 600 of a control device suitable for implementing the embodiments of the present disclosure. Figure 6 The shown control device is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0469] As Figure 6 shown, the computer system 600 includes a central processing unit (CPU, Central Processing Unit) 601, which may perform various appropriate actions and processes according to a program stored in a read - only memory (ROM, Read Only Memory) 602 or a program loaded from a storage section 608 into a random access memory (RAM, Random Access Memory) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input / output (I / O, Input / Output) interface 605 is also connected to the bus 604.

[0470] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as required. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 610 as required so that a computer program read therefrom is installed into the storage section 608 as required.

[0471] In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 609 and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the methods of the present disclosure are performed. It should be noted that the computer-readable medium described in the present disclosure can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0472] Computer program code for performing the operations of this disclosure may be written in one or more programming languages or combinations thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0473] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system that performs the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0474] The units involved in the embodiments described in this disclosure may be implemented in software or in hardware. The units described may also be provided in a processor. For example, it may be described as: a processor includes an acquisition unit and a depth image generation unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the acquisition unit may also be described as "the unit for acquiring the first photo and the second photo".

[0475] As another aspect, the present disclosure also provides a computer-readable medium, which may be included in the device described in the above embodiments; or may exist separately without being assembled into the device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the device, the device is caused to: obtain a first photo and a second photo, wherein the first photo and the second photo are respectively photos obtained by an image sensor shooting a target scene according to first shooting configuration information and second shooting configuration information, the first shooting configuration information includes a first light transmission function and a first light intensity function, the second shooting configuration information includes a second light transmission function and a second light intensity function, the first light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the first photo with respect to the image sensor for shooting the first photo and time, the second light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the second photo with respect to the image sensor for shooting the second photo and time, the first light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the first photo and time, the second light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the second photo and time, the first light transmission function is not a constant and / or the second light transmission function is not a constant, the first light transmission function is different from the second light transmission function and / or the first light intensity function is different from the second light intensity function; generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo and the second shooting configuration information.

[0476] The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solution formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. A system for generating a depth image, comprising an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors, the dynamic apertures being configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device, the control device being configured to: Obtain a first photo and a second photo, wherein, The first photo and the second photo are respectively photos obtained by the image sensor photographing a target scene according to first shooting configuration information and second shooting configuration information. The first shooting configuration information includes a first light transmittance function and a first light intensity function, and the second shooting configuration information includes a second light transmittance function and a second light intensity function. The first light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the first photo with respect to the image sensor for photographing the first photo and time. The second light transmittance function is used to characterize the correspondence between the light transmittance of the dynamic aperture for photographing the second photo with respect to the image sensor for photographing the second photo and time. The first light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the first photo and time. The second light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when photographing the second photo and time. The first light transmittance function is not a constant and / or the second light transmittance function is not a constant, the first light transmittance function is different from the second light transmittance function and / or the first light intensity function is different from the second light intensity function; Generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

2. The system according to claim 1, wherein The control device is further configured to: Generate a three-dimensional model of the target scene according to the first photo, the second photo, and the depth image.

3. The system according to claim 2, wherein, The at least one set of dynamic apertures and corresponding image sensors includes a first dynamic aperture and a corresponding first image sensor; And The obtaining of the first photo and the second photo includes: Simultaneously performing a first control operation, a second control operation, and a third control operation. Among them, the first control operation includes controlling the light intensity of the first illumination light emitted by the illumination source to the target scene within a preset exposure duration according to the first light intensity function. The pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, and the first preset ratio is greater than zero and less than or equal to 1. The second control operation includes controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the first light transmittance function. The third control operation includes controlling the first image sensor to photograph the target scene; Determine the photo taken by the first image sensor as the first photo; Simultaneously perform a fourth control operation, a fifth control operation, and a sixth control operation. Among them, the fourth control operation includes controlling the light intensity of the second illumination light emitted by the illumination source to the target scene within the preset exposure duration according to the second light intensity function. Wherein, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, and the second preset ratio is greater than zero and less than or equal to 1. The fifth control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the second light transmittance function. The sixth control operation includes controlling the first image sensor to take a photo of the target scene; Determine the photo taken by the first image sensor as the second photo.

4. The system according to claim 1, wherein The at least one group of dynamic apertures and corresponding image sensors include a first dynamic aperture and a corresponding first image sensor, and a second dynamic aperture and a corresponding second image sensor; And The obtaining of the first photo and the second photo includes: Simultaneously perform a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation. Among them, the seventh control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the first light transmittance function. The eighth control operation includes controlling the light transmittance of the second dynamic aperture to the second image sensor within the preset exposure duration according to the second light transmittance function. The ninth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration. Wherein, the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The third illumination light includes first illumination light that reaches the first image sensor after being reflected by the target scene and passing through the optical system and the first dynamic aperture. The third illumination light includes second illumination light that reaches the second image sensor after being reflected by the target scene and passing through the optical system and the second dynamic aperture. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function. The tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously take photos of the target scene; Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

5. The system according to claim 4, wherein, The optical system includes a filter configured to separate light with at least one wavelength belonging to a first preset wavelength set and light with at least one wavelength belonging to a second preset wavelength set. The third illumination light includes first illumination light with at least one wavelength belonging to the first preset wavelength set and second illumination light with at least one wavelength belonging to the second preset wavelength set. The first illumination light is reflected by the target scene, and the corresponding light reaches the first image sensor through the filter and the first dynamic aperture. The second illumination light is reflected by the target scene, and the corresponding light reaches the second image sensor through the filter and the second dynamic aperture; and Controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: Simultaneously performing an eleventh control operation and a twelfth control operation. The eleventh control operation includes controlling the illumination source to emit first illumination light with at least one wavelength belonging to the first preset wavelength set and the light intensity conforming to the first light intensity function to the target scene within the preset exposure duration. The twelfth control operation includes controlling the illumination source to emit second illumination light with at least one wavelength belonging to the second preset wavelength set and the light intensity conforming to the second light intensity function to the target scene within the preset exposure duration.

6. The system according to claim 4, wherein, The optical system includes a polarizer; the polarizer is configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The third illumination light includes first illumination light with a polarization state being the first preset polarization state and second illumination light with a polarization state being the second preset polarization state. The first illumination light is reflected by the target scene, and the corresponding light reaches the first image sensor through the polarizer and the first dynamic aperture. The second illumination light is reflected by the target scene, and the corresponding light reaches the second image sensor through the polarizer and the second dynamic aperture; and Controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: Simultaneously performing a thirteenth control operation and a fourteenth control operation. The thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state being the first preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to the first light intensity function. The fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state being the second preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to the second light intensity function.

7. The system according to claim 4, wherein The optical system includes a first beam-splitting component, a first filter component, and a second filter component. The first filter component transmits light with at least one wavelength belonging to a first preset wavelength set, and the second filter component transmits light with at least one wavelength belonging to a second preset wavelength set. The first beam-splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor are located on a first side of the first beam-splitting component, and the target scene, the second filter component, the second dynamic aperture, and the second image sensor are located on a second side of the first beam-splitting component; and Controlling the illumination source to emit third illumination light to the target scene during the preset exposure duration includes: Simultaneously performing a fifteenth control operation and a sixteenth control operation. The fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene during the preset exposure duration, and the sixteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene during the preset exposure duration.

8. The system according to claim 4, wherein, The optical system includes a second beam-splitting component and a third filter component. The second beam-splitting component is configured to split light and transmit light with at least one wavelength belonging to the first preset wavelength set, and the third filter component is configured to transmit light with at least one wavelength belonging to the second preset wavelength set. The first dynamic aperture and the first image sensor are located on a first side of the second beam-splitting component, and the target scene, the third filter component, the second dynamic aperture, and the second image sensor are located on a second side of the second beam-splitting component; and Controlling the illumination source to emit third illumination light to the target scene during the preset exposure duration includes: Simultaneously performing a seventeenth control operation and an eighteenth control operation. The seventeenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene during the preset exposure duration, and the eighteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene during the preset exposure duration.

9. The system according to claim 2, wherein The at least one group of dynamic apertures and the corresponding image sensors include a first dynamic aperture and the corresponding first image sensor. The first image sensor is an image sensor array alternately provided with first image sensor pixel units and second image sensor pixel units. The first image sensor pixel units are provided with filters that transmit light with at least one wavelength belonging to the first preset wavelength set, and the second image sensor pixel units are provided with filters that transmit light with at least one wavelength belonging to the second preset wavelength set; and Obtaining the first photo and the second photo includes: Simultaneously perform the nineteenth control operation, the twentieth control operation, the twenty-first control operation, and the twenty-second control operation. Among them, the nineteenth control operation includes controlling the illumination source to emit first illumination light including at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration. Wherein, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1. The pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the first preset wavelength set within the preset exposure duration according to the first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the second preset wavelength set within the preset exposure duration according to the second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; Obtain the photo taken by the first image sensor and determine the obtained photo as the third photo; Generate a first photo using the pixel values of each pixel point corresponding to the pixel units of the first image sensor in the third photo; Generate a second photo using the pixel values of each pixel point corresponding to the pixel units of the second image sensor in the third photo.

10. The system according to claim 2, wherein, The at least one set of dynamic aperture and the corresponding image sensor include a first dynamic aperture and the corresponding first image sensor and a second image sensor. The first image sensor is located on the first side of the first dynamic aperture, and the second image sensor and the target scene are located on the second side of the first dynamic aperture; And The obtaining of the first photo and the second photo includes: Simultaneously perform the twenty-third control operation, the twenty-fourth control operation, and the twenty-fifth control operation. Among them, the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to the first light transmittance function and the second light transmittance function respectively. The twenty-fourth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration. Among them, the first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing the corresponding light to reach the first image sensor. The second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing the corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function. The pulse width of the third illumination light is less than the third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene; Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

11. The system according to any one of claims 3-10, wherein, Generating the depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: For each pixel point with coordinates (m, n) in the first photo and the second photo, establish a first equation and a second equation. Among them, the first equation is: Among them, R(m,n) is the reflectivity of the area corresponding to the pixel point with coordinates (m,n) in the target scene, f1 is the first light intensity function, t is the time variable, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m,n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor, t d (m,n) is the duration of light from the position where the illumination source is located to the area corresponding to the pixel point with coordinates (m,n) in the target scene, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel point with coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel point with coordinates (m,n) in the target scene, the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m,n) is the pixel value of the pixel point with coordinates (m,n) in the first photo; The second equation is: where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor. h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; Obtain a third equation according to the first equation and the second equation. The third equation is: Solve the third equation to obtain z(m, n), and determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n), where S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; Generate the depth image of the target scene based on the determined depth value of each pixel point.

12. The system according to any one of claims 3-10, wherein, The control device is further configured to: For each shooting configuration information in the set of shooting configuration information, obtain a third photo and a fourth photo corresponding to the shooting configuration information based on the shooting configuration information, and generate a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo, where the shooting configuration information includes an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter, and the third photo and the fourth photo corresponding to the shooting configuration information are photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information respectively. The third shooting parameter includes a third light transmission function and a third light intensity function, and the fourth shooting parameter includes a fourth light transmission function and a fourth light intensity function. The third light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time, and the fourth light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time. The third light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the third photo and time, and the fourth light intensity function is used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when shooting the fourth photo and time. The third light transmission function is not a constant and / or the fourth light transmission function is not a constant, the third light transmission function is different from the fourth light transmission function and / or the third light intensity function is different from the fourth light intensity function, the exposure start time parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are different from each other and / or the exposure duration parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are different from each other, the third shooting parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are the same and the fourth shooting parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are the same, or the third shooting parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are different from each other and the fourth shooting parameters in the shooting configuration information of each shooting configuration information in the set of shooting configuration information are different from each other; Generate a depth image of the target scene based on the partial scene depth images of the target scene corresponding to each shooting configuration information pair in the set of preset shooting configuration information pairs.

13. The system according to claim 3, wherein, The first dynamic aperture and the first image sensor are the same device; and The controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the first light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmission function; and The controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the second light transmission function includes: Control the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the second light transmission function.

14. The system according to any one of claims 4-8, wherein, The first dynamic aperture and the first image sensor are the same device; and The controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the first light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the first light transmission function; and The controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to the second light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the first light transmission function.

15. The system according to any one of claims 3-10 and 13, wherein the dynamic aperture is further configured to, under the control of the control device, make the wavelength of the light incident on the dynamic aperture different from the wavelength of the light emitted from the dynamic aperture, and the wavelength of the light emitted from the dynamic aperture is related to the preset wavelength sensitive range of the image sensor corresponding to the dynamic aperture.

16. The system according to any one of claims 3-10, wherein, The control device is further configured to: Before acquiring the first photo and the second photo, simultaneously perform a twenty-sixth control operation and a twenty-seventh control operation, where the twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the first light transmission function, the twenty-seventh control operation includes controlling the first image sensor to take a photo of the target scene, and determining the photo taken by the first image sensor as the background light photo; and The generating the depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: Generating the depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

17. The system according to claim 16, wherein, The generating the depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: For each pixel point with coordinates (m,n) in the background light photo, the first photo, and the second photo, establish a fourth equation, a fifth equation, and a sixth equation, where the fourth equation is: Among them, S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area corresponding to the pixel point with coordinates (m, n) in the target scene under the background light, h1 is the first light transmission function, t is the time variable, t0 is the start time of exposure, and τ is the preset exposure duration; The fifth equation is: Among them, R(m,n) is the reflectivity of the area corresponding to the pixel point with coordinates (m,n) in the target scene, f1 is the first light intensity function, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m,n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor, t d (m,n) is the duration for light to reach the area corresponding to the pixel point with coordinates (m,n) in the target scene from the position where the illumination source is located, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel point with coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel point with coordinates (m,n) in the target scene, the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, S1(m,n) is the pixel value of the pixel point with coordinates (m,n) in the first photo; The sixth equation is: where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor, h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; Obtain a seventh equation according to the fourth equation, the fifth equation, and the sixth equation, and the seventh equation is: Solve the seventh equation to obtain z(m, n), and determine the depth value of the pixel at coordinates (m, n) in the generated depth image according to z(m, n), where S b (m, n), S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; Generate the depth image of the target scene based on the determined depth value of each pixel point.

18. The system according to claim 1, wherein The first light transmission function is related to the coordinates of each pixel point in the first photo, and the second light transmission function is related to the coordinates of each pixel point in the second photo.

19. The system according to claim 1, wherein The dynamic aperture is an image intensifier.

20. The system according to claim 1, wherein The dynamic aperture is a Fabry-Perot interferometer containing a nonlinear crystal.

21. The system according to claim 1, wherein The dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of the control device.

22. A method for generating a depth image, applied to a control device in a system for generating a depth image, the system for generating a depth image including an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors, the dynamic aperture being configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device, the method including: Obtaining a first photo and a second photo, where the first photo and the second photo are respectively photos obtained by the image sensor capturing a target scene according to first shooting configuration information and second shooting configuration information, the first shooting configuration information including a first light transmittance function and a first light intensity function, the second shooting configuration information including a second light transmittance function and a second light intensity function, the first light transmittance function being used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the first photo with respect to the image sensor for capturing the first photo and time, the second light transmittance function being used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the second photo with respect to the image sensor for capturing the second photo and time, the first light intensity function being used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the first photo and time, the second light intensity function being used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the second photo and time, the first light transmittance function not being a constant and / or the second light transmittance function not being a constant, the first light transmittance function being different from the second light transmittance function and / or the first light intensity function being different from the second light intensity function; Generating a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

23. The method according to claim 22, wherein, The method further includes: Generating a three-dimensional model of the target scene according to the first photo, the second photo, and the depth image.

24. The method according to claim 23, wherein The at least one set of dynamic apertures and corresponding image sensors includes a first dynamic aperture and a corresponding first image sensor; And The obtaining the first photo and the second photo includes: Simultaneously performing a first control operation, a second control operation, and a third control operation, where the first control operation includes controlling the light intensity of the first illumination light emitted by the illumination source to the target scene within a preset exposure duration according to the first light intensity function, where the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio being greater than zero and less than or equal to 1, the second control operation includes controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the first light transmittance function, and the third control operation includes controlling the first image sensor to capture a photo of the target scene; Determining the photo captured by the first image sensor as the first photo; Simultaneously perform a fourth control operation, a fifth control operation, and a sixth control operation, wherein the fourth control operation includes controlling the light intensity of the second illumination light emitted by the illumination source to the target scene within the preset exposure duration according to the second light intensity function, wherein the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, and the second preset ratio is greater than zero and less than or equal to 1. The fifth control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the second light transmittance function. The sixth control operation includes controlling the first image sensor to take a photo of the target scene; Determine the photo taken by the first image sensor as the second photo.

25. The method according to claim 22, wherein The at least one group of dynamic apertures and the corresponding image sensors include a first dynamic aperture and the corresponding first image sensor, and a second dynamic aperture and the corresponding second image sensor; And The obtaining of the first photo and the second photo includes: Simultaneously perform a seventh control operation, an eighth control operation, a ninth control operation, and a tenth control operation, wherein the seventh control operation includes controlling the light transmittance of the first dynamic aperture to the first image sensor within the preset exposure duration according to the first light transmittance function. The eighth control operation includes controlling the light transmittance of the second dynamic aperture to the second image sensor within the preset exposure duration according to the second light transmittance function. The ninth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration, wherein the pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The third illumination light includes first illumination light that is reflected by the target scene and causes corresponding light to reach the first image sensor through the optical system and the first dynamic aperture. The third illumination light includes second illumination light that is reflected by the target scene and causes corresponding light to reach the second image sensor through the optical system and the second dynamic aperture. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function. The tenth control operation includes controlling the first image sensor and the second image sensor to simultaneously take photos of the target scene; Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

26. The method according to claim 25, wherein The optical system includes a filter configured to separate light with at least one wavelength belonging to a first preset wavelength set and light with at least one wavelength belonging to a second preset wavelength set. The third illumination light includes first illumination light with at least one wavelength belonging to the first preset wavelength set and second illumination light with at least one wavelength belonging to the second preset wavelength set. The first illumination light is reflected by the target scene, and the corresponding light reaches the first image sensor through the filter and the first dynamic aperture. The second illumination light is reflected by the target scene, and the corresponding light reaches the second image sensor through the filter and the second dynamic aperture; And The controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: Simultaneously performing an eleventh control operation and a twelfth control operation. The eleventh control operation includes controlling the illumination source to emit first illumination light with at least one wavelength belonging to the first preset wavelength set and the light intensity conforming to the first light intensity function to the target scene within the preset exposure duration. The twelfth control operation includes controlling the illumination source to emit second illumination light with at least one wavelength belonging to the second preset wavelength set and the light intensity conforming to the second light intensity function to the target scene within the preset exposure duration.

27. The method according to claim 25, wherein, The optical system includes a polarizer; the polarizer is configured to separate light with polarization states being a first preset polarization state and a second preset polarization state respectively. The third illumination light includes first illumination light with a polarization state being the first preset polarization state and second illumination light with a polarization state being the second preset polarization state. The first illumination light is reflected by the target scene, and the corresponding light reaches the first image sensor through the polarizer and the first dynamic aperture. The second illumination light is reflected by the target scene, and the corresponding light reaches the second image sensor through the polarizer and the second dynamic aperture; And The controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: Simultaneously performing a thirteenth control operation and a fourteenth control operation. The thirteenth control operation includes controlling the light intensity of the first illumination light with a polarization state being the first preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to the first light intensity function. The fourteenth control operation includes controlling the light intensity of the second illumination light with a polarization state being the second preset polarization state emitted by the illumination source to the target scene within the preset exposure duration according to the second light intensity function.

28. The method according to claim 25, wherein, The optical system includes a first beam splitting component, a first filter component, and a second filter component. The first filter component transmits light with at least one wavelength belonging to a first preset wavelength set. The second filter component transmits light with at least one wavelength belonging to a second preset wavelength set. The first beam splitting component does not have a filtering function. The first filter component, the first dynamic aperture, and the first image sensor are located on a first side of the first beam splitting component. The target scene, the second filter component, the second dynamic aperture, and the second image sensor are located on a second side of the first beam splitting component; and The controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: simultaneously performing a fifteenth control operation and a sixteenth control operation, where the fifteenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within the preset exposure duration, and the sixteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration.

29. The method according to claim 25, wherein, The optical system includes a second beam splitting component and a third filter component. The second beam splitting component is configured to split light and transmit light with at least one wavelength belonging to a first preset wavelength set. The third filter component is configured to transmit light with at least one wavelength belonging to a second preset wavelength set. The first dynamic aperture and the first image sensor are located on a first side of the second beam splitting component. The target scene, the third filter component, the second dynamic aperture, and the second image sensor are located on a second side of the second beam splitting component; and The controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration includes: simultaneously performing a seventeenth control operation and an eighteenth control operation, where the seventeenth control operation includes controlling the illumination source to emit first illumination light including light with at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within the preset exposure duration, and the eighteenth control operation includes controlling the illumination source to emit second illumination light including light with at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration.

30. The method according to claim 23, wherein The at least one group of dynamic apertures and the corresponding image sensors include a first dynamic aperture and the corresponding first image sensor. The first image sensor is an image sensor array alternately provided with first image sensor pixel units and second image sensor pixel units. The first image sensor pixel units are provided with filters that transmit light with at least one wavelength belonging to the first preset wavelength set. The second image sensor pixel units are provided with filters that transmit light with at least one wavelength belonging to the second preset wavelength set; and The obtaining the first photo and the second photo includes: Simultaneously perform the nineteenth control operation, the twentieth control operation, the twenty-first control operation, and the twenty-second control operation. Among them, the nineteenth control operation includes controlling the illumination source to emit first illumination light including at least one wavelength belonging to the first preset wavelength set and having an optical intensity conforming to the first optical intensity function to the target scene within a preset exposure duration. The twentieth control operation includes controlling the illumination source to emit second illumination light including at least one wavelength belonging to the second preset wavelength set and having an optical intensity conforming to the second optical intensity function to the target scene within the preset exposure duration. Wherein, the pulse width of the first illumination light is less than a first preset ratio of the preset exposure duration, the first preset ratio is greater than zero and less than or equal to 1, the pulse width of the second illumination light is less than a second preset ratio of the preset exposure duration, the second preset ratio is greater than zero and less than or equal to 1. The twenty-first control operation includes controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the first preset wavelength set within the preset exposure duration according to the first light transmittance function and / or controlling the light transmittance of the first dynamic aperture for light with wavelengths belonging to the second preset wavelength set within the preset exposure duration according to the second light transmittance function. The twenty-second control operation includes controlling the first image sensor to take a photo of the target scene; Obtain the photo taken by the first image sensor and determine the obtained photo as the third photo; Generate a first photo using the pixel values of each pixel point corresponding to the pixel units of the first image sensor in the third photo; Generate a second photo using the pixel values of each pixel point corresponding to the pixel units of the second image sensor in the third photo.

31. The method according to claim 23, wherein The at least one set of dynamic aperture and the corresponding image sensor include a first dynamic aperture and the corresponding first image sensor and a second image sensor. The first image sensor is located on the first side of the first dynamic aperture, and the second image sensor and the target scene are located on the second side of the first dynamic aperture; And The obtaining of the first photo and the second photo includes: Simultaneously perform the twenty-third control operation, the twenty-fourth control operation, and the twenty-fifth control operation. Among them, the twenty-third control operation includes controlling the light transmittance and reflectance of the first dynamic aperture within a preset exposure duration according to the first light transmittance function and the second light transmittance function respectively. The twenty-fourth control operation includes controlling the illumination source to emit third illumination light to the target scene within the preset exposure duration. Among them, the first illumination light included in the third illumination light is reflected by the target scene and transmitted through the first dynamic aperture, causing the corresponding light to reach the first image sensor. The second illumination light included in the third illumination light is reflected by the target scene and reflected by the first dynamic aperture, causing the corresponding light to reach the second image sensor. The light intensity of the first illumination light within the preset exposure duration conforms to the first light intensity function, and the light intensity of the second illumination light within the preset exposure duration conforms to the second light intensity function. The pulse width of the third illumination light is less than a third preset ratio of the preset exposure duration, and the third preset ratio is greater than zero and less than or equal to 1. The twenty-fifth control operation includes controlling the first image sensor and the second image sensor to simultaneously take pictures of the target scene; Determine the photo taken by the first image sensor as the first photo, and determine the photo obtained by calibrating the spatial position of the photo taken by the second image sensor to the coordinate system of the first image sensor as the second photo.

32. The method according to any one of claims 24-31, wherein, Generating the depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: For each pixel point with coordinates (m, n) in the first photo and the second photo, establish a first equation and a second equation. Among them, the first equation is: Wherein, R(m,n) is the reflectivity of the area corresponding to the pixel point with coordinates (m,n) in the target scene, f1 is the first light intensity function, t is the time variable, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m,n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor, t d (m,n) is the duration for light to reach the area corresponding to the pixel point with coordinates (m,n) in the target scene from the position where the illumination source is located, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel point with coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, wherein the first side is the line segment connecting the position where the first image sensor is located and the area corresponding to the pixel point with coordinates (m,n) in the target scene, the second side is the line segment connecting the position where the first image sensor is located and the position where the illumination source is located, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, S1(m,n) is the pixel value of the pixel point with coordinates (m,n) in the first photo; The second equation is: where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor. h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; Obtain a third equation according to the first equation and the second equation. The third equation is: Solve the third equation to obtain z(m, n), and determine the depth value of the pixel point with coordinates (m, n) in the generated depth image according to z(m, n), where S1(m, n), S2(m, n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; Generate the depth image of the target scene based on the determined depth value of each pixel point.

33. The method according to any one of claims 24-31, wherein, The method further includes: For each shooting configuration information in the set of shooting configuration information, obtain a third photo and a fourth photo corresponding to the shooting configuration information based on the shooting configuration information, and generate a partial scene depth image of the target scene corresponding to the shooting configuration information based on the obtained third photo and fourth photo, where the shooting configuration information includes an exposure start time parameter, an exposure duration parameter, a third shooting parameter, and a fourth shooting parameter, and the third photo and the fourth photo corresponding to the shooting configuration information are photos obtained by the image sensor using the exposure start time indicated by the exposure start time parameter in the shooting configuration information and the exposure duration indicated by the exposure duration parameter, and shooting the target scene according to the third shooting parameter and the fourth shooting parameter in the shooting configuration information respectively. The third shooting parameter includes a third light transmission function and a third light intensity function, and the fourth shooting parameter includes a fourth light transmission function and a fourth light intensity function. The third light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the third photo with respect to the image sensor for shooting the third photo and time. The fourth light transmission function is used to characterize the correspondence between the light transmittance of the dynamic aperture for shooting the fourth photo with respect to the image sensor for shooting the fourth photo and time. The third light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the third photo and time. The fourth light intensity function is used to characterize the correspondence between the light intensity of the light emitted from the illumination source to the target scene when shooting the fourth photo and time. The third light transmission function is not a constant and / or the fourth light transmission function is not a constant. The third light transmission function is different from the fourth light transmission function and / or the third light intensity function is different from the fourth light intensity function. The exposure start time parameters in the shooting configuration information of the set of shooting configuration information are different from each other and / or the exposure duration parameters in the shooting configuration information of the set of shooting configuration information are different from each other. The third shooting parameters in the shooting configuration information of the set of shooting configuration information are all the same and the fourth shooting parameters in the shooting configuration information of the set of shooting configuration information are all the same, or the third shooting parameters in the shooting configuration information of the set of shooting configuration information are different from each other and the fourth shooting parameters in the shooting configuration information of the set of shooting configuration information are different from each other; Generate a depth image of the target scene based on the partial scene depth images of the target scene corresponding to each shooting configuration information pair in the set of preset shooting configuration information pairs.

34. The method according to claim 24, wherein The first dynamic aperture and the first image sensor are the same device; and The controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the first light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification multiple of the first image sensor within the preset exposure duration according to the first light transmission function; and The controlling the light transmittance of the first dynamic aperture with respect to the first image sensor within the preset exposure duration according to the second light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the second light transmission function.

35. The method according to any one of claims 25-29, wherein The first dynamic aperture and the first image sensor are the same device; and The controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the first light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the first light transmission function; and The controlling the light transmittance of the second dynamic aperture for the second image sensor within the preset exposure duration according to the second light transmission function includes: Controlling the photoelectric conversion efficiency and / or the photoelectric amplification factor of the first image sensor within the preset exposure duration according to the first light transmission function.

36. According to the method of any one of claims 24 - 31 and 34, the dynamic aperture is further configured to, under the control of the control device, make the wavelength of the light incident on the dynamic aperture different from the wavelength of the light emitted from the dynamic aperture, and the wavelength of the light emitted from the dynamic aperture is related to the preset wavelength sensitive range of the image sensor corresponding to the dynamic aperture.

37. According to the method of any one of claims 24-31, wherein, Before acquiring the first photo and the second photo, the method further includes: Simultaneously performing a twenty-sixth control operation and a twenty-seventh control operation, where the twenty-sixth control operation includes controlling the light transmittance of the first dynamic aperture for the first image sensor within the preset exposure duration according to the first light transmission function, the twenty-seventh control operation includes controlling the first image sensor to take a photo of the target scene, and determining the photo taken by the first image sensor as the background light photo; and The generating the depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: Generating the depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

38. The method according to claim 37, wherein, The generating the depth image of the target scene according to the background light photo, the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information includes: For each pixel point with coordinates (m, n) in the background light photo, the first photo, and the second photo, establishing a fourth equation, a fifth equation, and a sixth equation, where the fourth equation is: Among them, S b (m, n) is the pixel value of the pixel point with coordinates (m, n) in the background light photo, P0(m, n) is the light intensity of the area corresponding to the pixel point with coordinates (m, n) in the target scene under the background light, h1 is the first light transmission function, t is the time variable, t0 is the start time of exposure, and τ is the preset exposure duration; The fifth equation is: where R(m,n) is the reflectivity of the area corresponding to the pixel point with coordinates (m,n) in the target scene, f1 is the first light intensity function, is the light intensity when the first illumination light reaches the area corresponding to the pixel point with coordinates (m,n) in the first photo in the target scene from the illumination source at time t and then returns to the first image sensor, t d (m,n) is the duration for light to travel from the position of the illumination source to the area corresponding to the pixel point with coordinates (m,n) in the target scene, c is the speed of light, z(m,n) is the distance between the area corresponding to the pixel point with coordinates (m,n) in the generated depth image in the target scene and the first image sensor, L is the distance between the first image sensor and the illumination source, α is the angle between the first side and the second side, where the first side is the line segment connecting the position of the first image sensor and the area corresponding to the pixel point with coordinates (m,n) in the target scene, the second side is the line segment connecting the position of the first image sensor and the position of the illumination source, h1 is the first light transmission function, t0 is the start time of exposure, τ is the preset exposure duration, and S1(m,n) is the pixel value of the pixel point with coordinates (m,n) in the first photo; The sixth equation is: where f2 is the second light intensity function, is the light intensity when the second illumination light reaches the area corresponding to the pixel point with coordinates (m, n) in the target scene from the illumination source at time t and then returns to the first image sensor. h2 is the second light transmission function, and S2(m, n) is the pixel value of the pixel point with coordinates (m, n) in the second photo; Obtaining a seventh equation according to the fourth equation, the fifth equation, and the sixth equation, where the seventh equation is: Solve the seventh equation to obtain z(m,n), and determine the depth value of the pixel at coordinates (m,n) in the generated depth image according to z(m,n), where S b (m,n), S1(m,n), S2(m,n), h1, h2, t0, τ, f1, f2, the speed of light c, L, and α are all known; Generating the depth image of the target scene based on the determined depth value of each pixel point.

39. The method according to claim 22, wherein, The first light transmission function is related to the coordinates of each pixel point in the first photo, and the second light transmission function is related to the coordinates of each pixel point in the second photo.

40. The method according to claim 22, wherein, The dynamic aperture is an image intensifier.

41. The method according to claim 22, wherein The dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

42. The method according to claim 22, wherein, The dynamic aperture is configured to dynamically change the light transmittance to a positive number greater than or equal to 0 and less than or equal to 1 or greater than 1 under the control of the control device.

43. A device for generating a depth image, applied to a control device in a system for generating a depth image, the system for generating a depth image including an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors, the dynamic aperture being configured to dynamically change the light transmittance, the exposure start time, and the exposure end time under the control of the control device, the device including: An acquisition unit configured to acquire a first photo and a second photo, where the first photo and the second photo are respectively photos obtained by the image sensor capturing a target scene according to first shooting configuration information and second shooting configuration information, the first shooting configuration information including a first light transmittance function and a first light intensity function, the second shooting configuration information including a second light transmittance function and a second light intensity function, the first light transmittance function being used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the first photo with respect to the image sensor for capturing the first photo and time, the second light transmittance function being used to characterize the correspondence between the light transmittance of the dynamic aperture for capturing the second photo with respect to the image sensor for capturing the second photo and time, the first light intensity function being used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the first photo and time, the second light intensity function being used to characterize the correspondence between the light intensity of the light emitted by the illumination source to the target scene when capturing the second photo and time, the first light transmittance function not being a constant and / or the second light transmittance function not being a constant, the first light transmittance function being different from the second light transmittance function and / or the first light intensity function being different from the second light intensity function; A depth image generation unit configured to generate a depth image of the target scene according to the first photo, the first shooting configuration information, the second photo, and the second shooting configuration information.

44. A camera, wherein, The camera includes an illumination source, an optical system, a control device, and at least one set of dynamic apertures and corresponding image sensors, the dynamic aperture being configured to dynamically change the light transmittance, as well as the exposure start time and the exposure end time, between the exposure start time and the exposure end time under the control of the control device; wherein the control device is configured to: apply a voltage to the dynamic aperture that changes over time to control the light transmittance of the dynamic aperture.

45. The camera according to claim 44, wherein, The dynamic aperture is an image intensifier.

46. The camera according to claim 44, wherein, The dynamic aperture is a Fabry - Perot interferometer containing a nonlinear crystal.

47. An electronic device, including: One or more processors; A storage device having stored thereon one or more programs, When the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method according to any one of claims 22 - 42.

48. A computer-readable storage medium having a computer program stored thereon, wherein, The computer program, when executed by one or more processors, implements the method according to any one of claims 22 - 42.

Images