Structured light projector, structured light system, and depth calculation method

Abstract

The structural light projector, the structural light system and the depth calculation method provided in the specification can make the super surface lens suitable for two or more different wavelengths of laser at the same time by customizing the size and arrangement angle of the diffraction unit of the super surface lens, so that two different structural light patterns are projected by two wavelengths of laser, so that the structural light projector can be applied to a wider range of application scenarios. The structural light system and the depth calculation method can select at least one of the two depth images corresponding to the two wavelengths according to the proportion of the speckle on the target object in the area occupied by the target object in the use scene, to calculate the depth information of the target object, thereby improving the accuracy of depth calculation.

Description

Technical Field

[0001] This specification relates to the field of depth camera technology, and in particular to a structured light projector, a structured light system, and a depth calculation method. Background Technology

[0002] Structured light distance measurement is an important depth measurement method that can measure the distance to a target and obtain a depth image containing the target's depth value. With the development of technology, structured light distance measurement is increasingly being applied in various fields, such as 3D reconstruction, facial recognition, human-computer interaction, consumer electronics, autonomous driving, and AR / VR. Current structured light solutions generally involve emitting a laser beam, which passes through a diffractive optical element (DOE) to form a specific structured light pattern, such as a speckle pattern. This pattern is then distorted by reflection from a target object (such as a hand or face), and the distorted pattern is received by an image sensor. The computer processes the pattern received by the image sensor and uses triangulation to determine the distance between the target object and the laser emitter.

[0003] However, in existing structured light solutions, the design of the DOE (Design for Equipment) element must match the wavelength of the laser emitted by the laser emitter. When the laser wavelength is fixed, the DOE element can only project a fixed structured light pattern. When using lasers of different wavelengths, existing DOE elements may fail to project a clear structured light pattern. Therefore, existing structured light solutions are only suitable for optimal design distances. Taking face recognition as an example, for a fixed speckle pattern, as the distance between the target object and the structured light system increases, the area occupied by the face decreases, and the number of speckles on the face gradually decreases with increasing distance, while the relative area of ​​each speckle on the face increases. This leads to a decrease in face accuracy resolution. Moreover, existing structured light solutions can only select one laser wavelength. A single wavelength limits the application scenarios or introduces additional power consumption.

[0004] Therefore, there is a need to provide a structured light projector, structured light system, and depth calculation method that can support multiple wavelengths and project a variety of different structured light patterns, in order to expand the application scenarios and improve the depth calculation results. Summary of the Invention

[0005] This specification provides a structured light projector, a structured light system, and a depth calculation method that can support multiple wavelengths and project various structured light patterns.

[0006] In a first aspect, this specification provides a structured light projector, including a laser component and a metasurface lens. The laser component includes a first laser and a second laser. The first laser emits a first laser beam of a first wavelength when operating; the second laser emits a second laser beam of a second wavelength when operating, wherein the first wavelength is greater than the second wavelength. The metasurface lens includes an incident light side and an exit light side. The incident light side is opposite to the laser component, and the exit light side includes an anisotropic metasurface. The first laser beam enters the metasurface lens from the incident light side and exits from the exit light side to form a first structured light pattern. The second laser beam enters the metasurface lens from the incident light side and exits from the exit light side to form a second structured light pattern. The speckle density of the first structured light pattern is less than the speckle density of the second structured light pattern.

[0007] In some embodiments, the first laser is linearly polarized light vibrating along a first direction, and the second laser is linearly polarized light vibrating along a second direction, wherein the first direction and the second direction form a preset angle, the preset angle being greater than 0, so that the metasurface lens performs different phase modulation on the first laser and the second laser, thereby making the first structured light pattern different from the second structured light pattern.

[0008] In some embodiments, the preset included angle is a right angle.

[0009] In some embodiments, the metasurface lens further includes a lens substrate, the lens substrate including an incident surface and an exit surface, the incident surface being located on the light-incident side and the exit surface being located on the light-outceasing side, the metasurface being formed on the exit surface of the lens substrate; the metasurface includes a plurality of diffraction units, which are composed of nanopillars of subwavelength scales with different aspect ratios arranged in a predetermined manner and at a predetermined distance, wherein the size and rotation angle of each of the plurality of diffraction units are determined based on a first phase distribution corresponding to the first wavelength, a second phase distribution corresponding to the second wavelength, the position of the current diffraction unit, and the predetermined included angle.

[0010] In some embodiments, the metasurface lens further includes a dielectric elastomer located on the incident surface of the lens substrate. When the dielectric elastomer is driven by a voltage, the deformation of the dielectric elastomer causes the lens substrate to deform, and the plurality of diffraction units are displaced. By controlling the magnitude of the voltage, the deformation of the lens substrate is controlled, thereby controlling the displacement of the plurality of diffraction units, so that the focal point of the thin-film zoom lens moves within the predetermined range.

[0011] In some embodiments, the structured light projector further includes a deflection component located between the laser assembly and the metasurface lens to deflect the first laser and the second laser, so that the first laser and the second laser pass through the deflection component and then enter the metasurface lens perpendicularly.

[0012] Secondly, this specification also provides a structured light system, including the structured light projector, image sensor, and computing device described in the first aspect of this specification. The structured light projector projects a structured light pattern onto a target field of view during operation. The structured light pattern includes either the first structured light pattern or the second structured light pattern. The image sensor acquires a depth image of the target field of view during operation. The depth image includes an image formed by the structured light pattern projected onto a target object. The computing device is communicatively connected to the structured light projector and the image sensor during operation, controls the structured light pattern projected by the structured light projector, receives the depth image, and calculates target depth information of the target object based on the depth image.

[0013] In some embodiments, controlling the structured light pattern projected by the structured light projector and receiving the depth image includes: controlling the structured light projector to turn on the first laser and project the first structured light pattern into the target field of view; and receiving a first depth image acquired by the image sensor, the first depth image including an image formed by the first structured light pattern projected onto the target object, the depth image including the first depth image; determining the depth information missing ratio corresponding to the target object in the first depth image, the depth information missing ratio including the ratio of the area of ​​the target object in the first depth image that does not contain the first structured light pattern to the total area of ​​the target object; and controlling the structured light projector based on the depth information missing ratio.

[0014] In some embodiments, controlling the structured light projector based on the depth information missing ratio includes: determining that the depth information missing ratio exceeds a preset threshold range or is within the threshold range, controlling the structured light projector to turn on the second laser, project the second structured light pattern into the target field of view, and receiving a second depth image acquired by the image sensor, the second depth image including the image formed by the second structured light pattern projected onto the target object, the depth image including the second depth image; or determining that the depth information missing ratio of the first depth image is less than the threshold range, controlling the structured light projector to turn off.

[0015] In some embodiments, calculating the target depth information of the target object based on the depth image includes one of the following: determining that the proportion of missing depth information in the first depth image is less than the threshold range, and determining the target depth information based on the first depth image; determining that the proportion of missing depth information exceeds the threshold range, and determining the target depth information based on the second depth image; and determining that the proportion of missing depth information is within the threshold range, and determining the target depth information based on the first depth image and the second depth image.

[0016] In some embodiments, determining the target depth information based on the first depth image and the second depth image includes: determining first depth information of the target object based on the first depth image; determining second depth information of the target object based on the second depth image; and performing a fusion process on the first depth information and the second depth information to determine the target depth information.

[0017] Thirdly, this specification also provides a depth calculation method for the structured light system described in the second aspect of this specification. The depth calculation method includes, through the computing device, controlling the structured light pattern projected by the structured light projector and receiving the depth image; and calculating target depth information of the target object based on the depth image.

[0018] In some embodiments, controlling the structured light pattern projected by the structured light projector and receiving the depth image includes: controlling the structured light projector to turn on the first laser and project the first structured light pattern into the target field of view; and receiving a first depth image acquired by the image sensor, the first depth image including an image formed by the first structured light pattern projected onto the target object, the depth image including the first depth image; determining the depth information missing ratio corresponding to the target object in the first depth image, the depth information missing ratio including the ratio of the area of ​​the target object in the first depth image that does not contain the first structured light pattern to the total area of ​​the target object; and controlling the structured light projector based on the depth information missing ratio.

[0019] In some embodiments, controlling the structured light projector based on the depth information missing ratio includes: determining that the depth information missing ratio exceeds a preset threshold range or is within the threshold range, controlling the structured light projector to turn on the second laser, project the second structured light pattern into the target field of view, and receiving a second depth image acquired by the image sensor, the second depth image including the image formed by the second structured light pattern projected onto the target object, the depth image including the second depth image; or determining that the depth information missing ratio of the first depth image is less than the threshold range, controlling the structured light projector to turn off.

[0020] In some embodiments, calculating the target depth information of the target object based on the depth image includes one of the following: determining that the proportion of missing depth information in the first depth image is less than the threshold range, and determining the target depth information based on the first depth image; determining that the proportion of missing depth information exceeds the threshold range, and determining the target depth information based on the second depth image; and determining that the proportion of missing depth information is within the threshold range, and determining the target depth information based on the first depth image and the second depth image.

[0021] In some embodiments, determining the target depth information based on the first depth image and the second depth image includes: determining first depth information of the target object based on the first depth image; determining second depth information of the target object based on the second depth image; and performing a fusion process on the first depth information and the second depth information to determine the target depth information.

[0022] As can be seen from the above technical solutions, the structured light projector, structured light system, and depth calculation method provided in this specification, by customizing the size and arrangement angle of the diffraction units of the metasurface lens, enable the metasurface lens to be simultaneously applicable to two or more different wavelengths of laser light, allowing the two wavelengths of laser light to project two different structured light patterns, thus making the structured light projector applicable to a wider range of application scenarios. The structured light system and depth calculation method can select at least one from two depth images corresponding to two wavelengths based on the proportion of speckle on the target object's area in the application scenario, to calculate the depth information of the target object, thereby improving the accuracy of depth calculation.

[0023] Other functionalities of the structured light projector, structured light system, and depth calculation methods provided in this specification will be partially listed in the following description. The figures and examples described below will be readily apparent to those skilled in the art. The inventive aspects of the structured light projector, structured light system, and depth calculation methods provided in this specification can be fully understood through practice or use of the methods, apparatus, and combinations described in the detailed examples below. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram of a structured light projector provided according to an embodiment of this specification is shown;

[0026] Figure 2 A schematic diagram of the structure of a metasurface lens provided according to an embodiment of this specification is shown;

[0027] Figure 3 A schematic diagram of the structure of a diffraction unit provided according to an embodiment of this specification is shown;

[0028] Figure 4 A schematic diagram of another diffraction unit provided according to an embodiment of this specification is shown;

[0029] Figure 5 A schematic diagram of a structured light system provided according to an embodiment of this specification is shown;

[0030] Figure 6 A schematic diagram of a computing device according to an embodiment of this specification is shown; and

[0031] Figure 7 A flowchart of a depth calculation method provided according to an embodiment of this specification is shown. Detailed Implementation

[0032] The following description provides specific application scenarios and requirements for this specification, intended to enable those skilled in the art to make and use the contents of this specification. Various partial modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but rather to the widest scope consistent with the claims.

[0033] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not restrictive. For example, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” used herein may also include the plural forms. When used in this specification, the terms “comprising,” “including,” and / or “containing” mean that the associated integers, steps, operations, elements, and / or components are present, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups, or that other features, integers, steps, operations, elements, components, and / or groups may be added to the system / method.

[0034] Considering the following description, these and other features of this specification, as well as the operation and function of the related components of the structure, and the economy of assembly and manufacture of the parts, can be significantly improved. All of these form part of this specification with reference to the accompanying drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0035] The flowcharts used in this specification illustrate operations implemented according to some embodiments of this specification. It should be clearly understood that the operations in the flowcharts may not be implemented in a sequential order. Instead, the operations may be implemented in reverse order or simultaneously. Furthermore, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0036] In existing structured light schemes, to ensure the beam diffracted by a DOE element can be focused on the imaging plane, the DOE element's structural design must match the wavelength of the laser emitted by the laser emitter. When the focal length and laser wavelength are fixed, the phase distribution corresponding to each diffraction unit in the DOE element is also fixed. Therefore, the beam diffracted by the DOE element can only project a fixed structured light pattern. When the wavelength of the laser emitted by the laser emitter changes, the phase distribution required for the laser to image on the imaging plane also changes accordingly. At this time, the phase distribution formed by the existing DOE element can no longer meet the requirements of the new laser, and it may be impossible for the new laser to present a clear image on the imaging plane.

[0037] This specification provides a structured light projector that, by customizing the size and arrangement angle of the diffraction units of a metasurface lens, enables the metasurface lens to be simultaneously adapted to two or more different wavelengths of laser light, allowing the two wavelengths of laser light to project two different structured light patterns, thus making the structured light projector applicable to a wider range of application scenarios.

[0038] Figure 1A schematic diagram of a structured light projector 100 according to an embodiment of this specification is shown. The structured light projector 100 can be used to project a structured light pattern outward for depth information calculation. The structured light projector 100 can be applied to any scenario requiring depth calculation or distance measurement. For example, 3D reconstruction scenarios, face recognition scenarios, human-computer interaction scenarios, consumer electronics scenarios, autonomous driving scenarios, AR / VR scenarios, etc. For ease of demonstration, the following description will use the application of the structured light projector 100 to a face recognition scenario as an example. Those skilled in the art should understand that the application of the structured light projector 100 to other scenarios is also within the scope of protection of this specification. The structured light projector 100 may include a laser component 120 and a metasurface lens 140. In some embodiments, the structured light projector 100 may also include a bending member 160. In some embodiments, the structured light projector 100 may also include a base ( Figure 1 (Not shown in the image).

[0039] The base serves as a mounting component for the laser assembly 120, the metasurface lens 140, and the transition component 160. The laser assembly 120, the metasurface lens 140, and the transition component 160 can be mounted on the base.

[0040] Laser component 120 can emit laser light outwards. The laser light can be any form of electromagnetic wave. In some embodiments, the laser light can be visible light. In some embodiments, the laser light can be invisible light, such as infrared light, near-infrared light, etc. Since the user cannot see invisible light, it can be detected without human intervention. For ease of demonstration, we will describe the laser component 120 emitting invisible light (such as near-infrared light) as an example. Laser component 120 can emit laser light of various wavelengths. In some embodiments, laser component 120 can emit laser light of two different wavelengths. In some embodiments, laser component 120 can emit laser light of more than two different wavelengths. Those skilled in the art should understand that the difference between two or more different wavelengths of laser light is only in quantity; the design method of the structured light emitter 100 is the same. This specification will describe the laser component 120 emitting laser light of two different wavelengths as an example. Those skilled in the art should understand that the laser component 120 emitting laser light of more than two different wavelengths is also within the scope of this specification.

[0041] The laser assembly 120 may include a first laser 121 and a second laser 122. The first laser 121 emits a first laser beam of a first wavelength. The second laser 122 emits a second laser beam of a second wavelength. The first wavelength differs from the second wavelength. For ease of description, we will use an example where the first wavelength is greater than the second wavelength. In face recognition scenarios, commonly used infrared wavelengths are 850nm and 940nm. In low-light conditions, such as indoors, 850nm infrared light has a higher photoelectric conversion ratio and higher reflectivity than 940nm. In high-light conditions, such as outdoors, 940nm infrared light has higher anti-interference capability than 850nm. For ease of description, this specification uses an example where the first wavelength λ1 = 940nm and the second wavelength λ2 = 850nm. Those skilled in the art should understand that other wavelengths for the first wavelength λ1 and the second wavelength λ2 are also within the scope of this specification.

[0042] The first laser can be linearly polarized light vibrating along a first direction. The second laser can be linearly polarized light vibrating along a second direction. The first direction and the second direction are not parallel. The first direction can form a preset angle with the second direction. The preset angle is greater than 0 degrees. In some embodiments, the preset angle can be an acute angle. In some embodiments, the preset angle can be a right angle, meaning the first laser and the second laser are orthogonally polarized.

[0043] In some embodiments, the first laser 121 may include a first laser emitter 121-1 and a first collimator 121-2. The first laser emitter 121-1 can be used to emit the first laser beam. The first collimator 121-2 can be located on the light-emitting side of the first laser emitter 121-1 and collimates the first laser beam emitted by the first laser emitter 121-1 so that the first laser beam passing through the first collimator 121-2 is parallel light.

[0044] In some embodiments, the second laser 122 may include a second laser emitter 122-1 and a second collimator 122-2. The second laser emitter 122-1 can be used to emit the second laser. The second collimator 122-2 can be located on the light-emitting side of the second laser emitter 122-1 and collimates the second laser emitted by the second laser emitter 122-1 so that the second laser passing through the second collimator 122-2 is parallel light.

[0045] The first laser 121 and the second laser 122 can be installed in any relative position. In some embodiments, the first laser 121 and the second laser 122 can be installed in parallel. In some embodiments, the first laser 121 and the second laser 122 can be installed at a certain angle, such as orthogonal at 90°. In some embodiments, the first laser 121 and the second laser 122 can also be installed relative to each other. The propagation directions of the first laser and the second laser 122 can be the same or different. The laser projector 100 can adjust the propagation direction of the first laser and / or the second laser through the deflection member 160, so that the first laser and the second laser can propagate in a direction perpendicular to the metasurface lens 140 and enter the metasurface lens 140.

[0046] The deflection member 160 can be located between the laser assembly 120 and the metasurface lens 140. The deflection member 160 can deflect the first laser and the second laser, changing their optical paths so that they can enter the metasurface lens 140 perpendicularly after passing through the deflection member 160. The deflection member 160 can be any form capable of changing the propagation direction of the optical path. In some embodiments, the deflection member 160 can be a deflection prism. In some embodiments, the deflection member 160 can be a combination of multiple deflection prisms. In some embodiments, the deflection member 160 can be a combination of reflecting devices. In some embodiments, the deflection member 160 can be a combination of refractive devices.

[0047] A metasurface lens 140 can be located on the side of the laser assembly 120 and the deflector 160 from which the laser is emitted. The first or second laser emitted from the laser assembly 120 and the deflector 160 can pass through the metasurface lens 140. The metasurface lens 140 can perform phase modulation on the first and second lasers incident therein, causing them to present a clear structured light pattern on the imaging plane where the focal point is located. The metasurface lens 140 can include an incident light side 141 and an exit light side 142. The incident light side 141 can be opposite to the laser assembly 120. The exit light side 142 can be opposite to the incident light side 141 and located on the side away from the laser assembly 120. The first or second laser enters from the incident light side 141 and exits from the exit light side 142. The metasurface lens 140 refracts and diffracts the first or second laser light incident from the incident side 141 of the metasurface lens 140, causing the light rays emitted from the exit side 142 to converge at the focal point of the metasurface lens 140, forming a structured light pattern composed of a speckle pattern. Specifically, the first laser light, after passing through the metasurface lens 140, can form a clear first structured light pattern on the imaging plane. The second laser light, after passing through the metasurface lens 140, can form a clear second structured light pattern on the imaging plane. The structured light pattern can be either the first or the second structured light pattern. The imaging plane can be a plane perpendicular to the central optical path located at the focal point, or it can be any plane within the depth of field perpendicular to the central optical path, including the focal point.

[0048] Because the first wavelength λ1 of the first laser is different from the second wavelength λ2 of the second laser, the first structured light pattern is different from the second structured light pattern. The overall divergence angle of the speckle pattern is proportional to the incident wavelength. That is, the longer the incident wavelength, the larger the overall divergence angle of the speckle pattern. At the same time, the number of specks in the speckle pattern is inversely proportional to the incident wavelength. That is, the shorter the incident wavelength, the higher the photoelectric conversion efficiency corresponding to the wavelength, and the more specks in the corresponding speckle pattern. Since the first wavelength λ1 is greater than the second wavelength λ2, the speckle divergence angle of the first structured light pattern is greater than that of the second structured light pattern, and the number of specks in the first structured light pattern is less than that in the second structured light pattern. Therefore, the speckle density of the first structured light pattern is less than that of the second structured light pattern. In other words, when the target object is within the depth of field allowed by the focal length of the metasurface lens 140, the number of specks formed by the first structured light pattern on the surface of the target object is less than the number of specks formed by the second structured light pattern on the surface of the target object. In other words, within the depth of field allowed by the focal length of the metasurface lens 140, the farther the target object is from the metasurface lens 140, the fewer the number of speckles on the target object, the larger the area ratio of a single speckle to the target object, and the lower the accuracy of depth measurement. Therefore, the first structured light pattern is suitable for short-range depth information measurement, while the second structured light pattern is suitable for long-range depth information measurement.

[0049] like Figure 1 As shown, the metasurface lens 140 may include a lens substrate 144 and a metasurface 146. In some embodiments, the metasurface lens may also include a dielectric elastomer 148.

[0050] The lens substrate 144 can be made of a transparent thin-film material. The lens substrate 144 can be a layer of uniform thickness. The material of the lens substrate 144 includes, but is not limited to, silicon oxide, single-crystal silicon, silicon nitride, and silicon oxynitride. The lens substrate 144 can be a carrier for the metasurface 146. The lens substrate 144 can deform under external force, thereby changing the arrangement rule of the metasurface 146 and thus changing the focal length of the metasurface lens 140. The lens substrate 144 can include an incident surface 144-1 and an exit surface 144-2. The incident surface 144-1 faces the light-incident side 141, and the exit surface 144-2 faces the light-exiting side 142.

[0051] The metasurface 146 can be an anisotropic metasurface. Anisotropy means that all or part of the chemical, physical, and other properties of the metasurface 146 change with the direction, exhibiting different properties in different directions. For example, the refractive or diffraction properties of the metasurface 146 differ in orthogonal directions. Anisotropy is a common property in materials and media, varying greatly in scale, from crystals to various materials in daily life, and even the Earth's medium. The metasurface 146 can be disposed on the light-emitting side 142. Specifically, the metasurface 146 can be formed on the exit surface 144-2 of the lens substrate 144.

[0052] Figure 2 A schematic diagram of the structure of a metasurface lens 140 provided according to an embodiment of this specification is shown. Figure 2 for Figure 1 A top view. (e.g.) Figure 2 As shown, O-XY is the coordinate system of the metasurface lens 140. Point O is the geometric center of the metasurface lens 140. Figure 2 The metasurface lens 140 shown is rectangular. Those skilled in the art will understand that metasurface lenses 140 of other shapes are also within the scope of this specification. Figure 2 As shown, a metasurface 146 is disposed on the exit surface 144-2 of the lens substrate 144. The material of the metasurface 146 includes, but is not limited to, silicon, silicon nitride, silicon oxide, and silicon oxynitride. The refractive index of the metasurface 146 material is higher than that of the lens substrate 144. The metasurface 146 may include multiple diffraction units 146-1. The multiple diffraction units 146-1 can be a periodic array composed of nanopillars with different aspect ratios at subwavelength scales arranged in a predetermined manner and at predetermined distances. Specifically, the geometric centers corresponding to each diffraction unit 146-1 are arranged at a predetermined distance to form a predetermined array. The array can be a circular array, a ring array, a rectangular array, etc. The array can be a uniform array or a non-uniform array. The distance between the geometric centers of adjacent diffraction units 146-1 in the array can be the same or different. The position coordinates of the geometric center of each diffraction unit 146-1 have been pre-stored in other computing devices, such as the computing device 300 described later.

[0053] Multiple diffraction units 146-1 can perform phase modulation on the first laser and the second laser, so that the first laser or the second laser enters from the incident side 141 through multiple diffraction units 146-1, exits from the emitting side 142, and converges at the focal point of the metasurface lens 140. When the focal length of the metasurface lens 140 is determined, the phase distribution corresponding to the wavelength is only related to the wavelength and the position of the diffraction units 146-1. When the position of the diffraction units 146-1 is also determined, the phase distribution corresponding to the wavelength is only related to the wavelength. The phase distribution can be the phase modulation of the incident light by each diffraction unit 146-1 in the metasurface 146. In some embodiments, the phase distribution can also be the phase difference between the emitted light and the incident light corresponding to each diffraction unit 146-1 in the metasurface 146. The phase distribution corresponding to diffraction units 146-1 at different positions can be different. Similarly, the shape and size of diffraction units 146-1 at different positions can also be different. Specific phase distributions can be achieved by adjusting the size and rotation angle of diffraction units 146-1 at different positions in the metasurface 146. As mentioned earlier, the diffraction units 146-1 can be anisotropic nanopillars. The shape of the diffraction units 146-1 can be any anisotropic shape, such as cuboids, elliptical cylinders, prisms, or even other irregular shapes. Since the first wavelength λ1 of the first laser is different from the second wavelength λ2 of the second laser, the phase distributions corresponding to the first laser and the second laser are different at the same focal length. When designing the metasurface lens 140, the structure of each diffraction unit 146-1 needs to be customized to simultaneously meet the phase control requirements of the first laser and the second laser. Specifically, the size and rotation angle of each diffraction unit 146-1 are determined based on the first phase distribution corresponding to the first wavelength λ1, the second phase distribution corresponding to the second wavelength λ2, the current position of the diffraction unit 146-1, and the preset included angle.

[0054] Figure 3 A schematic diagram of the structure of a diffraction unit 146-1 provided according to an embodiment of this specification is shown. Figure 3 The diffraction unit 146-1 shown can be one of multiple diffraction units 146-1. For ease of description, we will... Figure 3 The geometric center of the diffraction unit 146-1 shown is defined as (X0, Y0) in the O-XY coordinate system. The diffraction unit 146-1 can be a single nanopillar or a combination of nanopillars of various shapes. For ease of description, Figure 3The diffraction unit 146-1 shown is a single nanopillar. When the position (X0, Y0) of the diffraction unit 146-1 is determined, the first phase distribution of the first wavelength λ1 corresponding to the current diffraction unit 146-1 (X0, Y0) can be determined based on the focal length of the metasurface lens 140 and the position (X0, Y0) of the diffraction unit 146-1 (X0, Y0). and the second phase distribution of the second wavelength λ2 When the first and second laser beams are incident perpendicularly to the diffraction unit 146-1 along a direction perpendicular to the XY plane, they can be decomposed into polarization components in the X and Y directions. When the diffraction unit 146-1 is a cuboid and its rotation angle θ is 0, the phase modulation of the polarization component in the X direction is mainly controlled by the length b of the diffraction unit 146-1, while the phase modulation of the polarization component in the Y direction is mainly controlled by the width a of the diffraction unit 146-1. Since the diffraction unit 146-1 is anisotropic, the phase modulation of the polarization component in the X direction and the phase modulation of the polarization component in the Y direction are different. As mentioned earlier, the first direction and the second direction form a preset angle. Therefore, the polarization components of the first and second laser beams in the X and Y directions are different. Thus, the phase modulation of the first and second laser beams by the diffraction unit 146-1 is also different. The specific first phase distribution of the first wavelength λ1 can be achieved by changing the length b, width a, rotation angle θ of the diffraction unit 146-1, and the height of the cuboid. and the second phase distribution of the second wavelength λ2

[0055] As previously mentioned, in some embodiments, the first direction is orthogonal to the second direction. In some embodiments, the first direction is the X direction and the second direction is the Y direction.

[0056] Figure 3 The shape of the diffraction unit 146-1 shown is a cuboid. Figure 3 The diffraction unit 146-1 shown is merely illustrative. Those skilled in the art will understand that other shapes of the diffraction unit 146-1 are also within the scope of this specification, such as elliptical cylinders.

[0057] Figure 4 A schematic diagram of another diffraction unit 146-1 provided according to an embodiment of this specification is shown. Figure 4 The diffraction unit 146-1 shown is a combination of various nano-micro pillars of different shapes. Its design method is similar to the aforementioned design method and will not be repeated here.

[0058] As previously described, multiple diffraction units 146-1 are distributed on the lens substrate 144 according to a predetermined pattern and distance. The lens substrate 144 is made of a transparent thin film material, which can deform under external force. When the lens substrate 144 deforms, the multiple diffraction units 146-1 will shift along with the deformation of the lens substrate 144, causing a change in the relative distance between the multiple diffraction units 146-1, thereby changing the focal length of the metasurface lens 140 and the position of the focal point, thus achieving a zoom function. By controlling the deformation of the lens substrate 144, the displacement of the multiple diffraction units 146-1 and the relative distance between the multiple diffraction units 146-1 can be controlled, thereby achieving control over the focal length of the metasurface lens 140, allowing the focal point of the metasurface lens 140 to move within a predetermined range. Figure 1 As shown, in some embodiments, the metasurface lens 140 may also include a dielectric elastomer 148.

[0059] A dielectric elastomer 148 can be located on the incident surface 144-1 of the lens substrate 144. The dielectric elastomer 148 can be a transparent, conductive elastomer. When the dielectric elastomer 148 is driven by a voltage, it deforms under the stimulation of the voltage. The deformation of the dielectric elastomer 148 varies depending on the magnitude of the voltage. The deformation of the dielectric elastomer 148 can cause the lens substrate 144 to deform, resulting in displacement of multiple diffraction units 146-1. The relative distance between the multiple diffraction units 146-1 changes, thereby changing the focal length and the position of the focal point of the metasurface lens 140, achieving a zoom function. The computing device 300 can control the deformation of the dielectric elastomer 148 by controlling the magnitude of the voltage, thereby controlling the deformation of the lens substrate 144 and the relative distance between the multiple diffraction units 146-1. This allows the focal point of the metasurface lens 140 to move within the predetermined range, making the structured light projector 100 suitable for depth information calculation and distance measurement of target objects at different distances, thus broadening its application scenarios. For example, the distance range for face recognition differs on different devices, and the distance measurement range differs between face recognition scenarios and 3D reconstruction scenarios. The structured light projector 100 provided in this specification can be applied to a variety of different smart devices without changing its structure.

[0060] It should be noted that in some embodiments, the metasurface lens 140 may not include the electrolyte elastomer 148. In this case, the metasurface lens 140 can be used at a fixed focal length without zooming.

[0061] Figure 5A schematic diagram of a structured light system 001 according to an embodiment of this specification is shown. The structured light system 001 can acquire depth information of a target object within a target field of view and perform depth calculation. The target field of view can be the working range of the structured light system 001. The target field of view can be any spatial region, such as a supermarket, shopping mall, restaurant, etc. The target object can be any object to be detected that requires depth information acquisition and depth calculation. The structured light system 001 can be applied to any occasion requiring depth information acquisition and depth calculation, such as 3D reconstruction scenarios, face recognition scenarios, autonomous driving scenarios, etc. For ease of demonstration, this specification will describe the application of the structured light system 001 to a face recognition scenario as an example. A face recognition scenario can be any scenario requiring face recognition of the object to be detected, such as face payment scenarios, face unlock scenarios, face authentication scenarios, etc. In a face recognition scenario, the target object can be the face to be recognized. Figure 5 As shown, the structured light system 001 may include the aforementioned structured light projector 100, image sensor 200, and computing device 300. In some embodiments, the structured light system 001 may further include a detection sensor 500. In some embodiments, the structured light system 001 may further include a light intensity sensor ( Figure 5 (Not shown in the image).

[0062] The structured light projector 100 can be communicatively connected to the computing device 300 during operation. Specifically, the first laser 121 and the second laser 122 in the laser assembly 120 of the structured light projector 100 can be communicatively connected to the computing device 300. The computing device 300 can control the structured light projector 100 based on the communication connection, thereby controlling the activation of one of the first laser 121 and the second laser 122 to project a structured light pattern into a target field of view. The structured light pattern includes either the first structured light pattern or the second structured light pattern. The computing device 300 can control the activation of the first laser 121 to project the first structured light pattern into the target field of view. The computing device 300 can also control the activation of the second laser 122 to project the second structured light pattern into the target field of view. In some embodiments, the dielectric elastomer 148 in the structured light head design 100 can also be electrically connected to the computing device 300. The computing device 300 can control the focal length of the metasurface lens 140 by controlling the power applied to the dielectric elastomer 148 and the deformation of the lens substrate 144.

[0063] The image sensor 200 can communicate with the computing device 300 during operation. The image sensor 200 can convert captured optical images into image signals, utilizing the photoelectric conversion function of optoelectronic devices to convert the optical image on the image sensor 200 into an electrical signal proportional to the optical image. The image sensor 200 can be any type of camera or webcam. The image sensor 200 can acquire depth images of the target field of view. Specifically, when the target object is within the target field of view, the computing device 300 can control the structured light projector 100 to project the structured light pattern onto the target object, and control the image sensor 200 to acquire the image formed by the reflection of the structured light pattern by the target object. That is, the depth image includes the image formed by the structured light pattern projected onto the target object. The image sensor 200 can acquire either a first depth image formed by the first structured light pattern projected onto the target object, or a second depth image formed by the second structured light pattern projected onto the target object. The depth image may include either the first depth image or the second depth image.

[0064] It should be noted that when the first and second lasers are infrared light, the image sensor 200 can be a sensor capable of capturing infrared light, such as an infrared camera, etc.

[0065] When the computing device 300 is in operation, it can communicate with the structured light projector 100 and the image sensor 200 to control the structured light projector 100 to project the structured light pattern into the target field of view and receive the depth image acquired by the image sensor 200, and calculate the depth information of the target object based on the depth image. The depth information may include the vertical distance between different positions of the target object and the image sensor 200. The computing device 300 may store data or instructions for performing the depth calculation method described in this specification, and may execute or be used to execute the data and / or instructions. The communication connection refers to any form of connection that enables direct or indirect reception of information. In some embodiments, the computing device 300 may transmit data to the structured light projector 100 and the image sensor 200 through a wireless communication connection; in some embodiments, the computing device 300 may also transmit data to the structured light projector 100 and the image sensor 200 through a direct wire connection; in some embodiments, the computing device 300 may also establish an indirect connection with the structured light projector 100 and the image sensor 200 through a direct wire connection to other circuits, thereby realizing the transmission of data between them.

[0066] The computing device 300 may include hardware devices with data processing capabilities and the necessary programs to drive the hardware devices. Of course, the computing device 300 may also be solely a hardware device with data processing capabilities, or simply a program running on the hardware device. In some embodiments, the computing device 300 may include mobile devices, tablets, laptops, built-in devices in motor vehicles, or similar content, or any combination thereof. In some embodiments, the mobile device may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, the smart home device may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, the smart mobile device may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, the virtual reality device or augmented reality device may include virtual reality headsets, virtual reality glasses, virtual reality patches, augmented reality headsets, augmented reality glasses, augmented reality patches, or similar content, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, head-mounted displays, Gear VR, etc. In some embodiments, the built-in device in the motor vehicle may include an in-vehicle computer, an in-vehicle television, etc. In some embodiments, the computing device 300 may be a device with positioning technology for locating the position of the computing device 300.

[0067] Figure 6 A schematic diagram of a computing device 300 is shown. The computing device 300 can perform the depth calculation method described in this specification. The depth calculation method is described in other parts of this specification. Figure 6 As shown, the computing device 300 may include at least one storage medium 330 and at least one processor 320. In some embodiments, the computing device 300 may also include a communication port 350 and an internal communication bus 310.

[0068] The internal communication bus 310 can connect different system components, including storage medium 330, processor 320 and communication port 350.

[0069] The communication port 350 is used for data communication between the computing device 300 and the outside world. For example, the communication port 350 can be used for data communication between the computing device 300 and the structured light projector 100 and the image sensor 200. The communication port 350 can be a wired communication port or a wireless communication port.

[0070] Storage medium 330 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 332, a read-only storage medium (ROM) 334, or a random access storage medium (RAM) 336. Storage medium 330 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the deep learning methods provided in this specification.

[0071] At least one processor 320 can be communicatively connected to at least one storage medium 330 and a communication port 350 via an internal communication bus 310. The at least one processor 320 is used to execute the at least one instruction set described above. When the structured light system 001 is running, the at least one processor 320 reads the at least one instruction set and executes the depth computing method provided in this specification according to the instructions of the at least one instruction set. The processor 320 can execute all the steps included in the depth computing method. The processor 320 can be in the form of one or more processors. In some embodiments, the processor 320 may include one or more hardware processors, such as a microcontroller, microprocessor, reduced instruction set computer (RISC), application-specific integrated circuit (ASIC), application-specific instruction set processor (ASIP), central processing unit (CPU), graphics processing unit (GPU), physical processing unit (PPU), microcontroller unit, digital signal processor (DSP), field-programmable gate array (FPGA), advanced RISC machine (ARM), programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 320 is described in this specification for the computing device 300. However, it should be noted that the computing device 300 in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 320 of the computing device 300 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 320 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0072] In some embodiments, the structured light system 001 may further include a detection sensor 500. The detection sensor 500 can monitor the target field of view and generate monitoring data to detect whether a target object has entered the target field of view. The computing device 300 can be communicatively connected to the detection sensor 500 for data transmission. The computing device 300 can receive the monitoring data of the detection sensor 500 on the target field of view and can determine whether a target object has entered the field of view based on changes in the monitoring data. Specifically, when no target object has entered the target field of view, the monitoring data remains essentially unchanged or changes only slightly, less than a threshold; when a target object has entered the target field of view, the monitoring data changes, and the change is greater than the threshold. When the computing device 300 determines that a target object has entered the target field of view, the computing device 300 can control the structured light projector 100 to start and project the structured light pattern into the target field of view, and execute data or instructions of the depth calculation method described in this specification to calculate the depth information of the target object.

[0073] In some embodiments, the detection sensor 500 may include a distance sensor. In some embodiments, the image sensor 200 may be used as the detection sensor 500. The distance sensor includes at least one of an infrared sensor, a laser sensor, an ultrasonic sensor, and a radar sensor. The distance sensor can send a distance signal into the target field of view and receive distance signals reflected back by objects within the target field of view. When a target object enters the field of view, the distance signal received by the distance sensor changes. In some embodiments, the computing device 300 can calculate the distance range of the target object from the distance sensor based on the data detected by the distance sensor, and control the voltage on the dielectric elastomer 148 according to the distance range to control the focal length of the metasurface lens 140, so that the target object is within the depth of field allowed by the focal length of the metasurface lens 140. When the detection sensor 500 is the image sensor 200, the image sensor 200 can receive light signals within the target field of view. When a target object enters the target field of view, the illumination intensity data of the light signals received by the image sensor 200 changes.

[0074] In some embodiments, the structured light system 001 may further include a light intensity sensor ( Figure 5(Not shown in the image). The light intensity sensor can be used to monitor the light intensity of the target field of view. The computing device 300 can be communicatively connected to the light intensity sensor for data transmission. The computing device 300 can receive the light intensity data monitored by the light intensity sensor for the target field of view and can determine whether to use a first laser or a second laser based on the light intensity data. As mentioned above, the first wavelength has better resistance to strong light interference than the second wavelength. Under strong light, the computing device 300 can control the first laser 121 in the structured light projector 100 to turn on, so as to project the first structured light pattern into the target field of view to obtain a clearer depth image. Under non-strong light, the photoelectric conversion efficiency of the second wavelength is higher than that of the first wavelength, and the computing device 300 can control the second laser 122 in the structured light projector 100 to turn on, so as to project the second structured light pattern into the target field of view to reduce the overall power consumption of the structured light system 001.

[0075] In some embodiments, the image sensor 200 can be used as a light intensity sensor. When the light intensity sensor is the image sensor 200, the image sensor 200 can receive light signals within the target field of view and generate light intensity data based on the light signals.

[0076] Figure 7 A flowchart of a depth calculation method P100 provided according to an embodiment of this specification is shown. As previously described, a computing device 300 can execute the depth calculation method P100 provided in this specification. Specifically, a processor 320 in the computing device 300 can read an instruction set stored in its local storage medium and then execute the depth calculation method P100 provided in this specification according to the provisions of the instruction set. In some embodiments, the method P100 may include:

[0077] S120: Control the structured light pattern projected by the structured light projector 100 and receive the depth image.

[0078] In some embodiments, prior to step S120, the computing device 300 may further control the focal length of the metasurface lens 140 based on the distance between the target object detected by the detection sensor 500 and the detection sensor 500, so that the target object is within the depth of field allowed by the focal length of the metasurface lens 140. Therefore, the structured light system 001 and the structured light projector 100 can broaden their application scenarios by changing the focal length of the metasurface lens 140.

[0079] As previously stated, with the focal length of the metasurface lens 140 fixed, when the target object is within the depth of field allowed by the focal length of the metasurface lens 140, the greater the distance between the target object and the structured light projector 100, the fewer speckles will be covering the target object. As previously stated, the speckle density of the second structured light pattern projected by the second laser is greater than the speckle density of the first structured light pattern projected by the first laser. Therefore, with the focal length of the metasurface lens 140 fixed, the second laser is more suitable for acquiring depth information at long distances, while the first laser is more suitable for acquiring depth information at short distances.

[0080] Specifically, step S120 may include:

[0081] S122: Control the structured light projector 100 to turn on the first laser 121 and project the first structured light pattern into the target field of view.

[0082] In some embodiments, the computing device 300 may default to first activating the first laser 121 to project a first structured light pattern. In some embodiments, the computing device 300 may default to first activating the second laser 122 to project a second structured light pattern. For ease of demonstration, we will use the example of the computing device 300 defaulting to first activating the first laser 121 to project the first structured light pattern. When the computing device 300 determines, based on the detection data from the detection sensor 500, that a target object has entered the target field of view, the computing device 300 may control the first laser 121 of the structured light projector 100 to activate, thereby projecting the first structured light pattern onto the target object.

[0083] S124: Receive the first depth image acquired by the image sensor 200.

[0084] The computing device 300 can also control the image sensor 200 to acquire depth images within the target's field of view. The depth images may include the first depth image. The first depth image includes the image formed by the first structured light pattern projected onto the target object.

[0085] S126: Determine the proportion of missing depth information corresponding to the target object in the first depth image.

[0086] The computing device 300 can calculate the proportion of missing depth information for the target object in the first depth image based on the first depth image. In some embodiments, the proportion of missing depth information may include the ratio of the area of ​​the target object in the first depth image that does not contain the first structured light pattern to the total area of ​​the target object. As mentioned above, the first structured light pattern is a speckle pattern. When the speckle pattern illuminates the target object, part of the target object is covered by the speckle pattern, and part is not covered by the speckle pattern. The computing device 300 can calculate the ratio of the area of ​​the target object in the first depth image that is not covered by the speckle pattern to the total area of ​​the target object as the proportion of missing depth information. In some embodiments, the proportion of missing depth information may also be the ratio of the number of pixels in the first pixel point corresponding to the target object in the first depth image where depth information cannot be calculated to the total number of pixels corresponding to the target object. As mentioned above, depth information cannot be obtained from the locations of the target object not covered by the speckle pattern. The computing device 300 can use the ratio of the number of pixels in the first depth image where depth information of the target object cannot be obtained to all the first pixels corresponding to the target object as the proportion of missing depth information.

[0087] S128: Control the structured light projector 100 based on the missing depth information ratio.

[0088] The computing device 300 can determine whether the quality of the first depth image meets the usage requirements based on the proportion of missing depth information. Specifically, step S128 may include one of the following:

[0089] S128-2: If the proportion of missing depth information exceeds a preset threshold range or is within the threshold range, control the structured light projector 100 to turn on the second laser 122, project the second structured light pattern into the target field of view, and receive the second depth image acquired by the image sensor 200.

[0090] When the proportion of missing depth information exceeds a preset threshold range or falls within the threshold range, the accuracy of the depth information of the target object calculated using the first depth image is low. To improve the accuracy of the depth calculation of the target object, the computing device 300 can control the structured light projector 100 to activate the second laser 122 and project the second structured light pattern into the target's field of view. The computing device 300 can also control the image sensor 200 to acquire depth images within the target's field of view. The depth image may include the second depth image. The first depth image includes the image formed by the first structured light pattern projected onto the target object.

[0091] The threshold can be obtained based on machine learning or experimental statistics.

[0092] S128-4: Determine that the proportion of missing depth information in the first depth image is less than the threshold range, and control the structured light projector 100 to turn off.

[0093] When the proportion of missing depth information is less than a preset threshold, the accuracy of the depth information of the target object calculated using the first depth image is high. At this time, the computing device 300 can control the structured light projector 100 to turn off.

[0094] like Figure 7 As shown, the method P100 may further include:

[0095] S140: Calculate the target depth information of the target object based on the depth image.

[0096] Specifically, step S140 may include one of the following:

[0097] S142: Determine that the proportion of missing depth information in the first depth image is less than the threshold range, and determine the target depth information based on the first depth image.

[0098] When the proportion of missing depth information is less than a preset threshold range, the accuracy of the depth information of the target object calculated using the first depth image is high. In this case, the computing device 300 can calculate the depth information of the target object based on the first depth image. Specifically, the computing device 300 can determine the depth information of the target object based on the deformation of the first structured light pattern on the target object in the first depth image. For locations on the target object not covered by the first structured light pattern, the computing device 300 can obtain depth information through interpolation.

[0099] S144: Determine that the proportion of missing depth information exceeds the threshold range, and determine the target depth information based on the second depth image.

[0100] When the proportion of missing depth information exceeds a preset threshold, the accuracy of the depth information of the target object calculated using the first depth image is low. In this case, the computing device 300 can calculate the depth information of the target object based on the second depth image. Specifically, the computing device 300 can determine the depth information of the target object based on the deformation of the second structured light pattern on the target object in the second depth image. For locations on the target object not covered by the second structured light pattern, the computing device 300 can obtain the depth information through interpolation.

[0101] S146: Determine the proportion of missing depth information within the threshold range, and determine the target depth information based on the first depth image and the second depth image.

[0102] When the proportion of missing depth information is within a preset threshold range, the computing device 300 can fuse the first depth image and the second depth image to calculate the depth information of the target object. Specifically, step S146 may be: the computing device 300 determines the first depth information of the target object based on the first depth image; the computing device 300 determines the second depth information of the target object based on the second depth image; the computing device 300 performs a fusion process on the first depth information and the second depth information to determine the target depth information. The fusion process may be a weighted summation.

[0103] In some embodiments, method P100 may further consist of: a computing device 300 measuring the illumination intensity of the target field of view using the illumination intensity sensor; when the illumination intensity exceeds an illumination intensity threshold, the computing device 300 activating the first laser 121 to project the first structured light pattern onto the target object; then, the computing device 300 controlling the image sensor 200 to acquire a first depth image formed by the first structured light pattern projected onto the target object; the computing device 300 obtaining the first depth image from the image sensor 200 and calculating the depth information of the target object based on the deformation of the first structured light pattern in the first depth image; when the illumination intensity does not exceed the illumination intensity threshold, the computing device 300 activating the second laser 122 to project the second structured light pattern onto the target object; then, the computing device 300 controlling the image sensor 200 to acquire a second depth image formed by the second structured light pattern projected onto the target object; the computing device 300 obtaining the second depth image from the image sensor 200 and calculating the depth information of the target object based on the deformation of the second structured light pattern in the second depth image.

[0104] In summary, the structured light projector 100, structured light system 001, and depth calculation method P100 provided in this specification, by customizing the size and arrangement angle of the diffraction units 146-1 of the metasurface lens 140, enable the metasurface lens 140 to be simultaneously applicable to two or more different wavelengths of laser light, allowing the two wavelengths of laser light to project two different structured light patterns. This makes the structured light projector 100 applicable to a wider range of application scenarios. The structured light system 001 and depth calculation method P100 can select at least one from two depth images corresponding to two wavelengths based on the proportion of speckle on the target object's area in the application scenario, to calculate the depth information of the target object, thereby improving the accuracy of depth calculation.

[0105] This specification, in another aspect, provides a non-transitory storage medium storing at least one set of executable instructions for performing depth calculations. When the executable instructions are executed by a processor, they instruct the processor to implement the steps of the depth calculation method P100 described herein. In some possible embodiments, various aspects of this specification can also be implemented as a program product comprising program code. When the program product is run on a computing device 300, the program code causes the computing device 300 to perform the steps of the depth calculations described herein. The program product for implementing the above method may employ a portable compact disc read-only memory (CD-ROM) containing program code and may run on the computing device 300. However, the program product of this specification is not limited thereto. In this specification, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system (e.g., processor 320). The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit programs for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof. Program code for performing the operations described herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on computing device 300, partially on computing device 300, as a standalone software package, partially on computing device 300 and partially on a remote computing device, or entirely on a remote computing device.

[0106] This specification also provides an electronic device. The electronic device includes a housing and a structured light system 001 provided herein. The structured light system 001 can be mounted on the housing. In some embodiments, the electronic device may further include other modules, such as a display device, a human-computer interaction device, etc.

[0107] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0108] In summary, after reading this detailed disclosure, those skilled in the art will understand that the foregoing detailed disclosure is presented by way of example only and is not restrictive. Although not explicitly stated herein, those skilled in the art will understand that this specification requires various reasonable changes, improvements, and modifications to the embodiments. These changes, improvements, and modifications are intended to be made by this specification and are within the spirit and scope of the exemplary embodiments described herein.

[0109] Furthermore, certain terms in this specification have been used to describe embodiments of this specification. For example, "an embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of this specification. Therefore, it is to be emphasized and understood that two or more references to "an embodiment" or "an embodiment" or "alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. Moreover, specific features, structures, or characteristics may be suitably combined in one or more embodiments of this specification.

[0110] It should be understood that in the foregoing description of the embodiments in this specification, various features are combined in a single embodiment, drawing, or description for the purpose of simplifying the description and aiding in the understanding of a feature. However, this does not mean that the combination of these features is necessary, and those skilled in the art may readily identify some of the devices as separate embodiments when reading this specification. That is, the embodiments in this specification can also be understood as an integration of multiple secondary embodiments. It is also valid when each secondary embodiment contains fewer than all the features of a single foregoing disclosed embodiment.

[0111] Each patent, patent application, publication of the patent application, and other materials such as articles, books, specifications, publications, documents, articles, etc., cited herein may be incorporated by reference. The entire contents used for all purposes, except for any history of prosecution documents associated with it, that may be inconsistent with or conflict with this document, or that may have a limiting effect on the widest extent of the claims, are now or hereafter associated with this document. For example, in the event of any inconsistency or conflict between the description, definition, and / or use of terms associated with any of the included materials and the terms, description, definition, and / or used in connection with this document, the terms used herein shall prevail.

[0112] Finally, it should be understood that the embodiments disclosed herein are illustrative of the principles of the embodiments described in this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are merely examples and not limitations. Those skilled in the art can implement the applications described in this specification using alternative configurations based on the embodiments in this specification. Therefore, the embodiments in this specification are not limited to the embodiments precisely described in the applications.

Claims

1. A structured light projector, comprising: Laser components, including: A first laser emits a first laser beam of a first wavelength during operation, wherein the first laser beam is linearly polarized light vibrating along a first direction; and A second laser emits a second laser beam of a second wavelength during operation. This second laser beam is linearly polarized light vibrating along a second direction. The first wavelength is greater than the second wavelength, and the first direction forms a preset angle with the second direction, where the preset angle is greater than 0. Metasurface lenses, including an incident light side and an exit light side; The light-incident side is opposite to the laser component, and the light-outceasing side includes an anisotropic metasurface. The metasurface includes multiple diffraction units, wherein the size and rotation angle of each of the multiple diffraction units are determined based on the first phase distribution corresponding to the first wavelength, the second phase distribution corresponding to the second wavelength, the current position of the diffraction unit, and the preset angle, so that the multiple diffraction units simultaneously control the first laser and the second laser to form different structured light patterns. The first laser beam enters the metasurface lens from the incident light side and exits from the exit light side, forming a first structured light pattern; The second laser beam enters the metasurface lens from the incident light side and exits from the exit light side, forming a second structured light pattern. The speckle density of the first structured light pattern is less than that of the second structured light pattern.

2. The structured light projector as claimed in claim 1, wherein, The preset included angle is a right angle.

3. The structured light projector as claimed in claim 1, wherein, The metasurface lens also includes: A lens substrate includes an incident surface and an exit surface, the incident surface being located on the light-incident side and the exit surface being located on the light-exiting side, the metasurface being formed on the exit surface of the lens substrate; and The plurality of diffraction units are composed of subwavelength nanopillars with different aspect ratios arranged in a predetermined manner and at a predetermined distance.

4. The structured light projector as described in claim 3, wherein, The metasurface lens also includes: A dielectric elastomer is located on the incident surface of the lens substrate. When the dielectric elastomer is driven by a voltage, the deformation of the dielectric elastomer causes the lens substrate to deform, and the plurality of diffraction units are displaced. By controlling the magnitude of the voltage, the deformation of the lens substrate is controlled, thereby controlling the displacement of the plurality of diffraction units, so that the focal point of the thin-film zoom lens moves within a predetermined range.

5. The structured light projector as claimed in claim 1, wherein, Also includes: A deflection component, located between the laser assembly and the metasurface lens, deflects the first laser and the second laser, so that the first laser and the second laser pass through the deflection component and then enter the metasurface lens perpendicularly.

6. A structured light system, comprising: The structured light projector according to any one of claims 1-5 projects a structured light pattern onto a target field of view during operation, wherein the structured light pattern includes the first structured light pattern or the second structured light pattern; An image sensor, during operation, acquires a depth image of the target field of view, the depth image including an image formed by the structured light pattern projected onto the target object; as well as The computing device, during operation, communicates with the structured light projector and the image sensor, controls the structured light pattern projected by the structured light projector and receives the depth image, and calculates the target depth information of the target object based on the depth image.

7. The structured light system as described in claim 6, wherein, The process of controlling the structured light pattern projected by the structured light projector and receiving the depth image includes: The structured light projector is controlled to activate the first laser, projecting the first structured light pattern into the target field of view; and The image sensor acquires a first depth image, the first depth image including the image formed by the first structured light pattern projected onto the target object, and the depth image including the first depth image; Determine the proportion of missing depth information corresponding to the target object in the first depth image. The proportion of missing depth information includes the ratio of the area of ​​the target object in the first depth image that does not contain the first structured light pattern to the total area of ​​the target object. The structured light projector is controlled based on the proportion of missing depth information.

8. The structured light system of claim 7, wherein, The step of controlling the structured light projector based on the proportion of missing depth information includes: If the proportion of missing depth information exceeds a preset threshold range or falls within the threshold range, the structured light projector is controlled to activate the second laser, projecting the second structured light pattern into the target field of view, and receiving the second depth image acquired by the image sensor. The second depth image includes the image formed by the second structured light pattern projected onto the target object, and the depth image includes the second depth image; or If the proportion of missing depth information in the first depth image is determined to be less than the threshold range, the structured light projector is controlled to turn off.

9. The structured light system as described in claim 8, wherein, The calculation of the target depth information of the target object based on the depth image includes one of the following: The target depth information is determined based on the first depth image after determining that the proportion of missing depth information in the first depth image is less than the threshold range. If the proportion of missing depth information exceeds the threshold range, the target depth information is determined based on the second depth image; as well as The proportion of missing depth information is determined to be within the threshold range, and the target depth information is determined based on the first depth image and the second depth image.

10. The structured light system as claimed in claim 9, wherein, Determining the target depth information based on the first depth image and the second depth image includes: Determine the first depth information of the target object based on the first depth image; Determine the second depth information of the target object based on the second depth image; and The first depth information and the second depth information are fused to determine the target depth information.

11. A depth calculation method for the structured light system of claim 6, the depth calculation method comprising execution via the computing device: Controlling the structured light pattern projected by the structured light projector and receiving the depth image; and The target depth information of the target object is calculated based on the depth image.

12. The depth calculation method as described in claim 11, wherein, The process of controlling the structured light pattern projected by the structured light projector and receiving the depth image includes: The structured light projector is controlled to activate the first laser, projecting the first structured light pattern into the target field of view; and The image sensor acquires a first depth image, the first depth image including the image formed by the first structured light pattern projected onto the target object, and the depth image including the first depth image; Determine the proportion of missing depth information corresponding to the target object in the first depth image. The proportion of missing depth information includes the ratio of the area of ​​the target object in the first depth image that does not contain the first structured light pattern to the total area of ​​the target object. The structured light projector is controlled based on the proportion of missing depth information.

13. The depth calculation method as described in claim 12, wherein, The step of controlling the structured light projector based on the proportion of missing depth information includes: If the proportion of missing depth information exceeds a preset threshold range or falls within the threshold range, the structured light projector is controlled to activate the second laser, projecting the second structured light pattern into the target field of view, and receiving the second depth image acquired by the image sensor. The second depth image includes the image formed by the second structured light pattern projected onto the target object, and the depth image includes the second depth image; or If the proportion of missing depth information in the first depth image is determined to be less than the threshold range, the structured light projector is controlled to turn off.

14. The depth calculation method as described in claim 13, wherein, The calculation of the target depth information of the target object based on the depth image includes one of the following: The target depth information is determined based on the first depth image after determining that the proportion of missing depth information in the first depth image is less than the threshold range. If the proportion of missing depth information exceeds the threshold range, the target depth information is determined based on the second depth image; as well as The proportion of missing depth information is determined to be within the threshold range, and the target depth information is determined based on the first depth image and the second depth image.

15. The depth calculation method as described in claim 14, wherein, Determining the target depth information based on the first depth image and the second depth image includes: Determine the first depth information of the target object based on the first depth image; Determine the second depth information of the target object based on the second depth image; and The first depth information and the second depth information are fused to determine the target depth information.

Images