Structured light projection device and depth camera

Abstract

A structured light projection device (1000) and a depth camera (10), the structured light projection device (1000) comprising: a light emitting mechanism (1) for emitting first light; an angle adjusting mechanism (2) for adjusting the angle of the first light; and a diffraction mechanism (3) for converting the first light passing through the angle adjusting mechanism (2) into second light with a preset pattern. The field of view range of detection can be improved while ensuring the spatial density of the structured light preset pattern, thereby increasing the detection accuracy of depth information.

Description

Technical Field

[0001] This application relates to the field of intelligent vehicles, specifically to a structured light projection device, a depth camera, a control method for the structured light projection device, and a control device. Background Technology

[0002] In order to obtain information about objects in space and thus achieve functions such as facial recognition, gesture recognition, and action recognition, the depth information of objects becomes a key element in improving the ability to perceive objects in space.

[0003] Currently, depth information of objects is typically obtained through two methods: active and passive. Passive methods rely on the reflection of ambient light by the object to obtain its 3D information, such as binocular vision technology. Active methods involve emitting light signals at the object and receiving the signals returned by the object to obtain its 3D information, such as time-of-flight (ToF) technology and structured light technology.

[0004] Binocular vision technology uses imaging devices to acquire two images of a measured object from different positions. By calculating the positional deviation between corresponding points in the images, it obtains the object's three-dimensional information. However, this method is highly dependent on ambient light, and may not be able to accurately obtain depth information in complex lighting conditions such as a vehicle cabin.

[0005] Time-of-Flight (ToF) technology emits modulated continuous light pulses onto an object, then uses a sensor to receive the reflected light and calculates and converts the time difference between them to obtain depth information. However, this method suffers from insufficient point cloud density over large detection ranges and insufficient intensity of reflected light at longer detection distances, thus reducing detection accuracy.

[0006] Structured light technology projects an coded pattern onto a target object and then calculates the deformation of the coded pattern to determine depth information. However, this method can suffer from insufficient density of the coded pattern when the detection range is large, and a decrease in the signal-to-noise ratio of the coded pattern when the detection distance is long. Furthermore, when there are many objects in the space, the coded pattern cannot be projected onto the target object specifically, and other objects within the detection range can interfere with the depth information of the target object, which also leads to a decrease in detection accuracy.

[0007] Therefore, how to ensure a sufficiently large field of view for detection while maintaining the accuracy of depth information has become an urgent problem to be solved in the industry. Summary of the Invention

[0008] This application provides a structured light projection device, a depth camera, a control method for the structured light projection device, and a control device, which can increase the detection field of view and improve the detection accuracy of depth information.

[0009] The first aspect of this application provides a structured light projection device, comprising: a light emitting mechanism for emitting first light; an angle adjusting mechanism for adjusting the angle of the first light; and a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into second light with a preset pattern.

[0010] Adjusting the angle of the first beam refers to adjusting the angle of the first beam relative to its emission direction, so that the angle of the first beam deviates from its emission direction.

[0011] With the above settings, the angle of the first light is adjusted by the angle adjustment mechanism, thereby increasing the emission range of the first light and thus increasing the detection field of view of the light projection device. By adjusting the angle of the first light, the projection position of the first light can be adjusted, for example, according to the shape of the target area, so that the preset pattern can cover the target area and improve the accuracy of the depth information.

[0012] In one possible implementation, the device further includes a size adjustment mechanism for adjusting the size of the second light so that the second light covers the target area.

[0013] By adopting the above structure, the size of the preset pattern is adjusted by the size adjustment mechanism, so that the size of the preset pattern can be adjusted according to the size of the target area, and the density of the preset pattern in the target area is maintained within a certain range. This avoids the problem that some areas in the target area are not covered by the preset pattern due to the preset pattern being too sparse, thereby ensuring the detection accuracy of the structured light projection device.

[0014] In one possible implementation, the angle adjustment mechanism includes: a first adjustment module and a second adjustment module, the second adjustment module being located between the first adjustment module and the diffraction mechanism; the first adjustment module is used to adjust the first light within a first angle range; the second adjustment module is used to adjust the first light within a second angle range, the second angle range being greater than the first angle range.

[0015] With the above settings, the first adjustment module adjusts the angle of the first light within a small range (e.g., 0° to 5°) with high precision; the second adjustment module adjusts the angle of the first light within a large range (0° to 45°). The combination of the two achieves high precision and a large range adjustment of the emission angle of the first light.

[0016] In one possible implementation, the first adjustment module is also used to split the first light into multiple beams.

[0017] The above settings enable beam splitting of the emitted light, allowing the first beam to cover different target areas, avoiding interference caused by the first beam being emitted to other areas, and improving the signal-to-noise ratio.

[0018] In one possible implementation, the first adjustment module includes a liquid crystal phased array. In another possible implementation, the size adjustment mechanism includes a liquid crystal lens.

[0019] In one possible implementation, the structured light projection device is installed in one or more of the following locations within the vehicle's cabin: the center console, the location of the dome light switch, the side of the front seats facing the rear seats, and the center rearview mirror.

[0020] In one possible implementation, the structured light projection device is installed outside the vehicle's cabin, on a mobile phone, or in a smart home terminal.

[0021] In a second aspect, this application provides a depth camera, comprising: a camera and a structured light projection device provided in the first aspect of this application and its possible implementations.

[0022] A third aspect of this application provides a control method for a structured light projection device, comprising: acquiring an image of a target object to which a first structured light is projected; determining, based on the image, the image position of the target object in the image and the depth information corresponding to the image position; and determining, based on the image position and the depth information, the emission angle of a second structured light, such that the second structured light emitted by the structured light projection device covers the target area of ​​the target object corresponding to the image position. The structured light projection device includes: a light emitting mechanism for emitting a first light, an angle adjusting mechanism for adjusting the angle of the first light, and a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into structured light with a preset pattern.

[0023] The above settings enable control over the structured light projection area, allowing the projection direction of the structured light to be adjusted according to the location of the target area.

[0024] In one possible implementation, the first structured light is projected onto a pre-defined seating area within the vehicle cabin.

[0025] By implementing the above settings, the coverage area of ​​the first structured light within the cockpit can be reduced, interference caused by the first structured light being emitted into other areas of the cockpit can be avoided, and the signal-to-noise ratio can be improved.

[0026] In one possible implementation, before acquiring an image of the target object to which the first structured light is projected, the method further includes: acquiring the position of the target object within the vehicle cabin; determining the preset seat area where the target object is located based on the position; and emitting the first structured light into the preset seat area.

[0027] With the above settings, the first structured light can be emitted according to the preset seat area where the target object is located, so that the first structured light only covers the preset seat area, reducing the coverage range of the first structured light, avoiding interference caused by the first structured light being emitted to other areas in the cabin, and improving the signal-to-noise ratio.

[0028] In one possible implementation, the position of the target object within the vehicle cabin is obtained based on one or more of the following: seat pressure information, seat belt buckle information, and vital signs information.

[0029] In a fourth aspect, this application provides a control device for a structured light projection apparatus, comprising: an acquisition module for acquiring an image of a target object to which a first structured light is projected; a determination module for determining, based on the image, the image position of the target object in the image and the depth information corresponding to the image position; and a control module for determining, based on the image position and the depth information, the emission angle of a second structured light, such that the second structured light emitted by the structured light projection apparatus covers the target area of ​​the target object corresponding to the image position. The structured light projection apparatus includes: a light emitting mechanism for emitting a first light, an angle adjustment mechanism for adjusting the angle of the first light, and a diffraction mechanism for converting the first light passing through the angle adjustment mechanism into structured light with a preset pattern.

[0030] In one possible implementation, the first structured light is projected onto a pre-defined seating area within the vehicle cabin.

[0031] In one possible implementation, the acquisition module is further configured to acquire the position of the target object within the vehicle cabin; the determination module is further configured to determine the preset seat area where the target object is located based on the position; and the control module is further configured to emit a first structured light toward the preset seat area.

[0032] In one possible implementation, the position of the target object within the vehicle cabin is obtained based on one or more of the following: seat pressure information, seat belt buckle information, and vital signs information.

[0033] In a fifth aspect of this application, a structured light projection system is provided, comprising: a structured light projection device provided in the first aspect of this application, a camera, and a control device for the structured light projection device provided in the fourth aspect of this application, wherein the camera is used to acquire images.

[0034] In one possible implementation, the control device is connected to the structured light projection device via an onboard Ethernet interface, a controller area network interface, or a differential signal interface.

[0035] In one possible implementation, the camera is connected to the control device via an in-vehicle Ethernet interface, a controller area network interface, or a differential signal interface.

[0036] In a sixth aspect of this application, a computing device is provided, comprising: a processor and a memory, wherein the processor is coupled to the memory, and the memory is used to store programs or instructions, wherein when the programs or instructions are executed by the processor, the computing device causes the computing device to perform the control method of the structured light projection device provided in the third aspect of this application.

[0037] In a seventh aspect, this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a computer, cause the computer to perform the control method for the structured light projection device provided in the third aspect of this application.

[0038] In an eighth aspect, this application provides a computer program product including program instructions that, when executed by a computer, cause the computer to perform the control method for the structured light projection device provided in the third aspect of this application.

[0039] By setting the first adjustment module, the first light can be adjusted within a small angle range, achieving high-precision adjustment of the first light, thereby enabling the preset pattern to cover the target area. By setting the first adjustment module, the first light can also be split, so that when there are multiple target objects, the coded pattern is projected onto the location of the target objects, reducing the area of ​​the coded pattern projected onto other objects, thereby reducing the interference caused by other objects within the detection range to the target objects and improving the signal-to-noise ratio.

[0040] By setting a second adjustment module, the emission angle of the first light can be significantly reduced when the target is far from the structured light projection device, thus maintaining the spatial density of the encoded image; when the target is close to the structured light projection device, the emission range of the first light can be increased, achieving adjustment of the first light within a large angular range.

[0041] By setting up a size adjustment mechanism, the detection range of the structured light projection device can be increased while the density of the coding pattern can be adjusted, thus avoiding a decrease in detection accuracy due to an overly sparse coding pattern. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of an application scenario of the structured light projection device and depth camera provided in the embodiments of this application;

[0043] Figure 2 This is an illustration of another application scenario of the structured light projection device and depth camera provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram of the structured light projection device provided in the embodiments of this application;

[0045] Figures 4-5This is a schematic diagram of the structure of the first adjustment module of the structured light projection device provided in the embodiments of this application;

[0046] Figures 6A-6B This is a schematic diagram of the structure of the second adjustment module of the structured light projection device provided in the embodiments of this application;

[0047] Figures 7-8 This is a schematic diagram of the size adjustment mechanism of the structured light projection device provided in the embodiments of this application;

[0048] Figure 9 This is a schematic diagram of the structure of a depth camera with a structured light projection device provided in an embodiment of this application;

[0049] Figure 10 This is a flowchart of the control method for the structured light projection device provided in the embodiments of this application;

[0050] Figure 11 This is a schematic diagram of the control device of the structured light projection device provided in the embodiments of this application;

[0051] Figure 12 This is a schematic diagram of the structured light projection system provided in the embodiments of this application;

[0052] Figure 13 This is a schematic diagram of the modules of the computing device provided in the embodiments of this application.

[0053] It should be understood that the dimensions and shapes of the block diagrams in the above structural diagrams are for reference only and should not constitute an exclusive interpretation of the embodiments of this application. The relative positions and inclusion relationships between the block diagrams presented in the structural diagrams are only schematic representations of the structural relationships between the block diagrams, and are not intended to limit the physical connection methods of the embodiments of this application. Detailed Implementation

[0054] The technical solutions provided in this application will be further described below with reference to the accompanying drawings and embodiments. It should be understood that the system architecture and business scenarios provided in the embodiments of this application are mainly for illustrating possible implementations of the technical solutions of this application and should not be construed as the sole limitation on the technical solutions of this application. Those skilled in the art will recognize that the technical solutions provided in this application are equally applicable to similar technical problems as system architectures evolve and new business scenarios emerge.

[0055] Figure 1 This is a schematic diagram illustrating an application scenario of the structured light projection device and depth camera provided in the embodiments of this application. For example... Figure 1As shown, the depth camera 10 provided in this application embodiment can be applied, for example, in a vehicle cockpit system to recognize the face of the user 12 inside the cockpit 11, thereby determining the identity of the user 12 or whether the driver is fatigued; or to recognize the gestures of the user 12 inside the cockpit 11, thereby determining the user 12's intentions and commands; or to determine the user 12's line of sight, etc. The structured light projection device can be installed in one or more of the following locations: the center console, the location of the dome light switch, the side of the front seat facing the rear seat, and the center rearview mirror. Figure 1 In the example shown, the depth camera 10 can be arranged on the center console 13. The depth camera 10 includes a structured light projection device and a camera. The structured light projection device can emit structured light L1 and L2 with preset patterns (also referred to as coded patterns in this application) to the area where the head of the occupant 12 is located in the cockpit 11. The camera acquires an image of the user 12's head with the projected structured light. The controller obtains the deformation of the structured light L1 and L2 at the position of the user's head based on the image of the user's head 12, and thus obtains the depth information of the user 12's head.

[0056] Figure 2 This is a schematic diagram illustrating another application scenario of the structured light projection device and depth camera provided in the embodiments of this application. For example... Figure 2 As shown, the depth camera 10 provided in this embodiment can also be applied in a smart home terminal system, for example, in a smart door lock system 14. The structured light projection device can emit structured light L3 with a preset pattern to the area where the user 15's face is located. The camera acquires a facial image of the user 15 with the projected structured light. The smart door lock system identifies the user 15's face based on the deformation of the encoded pattern at the user 15's facial position, thereby determining the user 15's identity.

[0057] Figure 3 This is a schematic diagram of a structured light projection device provided in one embodiment of this application, as shown below. Figure 3 As shown, an embodiment of this application provides a structured light projection device 1000, which includes: a light emitting mechanism 1 for emitting a first light, an angle adjusting mechanism 2 located on the light output path of the light emitting mechanism 1 and used to adjust the angle of the first light, a diffraction mechanism 3 located on the light output path of the angle adjusting mechanism 2 and used to convert the light passing through the angle adjusting mechanism 2 into a second light with a preset pattern, and a size adjusting mechanism 4 located on the light output path of the diffraction mechanism 3 and used to adjust the size of the preset pattern.

[0058] Adjusting the angle of the first beam refers to adjusting the angle of the first beam relative to its emission direction, causing the angle of the first beam to deviate from its emission direction. For example, the angle of the first beam may be deviated by 35° relative to its emission direction.

[0059] like Figure 3 As shown, the angle adjustment mechanism 2 may include a first adjustment module 21 and a second adjustment module 22. The first adjustment module 21 is used to adjust the light within a first angle range, and the second adjustment module 22 is located between the first adjustment module 21 and the diffraction mechanism 3, and is used to adjust the light within a second angle range, which is greater than the first angle range. The first angle range may be, for example, 0° to 5°, and the second angle range may be, for example, 0° to 45°.

[0060] In some embodiments, such as Figure 4 and Figure 5 As shown, the first adjustment module 21 can be a liquid crystal phased array, which may include: a first substrate 211, a first electrode 212, a first alignment unit 213, a liquid crystal layer 214, a second alignment unit 215, a second electrode 216, and a second substrate 217, which are sequentially bonded and parallel to each other. The first alignment unit 213 and the second alignment unit 215 are used to align the liquid crystal molecules in the liquid crystal layer 214 at a certain preset angle. A potential difference can be formed between the first electrode 212 and the second electrode 216, such as... Figure 4 As shown, the cross-section of the liquid crystal molecules in the liquid crystal layer 214 can be, for example, elliptical. When the potential difference is zero, the major axis of the liquid crystal molecule cross-section can be parallel to the plane where the first substrate 211 is located. Alternatively, the major axis of the liquid crystal molecule cross-section can form a preset angle relative to the first substrate 211 according to the shapes of the first alignment unit 213 and the second alignment unit 215. When there is a potential difference between the first electrode 212 and the second electrode 216, such as... Figure 5 As shown, the liquid crystal molecules in the liquid crystal layer 214 will deflect. The alignment direction of the liquid crystal molecules affects the refractive index of the liquid crystal layer 214, causing a phase change in the light emitted by the light emitting mechanism 1 when it exits the liquid crystal phased array. When this phase distribution changes linearly in space, it will cause the beam angle to deflect. In some embodiments, the second electrode 216 may include a plurality of electrode units 2161, on which a voltage is applied, so that each electrode unit 2161 forms an independent electromotive force difference with the first electrode 212, thereby changing the alignment direction of the liquid crystal molecules, so that the beam can be projected onto the target object at different angles.

[0061] In some embodiments, the first adjustment module may further be a MEMS (Micro-Electro-Mechanical System) galvanometer. The MEMS galvanometer may include, for example, a MEMS lens and a coil for driving the MEMS lens. The coil is disposed around the lens. By energizing the coil, an electromagnetic force is generated around the MEMS lens, causing the MEMS lens to twist and deflecting the angle of the first light illuminating the MEMS lens. Of course, the MEMS galvanometer may also be an electrostatically driven MEMS galvanometer, a piezoelectrically driven MEMS galvanometer, or a thermoelectrically driven MEMS galvanometer; this application does not limit this.

[0062] like Figure 6A As shown, the second adjustment module 22 can be a birefringent prism. The birefringent prism can include multiple birefringent prism units 221. Figure 6A Three are shown, but this application is not limited to these. Multiple birefringent prism units 221 are arranged in a straight line along the light-emitting direction of the first adjustment module 21. Each birefringent prism unit 221 may include two birefringent crystals 2211 and 2212. The birefringent crystals may be triangular prisms, having an incident surface and an exit surface. The cross-section perpendicular to the incident and exit surfaces may be triangular or trapezoidal. Figure 6A In the example shown, the cross-section is triangular. The optical axes of the two birefringent crystals 2211 and 2212 are perpendicular to each other. By designing the wedge angle α of the birefringent crystals 2211 and 2212 of the birefringent prism, the angle of the emitted beam can be controlled, thus achieving angle adjustment.

[0063] In some embodiments, the second adjustment module may also be a microlens array, such as... Figure 6B As shown, the microlens array 222 may include lens units 2222 arranged in an array. The light beam incident on the microlens array 222 is refracted by the lens units 2222, thereby deflecting the angle of the outgoing light beam and adjusting the angle of the light beam.

[0064] By setting the first adjustment module, for example, the beam deflection can be adjusted within the range of 0° to 5°, achieving small-angle, high-precision adjustment of the beam angle; by setting the second adjustment module, for example, the beam deflection can be adjusted within the range of 0° to 45°, achieving large-angle, wide-range adjustment of the beam angle, thereby enabling the coverage area of ​​the preset pattern to be adjusted according to the position and shape of the target object.

[0065] In some embodiments, the deflection of liquid crystal molecules is achieved by adjusting the voltage of the motor unit of the first adjustment module, and the first light emitted from the first adjustment module can achieve a deflection of, for example, 3°. After passing through three birefringent prism units, the 3° deflected first light can be deflected sequentially by 15°, 30° and 45° according to the different wedge angles α of the birefringent crystal, thereby achieving a 45° deflection of the beam.

[0066] like Figure 7 and Figure 8 As shown, the size adjustment mechanism can be composed of a liquid crystal lens 40, which can be cylindrical and may include: a first substrate 401, a first electrode 402, a liquid crystal layer 403, a second electrode 404, and a second substrate 405 sequentially bonded together. The second electrode 404 includes multiple electrode units 4041, and a voltage is applied to each electrode unit 4041, so that each electrode unit 4041 forms an independent potential difference with the first electrode 402, thereby adjusting the focal length of the liquid crystal lens 40 as needed. By applying different voltages to different electrode units, the liquid crystal layer 403 forms a phase distribution similar to that of a lens, varying from the center to the periphery; this change in phase distribution can adjust the divergence angle of the projected light, thereby expanding or shrinking the coverage area of ​​the preset pattern. For example, as... Figure 8 As shown, by applying a voltage to the electrode unit 4041 near the center of the second electrode 404, the liquid crystal exhibits different distributions, thereby expanding the coverage area of ​​the preset pattern; as Figure 7 As shown, by applying a voltage to the electrode unit 4041 in the second electrode 404 that is far from the center, the liquid crystal is made to present different distributions, thereby reducing the coverage area of ​​the preset pattern.

[0067] It should be noted that, although Figure 7 and Figure 8 The electrode units shown are arranged in a ring shape, but this application is not limited to this. The electrode units can also be arranged on the second substrate in other ways, as long as the distribution of the voltage applied to the second electrode is concentric.

[0068] By setting up a liquid crystal lens, the magnification of the preset pattern projection can be flexibly adjusted. When the target object is far away, the coverage area of ​​the preset pattern can be reduced by changing the focal length, while maintaining the spatial density of the encoded image; when the target object is close, the coverage area of ​​the preset pattern can be enlarged by changing the focal length.

[0069] By combining the first adjustment module, the second adjustment module, and the liquid crystal lens, the beam emission angle and the size of the preset pattern can be adjusted so that the preset pattern covers the target object as closely as possible to reduce interference from other objects besides the target object and improve the signal-to-noise ratio.

[0070] In some embodiments, the light emitting mechanism 1 may be composed of a vertical cavity surface emitting laser (VCSEL) for emitting infrared laser, for example, emitting a near-infrared laser beam with a wavelength of 940 / 850nm. After beam expansion and collimation, the laser beam is incident perpendicularly onto the light incident surface (the extended plane of the first substrate) of the first adjustment module 21.

[0071] In some embodiments, the diffraction mechanism 3 may be composed of diffractive optical elements (DOEs), such as multi-line DOE modules, one-dimensional dot matrix and linear DOE modules, two-dimensional dot matrix DOE modules, circular and circular dot matrix DOE modules, crosshair DOE modules, random dot matrix DOE modules, etc., and this application does not limit it.

[0072] By setting an angle adjustment mechanism, the emission range of the first light is increased, thereby increasing the detection field of view of the light projection device; by adjusting the angle of the first light, the projection position of the first light can be adjusted according to the shape of the target area, so that the preset pattern can cover the target area and improve the accuracy of the depth information.

[0073] By setting a size adjustment mechanism, the size of the preset pattern can be adjusted according to the size of the target area, so that the density of the preset pattern in the target area is maintained within a certain range. This avoids the problem that some areas in the target area are not covered by the preset pattern due to the preset pattern being too sparse, thereby ensuring the detection accuracy of the structured light projection device.

[0074] Figure 9 A schematic diagram of the structure of a depth camera provided in an embodiment of this application is shown below. Figure 9 As shown, the depth camera provided in this application embodiment may include: a substrate 5000, a control device 6000, a structured light projection device 1000 and a camera 2000 disposed on the substrate 5000.

[0075] The control device 6000 may include a printed circuit board 4000 (PCB) and a drive unit 3000. The drive unit 3000 is used to drive the angle adjustment mechanism and the diffraction mechanism, thereby adjusting the emission range of the structured light L4.

[0076] In some embodiments, the camera 2000 can be a high-resolution wide-angle infrared camera with a focal length between 13mm and 38mm and a field of view between 60° and 118°.

[0077] Below, refer to Figure 10This application describes a control method for a structured light projection device provided in an embodiment. The control method for the structured light projection device in this embodiment can be executed by a terminal, such as a terminal in a smart driving vehicle or a depth camera, or by an electronic device applied within the terminal, such as a system-on-a-chip or a general-purpose chip.

[0078] like Figure 10 As shown, a control method for a structured light projection device provided in this application embodiment may include the following steps:

[0079] Step S1: Acquire an image of the target object to which the first structured light is projected.

[0080] The target object can be a human face or hand, etc., and the image with the preset pattern of the first structured light can be acquired by a camera.

[0081] In some embodiments, the coverage of the first structured light can be substantially consistent with the field of view of the camera. In this way, the structured light projection device can roughly detect all objects within the entire field of view of the camera and obtain image and depth information of all target objects within the entire field of view.

[0082] In some embodiments, the first structured light can be projected onto a predetermined seating area within the vehicle cabin, such as... Figure 1 As shown, the preset seating areas may include, for example, the driver's seat area A1, the front passenger seat area A2, the rear first seat area A3, the second seat area A4, and the third seat area A5. These areas A1, A2, A3, A4, and A5 are preset according to the location of the seats within the vehicle's cabin.

[0083] In some embodiments, before step S1, the method may further include: obtaining the position of the target object within the vehicle cabin; determining a preset seat area where the target object is located based on the position; and emitting the first structured light to the preset seat area.

[0084] The position of the target object within the vehicle cabin is obtained based on one or more of the following: seat pressure information, seat belt buckle information, and vital signs information.

[0085] In some embodiments, seat pressure information can be obtained through a pressure sensor installed on the seat cushion or a pressure sensor installed on the seat back. A corresponding preset seat area can be set according to the location of the pressure sensor. For example, when the pressure detected by the pressure sensor installed on the driver's seat exceeds a threshold, it is determined that the target object is located in the driver's seat area A1, and a first structured light is emitted to the driver's seat area A1. Seatbelt buckle information can be obtained through a controller area network (CLAN) signal. A corresponding preset seat area can be set according to the CLAN signal. When the CLAN signal indicates that the passenger seatbelt is buckled, it is determined that the target object is located in the passenger seat area A2, and a first structured light is emitted to the passenger seat area A2. Vital signs information can be obtained through a radar sensor. The preset seat area where the location of the living person detected by the radar sensor is located can be determined. For example, when the detected living person is located in the first rear seat, the first rear seat area A3 is determined to be the area where the target object is located, and a first structured light is emitted to the first rear seat area A3.

[0086] Step S2: Determine the image position of the target object in the image and the depth information corresponding to the image position based on the image.

[0087] Among these methods, neural network models can be used to identify images and pinpoint the location of target objects within them. For example, facial recognition can be used to identify the pixel coordinates of a face in an image and to determine the depth information corresponding to those pixel coordinates based on the deformation of a preset pattern.

[0088] Step S3: Determine the emission angle of the second structured light based on the image position and the depth information, so that the second structured light emitted by the structured light projection device covers the target area of ​​the target object corresponding to the image position.

[0089] The coverage area of ​​the second structured light is smaller than that of the first structured light. The structured light projection device includes: a light emitting mechanism for emitting first light, an angle adjusting mechanism for adjusting the angle of the first light, and a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into structured light with a preset pattern. For a detailed description of the structure of the structured light projection device, please refer to the description of the structured light projection device provided in the above embodiments of this application. For the sake of brevity, it will not be repeated here.

[0090] In some embodiments, the voltage of the electrode unit corresponding to the pixel coordinates of the face region in the image can be determined based on the pixel coordinates and the depth information corresponding to the pixel coordinates. By applying the corresponding voltage to the electrode unit, the emission angle of the second structured light can be controlled so that the second structured light can cover the face region. Figure 11This application provides a schematic diagram of the control device of a structured light projection apparatus according to an embodiment of the present application. Figure 11 As shown in the embodiment of this application, the control device for a structured light projection device includes the following modules:

[0091] The acquisition module 100 is used to acquire an image of the target object with a first structured light.

[0092] The first structured light is projected onto a pre-defined seat area inside the vehicle cabin.

[0093] The determining module 200 is used to determine the image position of the target object in the image and the depth information corresponding to the image position based on the image.

[0094] The control module 300 is used to determine the emission angle of the second structured light according to the image position and the depth information, so that the second structured light emitted by the structured light projection device covers the target area of ​​the target object corresponding to the image position.

[0095] The structured light projection device includes: a light emitting mechanism for emitting first light, an angle adjusting mechanism for adjusting the angle of the first light, and a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into structured light with a preset pattern.

[0096] In some embodiments, the acquisition module 100 is further configured to acquire the position of the target object within the vehicle cabin; the determination module 200 is further configured to determine the preset seat area where the target object is located based on the position; and the control module 300 is further configured to emit the first structured light toward the preset seat area.

[0097] In some embodiments, the position of the target object within the vehicle cabin is obtained based on one or more of the following: seat pressure information, seat belt buckle information, and vital signs information.

[0098] For a detailed description of the acquisition module 100, the determination module 200, and the control module 300, please refer to the relevant description of the control method of a structured light projection device provided in the embodiments of this application. For the sake of brevity, it will not be repeated here.

[0099] It should be noted that the above modules, namely the acquisition module 100, the determination module 200, and the control module 300, are used to execute the relevant steps of the above method. For example, the acquisition module 100 is used to execute the relevant content of step S1, and the determination module 200 is used to execute the relevant content of step S2, etc.

[0100] In this embodiment, the structured light projection device is presented in the form of modules. Here, "module" may include an application-specific integrated circuit (ASIC), a processor and memory executing one or more software or firmware programs, integrated logic circuits, and / or other devices that can provide the above-mentioned functions. Furthermore, the acquisition module 100, determination module 200, and control module 300 described above can be... Figure 13 The computing device shown is used to implement this.

[0101] Figure 12 This is a schematic diagram of a structured light projection system provided in an embodiment of this application, such as... Figure 12 As shown, the structured light projection system provided in this application embodiment includes: a structured light projection device 500, a camera 600, and a control device 700 for the structured light projection device.

[0102] The camera 700 is used to capture images of the target object onto which the first structured light is projected.

[0103] In some embodiments, the structured light projection device and the camera can both be located at the roof light switch position, and the control device can be located at the center console position. The control device can be connected to the structured light projection device via wired or wireless means. The control device can also be connected to the camera via wired or wireless means. The control device executes steps S1-S3 according to the target object of the first structured light being projected obtained by the camera, thereby realizing the control of the structured light projection device.

[0104] Among them, wired connections can be achieved through vehicle Ethernet interfaces, controller area network interfaces, or differential signal interfaces to connect the control device with the structured light projection device and the camera with the control device.

[0105] Wireless methods, such as Bluetooth or Wi-Fi, can be used to connect the control device to the structured light projection device and the camera to the control device.

[0106] Figure 13 This is a schematic structural diagram of a computing device 1500 provided in an embodiment of this application. The computing device 1500 includes a processor 1510 and a memory 1520.

[0107] The processor 1510 can be connected to the memory 1520. The memory 1520 can be used to store the program code and data. Therefore, the memory 1520 can be a storage unit inside the processor 1510, an external storage unit independent of the processor 1510, or a component that includes both the storage unit inside the processor 1510 and the external storage unit independent of the processor 1510.

[0108] Optionally, the computing device 1500 may also include a bus. The memory 1520 and communication interface can be connected to the processor 1510 via the bus. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0109] It should be understood that in the embodiments of this application, the processor 1510 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), ASICs, field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 1510 may employ one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0110] The memory 1520 may include read-only memory and random access memory, and provides instructions and data to the processor 1510. A portion of the processor 1510 may also include non-volatile random access memory. For example, the processor 1510 may also store device type information.

[0111] When the computing device 1500 is running, the processor 1510 executes the computer execution instructions in the memory 1520 to perform the operation steps of the above method.

[0112] It should be understood that the computing device 1500 according to the embodiments of this application can correspond to the corresponding subject in executing the methods according to the various embodiments of this application, and the above and other operations and / or functions of each module in the computing device 1500 are respectively for implementing the corresponding processes of the methods of this embodiment. For the sake of brevity, they will not be described in detail here.

[0113] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0118] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the computer software product stored in a storage medium includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs a control method for a structured light emitting device, the method including at least one of the schemes described in the above embodiments.

[0120] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0122] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, radio frequency circuitry, etc., or any suitable combination thereof.

[0123] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0124] The terms "first," "second," and similar expressions used in the specification and claims are used only to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that, where permissible, a specific order or sequence may be interchanged so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0125] The term "comprising" as used in the specification and claims should not be construed as limiting itself to what follows; it does not exclude other elements or steps. Therefore, it should be interpreted as specifying the presence of the mentioned feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components, or groups thereof. Thus, the statement "device comprising means A and B" should not be limited to a device consisting solely of components A and B.

[0126] The terms "an embodiment" or "an embodiment" as used in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in at least one embodiment of this application. Therefore, the terms "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. Furthermore, in one or more embodiments, the particular features, structures, or characteristics can be combined in any suitable manner, as will be apparent to those skilled in the art from this disclosure.

[0127] Note that the above are merely embodiments of this application and the technical principles employed. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, all of which fall within the scope of protection of this application.

Claims

1. A structured light projection device, characterized in that, include: A light-emitting mechanism, used to emit the first light; An angle adjustment mechanism is used to adjust the angle of the first light so as to project the light output by the structured light projection device onto a preset seat area. The preset seat area is determined according to the position of the target object in the vehicle cabin. The angle adjustment mechanism includes: a first adjustment module and a second adjustment module. The second adjustment module is located between the first adjustment module and the diffraction mechanism. The first adjustment module is used to adjust the first light within a first angle range and output at least one light beam with preliminary adjustment. The second adjustment module is used to adjust the at least one light beam output by the first adjustment module within a second angle range. The second angle range is greater than the first angle range. The diffraction mechanism is used to convert the first light passing through the angle adjustment mechanism into a second light with a preset pattern; as well as A size adjustment mechanism is used to adjust the size of the preset pattern of the second light according to the position and size of the target object, so that the preset pattern of the second light covers the target object.

2. The apparatus according to claim 1, characterized in that, The first adjustment module includes: a liquid crystal phased array.

3. The apparatus according to claim 1, characterized in that, The size adjustment mechanism includes a liquid crystal lens.

4. The apparatus according to any one of claims 1-3, characterized in that, The structured light projection device is installed in one or more of the following locations within the vehicle's cabin: The location of the center console, the position of the dome light switch, the side of the front seats facing the rear seats, and the center rearview mirror.

5. The apparatus according to any one of claims 1-3, characterized in that, The structured light projection device is installed on a mobile phone or smart home terminal.

6. A depth camera, characterized in that, include: The camera and the structured light projection device according to any one of claims 1-5.

7. A control method for a structured light projection device, characterized in that, include: Obtain the location of the target object within the vehicle's cabin; The preset seating area where the target object is located is determined based on the location; Projecting a first structured light onto the preset seating area; Acquire an image of the target object onto which the first structured light is projected; Determine the image position of the target object in the image and the depth information corresponding to the image position based on the image; Based on the image position and the depth information, the emission angle of the second structured light is determined, so that the second structured light emitted by the structured light projection device covers the target area of ​​the target object corresponding to the image position. The structured light projection device includes: a light emitting mechanism for emitting first light, an angle adjusting mechanism for adjusting the angle of the first light, a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into structured light with a preset pattern, and a size adjusting mechanism for adjusting the size of the preset pattern according to the position and size of the target object. The angle adjustment mechanism is used to project the light output by the structured light projection device onto the preset seat area. The angle adjustment mechanism includes: a first adjustment module and a second adjustment module. The second adjustment module is located between the first adjustment module and the diffraction mechanism. The first adjustment module is used to adjust the first light within a first angle range and output at least one light beam with preliminary adjustment. The second adjustment module is used to adjust the at least one light beam output by the first adjustment module within a second angle range. The second angle range is greater than the first angle range.

8. The control method according to claim 7, characterized in that, The position of the target object within the vehicle cabin is obtained according to one or more of the following: Seat pressure information, seat belt buckle information, and vital signs information.

9. A control device for a structured light projection device, characterized in that, include: The acquisition module is used to obtain the location of the target object inside the vehicle's cabin. The determining module is used to: determine the preset seating area where the target object is located based on the position; The control module is used to: project a first structured light onto the preset seating area; The acquisition module is also used to acquire an image of the target object to which the first structured light is projected; The determining module is further configured to determine the image position of the target object in the image and the depth information corresponding to the image position based on the image; The control module is used to determine the emission angle of the second structured light based on the image position and the depth information, so that the second structured light emitted by the structured light projection device covers the target area of ​​the target object corresponding to the image position. The structured light projection device includes: a light emitting mechanism for emitting first light, an angle adjusting mechanism for adjusting the angle of the first light, a diffraction mechanism for converting the first light passing through the angle adjusting mechanism into structured light with a preset pattern, and a size adjusting mechanism for adjusting the size of the preset pattern according to the position and size of the target object. The angle adjustment mechanism is used to project the light output by the structured light projection device onto the preset seat area. The angle adjustment mechanism includes: a first adjustment module and a second adjustment module. The second adjustment module is located between the first adjustment module and the diffraction mechanism. The first adjustment module is used to adjust the first light within a first angle range and output at least one light beam with preliminary adjustment. The second adjustment module is used to adjust the at least one light beam output by the first adjustment module within a second angle range. The second angle range is greater than the first angle range.

10. The control device according to claim 9, characterized in that, The position of the target object within the vehicle cabin is obtained according to one or more of the following: Seat pressure information, seat belt buckle information, and vital signs information.

11. A structured light projection system, characterized in that, include: The camera, the structured light projection device according to any one of claims 1-5, and the control device for the structured light projection device according to claim 9 or 10, wherein the camera is used to acquire images.

12. The structured light projection system according to claim 11, characterized in that, The control device is connected to the structured light projection device via one of the following: an onboard Ethernet interface, a controller area network interface, or a differential signal interface.

13. The structured light projection system according to claim 11, characterized in that, The camera is connected to the structured light projection device via one of the following: an in-vehicle Ethernet interface, a controller area network interface, or a differential signal interface.

Images