Methods and equipment for blending depth detection using adaptive projectors

Abstract

This invention provides a method and apparatus for performing hybrid depth detection using an adaptive projector. The method includes: obtaining distance information using an image processing circuit; determining a distance range using the image processing circuit based on the distance information; selecting a projection type using the image processing circuit to determine at least one selected projection type from a plurality of predetermined projection types corresponding to the distance range; projecting the at least one selected projection type using an adaptive projector to extract at least one corresponding image using a camera, and performing depth detection using the image processing circuit based on the corresponding image to generate a depth map; and selectively outputting the depth map as a result depth map or combining depth data to generate a combined depth map as a result depth map using the image processing circuit.

Description

Technical Field

[0001] This invention relates to a detection mechanism for electronic devices such as mobile devices, and more particularly to a method for hybrid depth detection using an adaptive projector, and related equipment such as a hybrid depth detection device, an image processing circuit in the hybrid depth detection device, a depth processor in the image processing circuit, and the adaptive projector in the hybrid depth detection device. Background Technology

[0002] Based on relevant technologies, facial recognition technology has been applied to mobile devices, and certain facial recognition methods for these devices have been proposed. However, some problems may arise. For example, security issues are introduced when one or more of these mobile devices fail to perform facial recognition correctly. Further improvements to facial recognition algorithms may encounter bottlenecks. Therefore, a novel method and related architecture are needed to enhance the overall performance of electronic devices without introducing side effects or with the likelihood of introducing side effects. Summary of the Invention

[0003] Therefore, one object of the present invention is to provide a method for performing mixed depth detection using an adaptive projector, and to provide related devices such as a mixed depth detection device, an image processing circuit in the mixed depth detection device, a depth processor in the image processing circuit, and the adaptive projector in the mixed depth detection device, to solve the above-mentioned problems.

[0004] At least one embodiment of the present invention provides a method for performing hybrid depth detection using an adaptive projector, wherein the method may include: obtaining distance information using an image processing circuit within a hybrid depth detection device, wherein the distance information indicates a distance between the hybrid depth detection device and one or more target objects; determining a distance range using the image processing circuit based on the distance information, wherein the distance range is selected from a plurality of predetermined distance ranges for the distance; selecting a projection type using the image processing circuit to determine at least one selected projection type corresponding to the distance range from a plurality of predetermined projection types, wherein the at least one selected projection type is selected from the plurality of predetermined projection types; projecting the at least one selected projection type using the adaptive projector within the hybrid depth detection device to extract at least one corresponding image using a camera; and performing at least one type of depth detection corresponding to the at least one selected projection type from multiple types of depth detection using the image processing circuit based on the at least one corresponding image. The image processing circuit selectively outputs the at least one depth map as a resultant depth map or combines depth data based on the at least one depth map to generate a combined depth map as the resultant depth map, in response to the at least one selected projection type.

[0005] At least one embodiment of the present invention provides a related device that operates according to the above method. Examples of the above device may include, but are not limited to: a hybrid depth detection device, an electronic product (e.g., an electronic device such as a mobile device) having the hybrid depth detection device, the image processing circuit in the hybrid depth detection device, a depth processor in the image processing circuit, the adaptive projector in the hybrid depth detection device, etc.

[0006] At least one embodiment of the present invention provides an apparatus for performing hybrid depth detection, wherein the apparatus may include an image processing circuit, and further include a camera and an adaptive projector coupled to the image processing circuit. For example, the image processing circuit may be used to obtain distance information, determine a distance range based on the distance information, and perform projection type selection to determine at least one selected projection type from a plurality of predetermined projection types corresponding to the distance range, wherein the distance information indicates a distance between the hybrid depth detection device and one or more target objects, the distance range is selected from a plurality of predetermined distance ranges for the distance, and the at least one selected projection type is selected from the plurality of predetermined projection types; the adaptive projector may be used to project the at least one selected projection type to allow the hybrid depth detection device to extract at least one corresponding image via the camera; and the camera may be used to extract the at least one corresponding image. In particular, the image processing circuit performs depth detection of at least one type corresponding to the at least one selected projection type among multiple types of depth detection based on the at least one corresponding image to generate at least one depth map; and in response to the at least one selected projection type, the image processing circuit selectively outputs the at least one depth map as a result depth map or performs depth data combination based on the at least one depth map to generate a combined depth map as the result depth map.

[0007] The method and related apparatus of the present invention (e.g., the hybrid depth detection device, the image processing circuit in the hybrid depth detection device, the depth processor in the image processing circuit, and the adaptive projector in the hybrid depth detection device) ensure that various electronic products equipped with the hybrid depth detection device can operate normally under various conditions. Furthermore, the method and related apparatus of the present invention can utilize the adaptive projector to perform different types of projection to capture corresponding images with the camera, and can perform different types of depth detection based on these corresponding images to generate different intermediate depth maps. Furthermore, depth data can be combined based on these intermediate depth maps to generate a combined depth map, such as a weighted depth map, for accurate and efficient hybrid depth detection. Compared to related technologies, the method and related apparatus of the present invention can enhance overall performance without introducing side effects or with the low likelihood of introducing side effects. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of a hybrid depth detection device according to an embodiment of the present invention.

[0009] Figure 2 Illustration based on an embodiment of the present invention Figure 1 Some implementation details of the adaptive projector are shown.

[0010] Figure 3 An adaptive field control scheme for a method of performing mixed depth detection using an adaptive projector is illustrated according to an embodiment of the present invention.

[0011] Figure 4 A projection result of pattern projection in an adaptive projection type control scheme according to an embodiment of the present invention is illustrated.

[0012] Figure 5 According to an embodiment of the present invention, some projection results of non-pattern projection in the adaptive projection type control scheme of the method are illustrated.

[0013] Figure 6 An adaptive pattern control scheme of the method is illustrated according to an embodiment of the present invention.

[0014] Figure 7 An adaptive field-of-illumination (FOI) control scheme according to an embodiment of the present invention is illustrated.

[0015] Figure 8 Illustration based on an embodiment of the present invention Figure 1 Some implementation details of the hybrid depth detection device shown.

[0016] Figure 9 A depth combination control scheme of the method is illustrated according to an embodiment of the present invention.

[0017] Figure 10 A workflow of the method is illustrated according to an embodiment of the present invention.

[0018] [Symbol Explanation]

[0019] 10: field

[0020] 11: Target object

[0021] 100: Mixed Depth Detection Device

[0022] 110: Adaptive Projector

[0023] 120: Image Extraction Module

[0024] 121: Camera

[0025] 130: Image processing circuit

[0026] 131: Deep Processor

[0027] 132: Image Processor

[0028] 100M: Electronic Products

[0029] D1, D2: Distance

[0030] W1, W2: Width

[0031] 140: Additional sensing module

[0032] S11~S13, S13', S14A, S14B, S15A, S15B, S16, S17: Steps Detailed Implementation

[0033] Embodiments of the present invention provide a method for performing hybrid depth detection using an adaptive projector, and at least a portion (e.g., part or all) of a hybrid depth detection device. The hybrid depth detection device can operate according to the described method to perform depth detection accurately and efficiently, achieving optimal performance for any electronic product equipped with the hybrid depth detection device. In particular, the hybrid depth detection device can perform the hybrid depth detection accurately and efficiently to generate one or more depth maps, such as depth maps of a face, enabling the electronic product to perform three-dimensional (3D) face recognition accurately and efficiently based on the one or more depth maps, but the invention is not limited thereto. Examples of the one or more depth maps may include, but are not limited to, depth maps of faces, indoor or outdoor locations, etc. Furthermore, the electronic product can utilize the hybrid depth detection device to generate at least one depth map corresponding to at least one user of the electronic product, for use in a pre-built 3D face database of the at least one depth map. It can also compare a target depth map (e.g., a target depth map of a person's face) among the one or more depth maps with any of the at least one depth map to determine whether that person is any of the at least one user. If the target depth map matches any of the depth maps (e.g., the person is that user), the electronic product can provide services to that person; otherwise (e.g., the person is not any of the at least one users), the electronic product can prevent that person from using any of the services. Examples of the electronic product may include, but are not limited to, mobile devices such as multifunction mobile phones, tablet computers, wearable devices, all-in-one (AIO) computers, and laptop computers.

[0034] Figure 1 This is a schematic diagram of a hybrid depth detection device 100 according to an embodiment of the present invention, wherein the hybrid depth detection device 100 and the adaptive projector 110 therein can be examples of the hybrid depth detection device and the adaptive projector, respectively. For better understanding, in Figure 1The upper part can be referenced to the X-axis, Y-axis, and Z-axis to depict a field 10 and one or more target objects 11, but the invention is not limited thereto. For example, the field 10 (e.g., its shape, related dimensions, and / or related angles) and the one or more target objects 11 (e.g., their object count, shape, related dimensions, and / or related positions) can be varied.

[0035] like Figure 1 As shown, in addition to the adaptive projector 110, the hybrid depth detection device 100 may also include an image-capturing module 120 and an image processing circuit 130, wherein the adaptive projector 110 and the image-capturing module 120 are coupled to the image processing circuit 130. For example, the image-capturing module 120 may include at least one camera (e.g., one or more cameras) such as a camera 121, and the image processing circuit 130 may include at least one processor (e.g., one or more processors) such as a depth processor 131 and an image processor 132 corresponding to depth-related processing and two-dimensional (2D) image processing, respectively, but the invention is not limited thereto. In some embodiments, the depth processor 131 and the image processor 132 may be integrated into the same processor.

[0036] The adaptive projector 110 can be used to project one or more predetermined patterns of invisible light (e.g., infrared (IR) or other wavelengths). For example, a laser module within the adaptive projector 110 may include a light emitter such as a laser emitter and certain optical elements, wherein the light emitter such as the laser emitter may include a vertical-cavity surface-emitting laser (VCSEL) array for emitting invisible light, and these optical elements may include a diffractive optical element (DOE), a microlens array, or a wafer-level optics (WLO) element for generating the one or more predetermined patterns. In particular, the WLO element may include at least one wafer-level lens (e.g., one or more wafer-level lenses). In addition, the adaptive projector 110 may also include one or more electrically-controlled optical elements, such as at least one liquid crystal (LC) lens (e.g., one or more LC lenses, any one of which may be located inside or outside the laser module), to change (e.g., reduce or enlarge) one or more projection fields of the one or more predetermined patterns, and / or switch between different types of projection (e.g., a first projection type for pattern projection of the one or more predetermined patterns, and a second projection type for non-pattern projection without patterns).

[0037] For better understanding, multiple light transmission (Tx) paths from the adaptive projector 110 to the field 10 (e.g., multiple invisible light Tx paths from the adaptive projector 110 to the field 10) can be collectively represented by the Tx direction (denoted as "Tx" for simplicity), and multiple light receiving (Rx) paths from the field 10 to the camera 121 (e.g., multiple invisible light Rx paths from the one or more target objects 11 to the camera 121) can be collectively represented by the Rx direction (denoted as "Rx" for simplicity).

[0038] During depth detection, such as hybrid depth detection, the hybrid depth detection device 100 (e.g., elements therein) can perform the following operations:

[0039] (1) Camera 121 can capture / extract an image containing image content pointing to one or more objects (e.g., the one or more target objects 11), the image typically does not have depth information and may represent a two-dimensional (2D) image;

[0040] (2) The image processing circuit 130 (e.g., its first partial circuit, such as image processor 132) can determine a region of interest (ROI) (e.g., the region of a face, such as the region surrounding the face of the person mentioned above) extracted by the camera 121 based on the image content of the image, for further processing by the image processing circuit 130 (e.g., its second partial circuit).

[0041] (3) The image processing circuit 130 (e.g., its second local circuitry such as the depth processor 131) can perform projection type selection to determine at least one selected projection type (e.g., one or more selected projection types) from a plurality of predetermined projection types (e.g., the first projection type and the second projection type) corresponding to a selected distance range, wherein the distance range may be selected from a plurality of predetermined distance ranges for the distance between the hybrid depth detection device 100 and the one or more target objects 11, and the at least one selected projection type may be selected from the plurality of predetermined projection types;

[0042] (4) The adaptive projector 110 can perform projection of the at least one selected projection type, in particular, projection of the plurality of predetermined projection types (e.g., the first projection type and the second projection type), such as pattern projection and non-pattern projection, to extract corresponding images with the camera 121 respectively, wherein the non-pattern projection can be designed to produce uniform / quasi-uniform illumination or partial-uniform illumination (e.g., flood-type illumination, as uniform as possible);

[0043] (5) The image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) can perform depth detection (e.g., one or more types of depth detection) corresponding to the at least one selected projection type among multiple types of depth detection (e.g., a first depth detection corresponding to the first projection type, such as pattern-detection-type depth detection, and a second depth detection corresponding to the second projection type, such as time-of-flight (TOF) type depth detection), in particular, performing the multiple types of depth detection (e.g., the first depth detection such as the pattern-detection-type depth detection, and the second depth detection such as the TOF type depth detection) based on these corresponding images respectively to generate a first depth map and a second depth map, wherein the first depth map and the second depth map can be regarded as intermediate depth maps; and

[0044] (6) The image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) can combine depth data based on the first depth map and the second depth map to generate a combined depth map such as a weighted depth map.

[0045] The depth data combination may be made with reference to the distance range, the distance between the mixed depth detection device 100 and the one or more target objects 11 (such as the distance indicated by the distance information), and / or the ROI.

[0046] For example, image processing circuit 130 (e.g., a second local circuit of image processing circuit 130 such as depth processor 131) can determine the respective weighting values ​​of the first depth map and the second depth map based on the distance range. In particular, the respective weighting values ​​of the first depth map and the second depth map can be adjusted (e.g., fine-tuned) for a predetermined region within the weighted depth map in a depth-by-depth / pixel-by-pixel manner to improve the depth accuracy at the edges of one or more objects and to generate the weighted depth map accurately and efficiently, but the invention is not limited thereto. For example, the predetermined region may include the entire weighted depth map. In some examples, the predetermined region may include a portion of the weighted depth map, such as a first adjustment region (e.g., a first fine-tuning region) in the weighted depth map corresponding to the ROI, and a second adjustment region (e.g., a second fine-tuning region) in the weighted depth map corresponding to a sub-region of the ROI, etc.

[0047] Since the one or more depth maps, such as the target depth map, the at least one depth map, such as any depth map, can be generated in the same way as the combined depth map, such as the weighted depth map, the mixed depth detection device 100 can perform the mixed depth detection accurately and efficiently.

[0048] against Figure 1 Certain implementation details of the illustrated architecture can be further described below. For example, a first local circuit of the image processing circuit 130, such as the image processor 132, can generate ROI information indicating the ROI, such as 2D ROI information indicating the ROI of the 2D image (in... Figure 1 (The ROI is labeled "2D ROI" for better understanding). Additionally, a second local circuit of the image processing circuit 130, such as a depth processor 131, can obtain the ROI information indicating the ROI from a first local circuit, such as an image processor 132, and can therefore inform / inform the ROI for related processing, such as adjusting (e.g., fine-tuning) the weighting value of the intermediate depth map as described above. Furthermore, the hybrid depth detection device 100 can utilize the image processing circuit 130 (e.g., depth processor 131) to generate one or more control signals to control the adaptive projector 110. For example, one or more drive circuits within the image processing circuit 130 can be used to generate the one or more control signals under the control of the depth processor 131 to control (e.g., drive) the adaptive projector 110, wherein the one or more control signals may carry projection type information indicating the at least one selected projection type (e.g., the first projection type and / or the second projection type) for informing / informing the adaptive projector 110 of the at least one selected projection type.

[0049] Since the hybrid depth detection device 100 (e.g., its components, such as the adaptive projector 110, image processing circuitry 130, and depth processor 131) can operate according to this method, the hybrid depth detection device 100 can perform the hybrid depth detection accurately and efficiently to generate one or more depth maps, such as a depth map of a face, enabling electronic products equipped with the hybrid depth detection device 100 to perform 3D face recognition accurately and efficiently based on the one or more depth maps. Thus, overall performance can be improved.

[0050] According to some embodiments, image processing circuit 130 (e.g., depth processor 131) can calculate a set of depths for the first depth map based on a projection result of a predetermined pattern (e.g., any one of the one or more predetermined patterns) and the differences between the predetermined patterns, wherein the projection result can indicate changes in the predetermined pattern due to the non-planar surface of a face. In particular, the predetermined pattern may include multiple sub-patterns (e.g., multiple invisible light tiles), and based on knowledge of relevant geometric relationships, image processing circuit 130 (e.g., depth processor 131) can calculate the set of depths based on the displacement of certain sub-patterns (e.g., certain invisible light tiles) among the multiple sub-patterns (e.g., the multiple invisible light tiles) of the predetermined pattern, because depth variations in the set of depths can correspond to these displacements.

[0051] For the depth calculations described above, when the baseline between the center of the adaptive projector 110 (e.g., the position of the optical axis of its optics on a reference plane of the hybrid depth detection device 100) and the center of the image extraction module 120 (e.g., the position of the optical axis of the camera 121 on the reference plane) is much smaller than the distance between the hybrid depth detection device 100 and the one or more target objects 11 (e.g., the ratio of the length of the baseline to the distance is less than a predetermined ratio), the image processing circuit 130 (e.g., depth processor 131) may omit the baseline in these depth calculations, wherein the length of the baseline may be forcibly set to zero, but the invention is not limited thereto. Taking the baseline into account, the image processing circuit 130 (e.g., depth processor 131) may use the actual value of the baseline length in these depth calculations.

[0052] According to some embodiments, the predetermined pattern can be implemented using structured-light (SL) technology, etc., where the pattern detection type depth detection can be referred to as SL-type depth detection, but the present invention is not limited thereto. Furthermore, during pattern detection type depth detection such as SL-type depth detection, the image processing circuit 130 (e.g., depth processor 131) can use these displacements as references for these depth calculations, but the present invention is not limited thereto. For example, during TOF-type depth detection, the image processing circuit 130 (e.g., depth processor 131) can use the phase or time difference of the light pulses arriving at the image sensor of camera 121 as an indicator of the distance from field 10 to the imaging system of camera 121 to perform these depth calculations.

[0053] Figure 2 Illustration based on an embodiment of the present invention Figure 1Some implementation details of the adaptive projector 110 are shown. The adaptive projector 110 can be implemented using various combinations of different groups of components, such as... Figure 2 The combinations shown in cases (a), (b), (c), (d), (e), and (f) are used to achieve this:

[0054] (a) The at least one LC lens includes an LC lens located outside the laser module, and the WLO element is located between the DOE / microlens array and the laser emitter;

[0055] (b) The at least one LC lens comprises an LC lens located outside the laser module and does not use WLO elements;

[0056] (c) In a Type-1 configuration, the at least one LC lens includes a first LC lens located outside the laser module and a second LC lens located inside the laser module, the DOE / microlens array is located between the first LC lens and the second LC lens, and the WLO element is located between the second LC lens and the laser emitter;

[0057] (d) The at least one LC lens comprises an LC lens located within the laser module, between the DOE / microlens array and the laser emitter, and does not use WLO elements;

[0058] (e) The at least one LC lens comprises a first LC lens located outside the laser module and a second LC lens located inside the laser module, between the DOE / microlens array and the laser emitter, and does not use WLO elements; and

[0059] (f) In a Type-2 configuration, the at least one LC lens includes a first LC lens located outside the laser module and a second LC lens located inside the laser module, the DOE / microlens array is located between the first LC lens and the WLO element, and the second LC lens is located between the WLO element and the laser emitter;

[0060] However, the invention is not limited thereto. For example, certain elements may be integrated into the same element and / or one or more elements may be added, removed or changed, as long as similar results can be achieved.

[0061] According to some embodiments, the laser emitter may include at least one invisible light source (e.g., one or more invisible light sources), such as the VCSEL array, an edge-emitting laser diode (LD), etc. In addition to the at least one invisible light source, the laser emitter may also include a beam limiting device (e.g., a collimator lens). The beam limiting device can receive the invisible light emitted by the at least one invisible light source and convert the emitted invisible light into beam-limited invisible light. For the sake of simplicity, similar details in these embodiments will not be repeated here.

[0062] According to certain embodiments, any LC lens of the at least one LC lens (e.g. Figure 2 Any of the LC lenses shown can be implemented by multiple LC lens elements (e.g., LC lens elements respectively aligned to the optical axis of the LC lens and positioned at predetermined positions according to the optical design of the LC lens; these may be referred to as cascaded LC lens elements for simplicity), but the invention is not limited thereto. In some embodiments, any LC lens can be implemented by a single LC lens element (e.g., a compact LC lens element designed to have all the optical features required for the LC lens).

[0063] For any of the multiple predetermined projection types (e.g., the first projection type and the second projection type), the image processing circuit 130 (e.g., a second local circuit such as the depth processor 131) can control the adaptive projector 110 (e.g., one of the LC lenses in the at least one LC lens) to change (e.g., reduce or enlarge) the projection field of that type of projection. For example, when this type of projection represents the first projection type for pattern projections of one or more predetermined patterns, the image processing circuit 130 (e.g., the second local circuit such as the depth processor 131) can control the adaptive projector 110 (e.g., the LC lens) to change (e.g., reduce or enlarge) the projection field of the pattern projection; and when this type of projection represents the second projection type for non-pattern projections without patterns, the image processing circuit 130 (e.g., the second local circuit such as the depth processor 131) can control the adaptive projector 110 (e.g., the LC lens) to change (e.g., reduce or enlarge) the projection field of the non-pattern projection; however, the invention is not limited thereto. In some embodiments, the image processing circuitry 130 (e.g., a second local circuit such as depth processor 131) may control the adaptive projector 110 (e.g., the LC lens) to change the projection field of only one of the plurality of predetermined projection types (e.g., for the second projection type that is not a pattern projection), rather than controlling the adaptive projector 110 (e.g., the LC lens) to change the projection field of all of the plurality of predetermined projection types. For example, for a normal / long-distance target object at a normal or long (or far) distance (e.g., a distance greater than the normal distance), the TOF-type depth detection can achieve better depth detection accuracy than the pattern detection-type depth detection, and for a short (or near) distance target object (e.g., a distance shorter than the normal distance), the pattern detection-type depth detection can achieve very high depth detection accuracy except for the object edges of the short-distance target object. Image processing circuit 130 (e.g., a second local circuit such as depth processor 131) can trigger non-pattern projection and perform TOF-type depth detection on normal / long-distance target objects to achieve the aforementioned superior depth detection accuracy. Furthermore, image processing circuit 130 (e.g., a second local circuit such as depth processor 131) can trigger pattern projection and perform pattern detection-type depth detection on short-distance target objects to achieve the aforementioned very high depth detection accuracy, and can also trigger non-pattern projection and perform TOF-type depth detection on short-distance target objects to adjust (e.g., fine-tune) certain depths at the object edges of short-distance target objects to improve depth detection accuracy at the object edges of short-distance target objects. Therefore, the method and related apparatus of the present invention can improve overall performance without introducing any side effects or in a manner unlikely to introduce side effects.

[0064] According to certain embodiments, one or more elements within the adaptive projector 110 (e.g., one or more of a DOE, microlens array, and WLO element) may be designed to project patterns within a narrower projection field, and the image processing circuitry 130 (e.g., depth processor 131) may control the adaptive projector 110 (e.g., an LC lens) to convert pattern projections into non-pattern projections within that narrower projection field; however, the invention is not limited thereto. For example, one or more elements within the adaptive projector 110 (e.g., one or more of a DOE, microlens array, and WLO element) may be designed to project patterns within a wider projection field, and the image processing circuitry 130 (e.g., depth processor 131) may control the adaptive projector 110 (e.g., an LC lens) to convert pattern projections into non-pattern projections within that wider projection field.

[0065] Figure 3 An adaptive field control scheme for a method of performing mixed depth detection using an adaptive projector 110 is illustrated according to an embodiment of the present invention. For better understanding, the mixed depth detection device 100 (e.g., its components, such as the adaptive projector 110 and image extraction module 120) can be integrated into an electronic product 100M, such as a mobile device, and can be based on... Figure 3 The adaptive field control scheme shown operates, in particular, with certain parameters (e.g., distances D1 and D2, widths W1 and W2, and associated angles), where distances D1 and D2 may represent a minimum operating distance and the normal operating distance, respectively, and the short distance may fall within the interval [D1, D2). These parameters (e.g., distances D1 and D2, widths W1 and W2, and associated angles) and the dimensions and shape of the electronic product 100M are for illustrative purposes only and should not be considered as limiting the invention. In some embodiments, these parameters and the dimensions and shape of the electronic product 100M may be varied.

[0066] like Figure 3 As shown in the upper part, the image processing circuit 130 (e.g., a second local circuit such as depth processor 131) can trigger pattern projection and perform pattern detection-type depth detection on short-range target objects. The image processing circuit 130 (e.g., the second local circuit such as depth processor 131) can control the adaptive projector 110 (e.g., the LC lens) to use a fixed projection field during pattern projection. Furthermore, the image processing circuit 130 (e.g., the second local circuit such as depth processor 131) can trigger non-pattern projection and perform TOF-type depth detection on short-range target objects to adjust (e.g., fine-tune) the depth of the object edges of the short-range target objects. The projection field of the non-pattern projection is variable as needed. Figure 3As shown in the lower half, the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) can trigger non-pattern projection and perform this TOF-type depth detection on normal / long-distance target objects. For the sake of simplicity, similar content will not be repeated here in this embodiment.

[0067] Figure 4 According to an embodiment of the present invention, a projection result of pattern projection in an adaptive projection type control scheme of the method is illustrated. For better understanding, it is assumed that the short-distance target object is a planar target, but the present invention is not limited thereto. Furthermore, the short distance can be greater than distance D1, and in particular, it can fall within the range of (D1, D2). The adaptive projector 110 can be designed to project the predetermined pattern with a fixed projection field. When being projected as the projection result, the plurality of sub-patterns of the predetermined pattern (e.g., the plurality of invisible light patches, such as...) Figure 4 The spots shown can be distributed across the fixed projection field. Image processing circuitry 130 (e.g., a second local circuit such as depth processor 131) can utilize camera 121 to extract a 3D raw image (labeled "3D raw" for brevity) for performing pattern detection-type depth detection, wherein the 3D raw image may contain at least a portion (e.g., a portion or all) of the projection result. For simplicity, similar details are not repeated here.

[0068] According to some embodiments, image processing circuitry 130 (e.g., a second local circuit such as depth processor 131) can control adaptive projector 110 (e.g., one of the at least one LC lenses) to switch between multiple projection types (e.g., a first projection type for patterned projection of one or more predetermined patterns, and a second projection type for non-patterned projection without patterns). For example, DOE, microlens array, and / or WLO elements can be used to generate the one or more predetermined patterns. When a switch from the first projection type to the second projection type is required, under the control of image processing circuitry 130 (e.g., the second local circuit such as depth processor 131), adaptive projector 110 (e.g., the LC lens) can temporarily disable the function of generating the one or more predetermined patterns, for example, by changing the focal length of the LC lens to prevent the one or more predetermined patterns from being output via a predetermined imaging path in an optical design of the imaging system (e.g., DOE, microlens array, and / or WLO elements), so that patterned projection becomes non-patterned projection. When it is necessary to switch from the second projection type to the first projection type, under the control of the image processing circuit 130 (e.g., the second local circuitry such as the depth processor 131), the adaptive projector 110 (e.g., the LC lens) can enable the function of generating one or more predetermined patterns. For example, by changing the focal length of the LC lens back to a predetermined focal length value in the optical design, the non-pattern projection is converted into a pattern projection. For the sake of simplicity, similar content in these embodiments will not be repeated here.

[0069] Figure 5 According to an embodiment of the present invention, some projection results of non-pattern projection in the adaptive projection type control scheme of the method are illustrated. For better understanding, it is assumed that short-distance target objects and normal / long-distance target objects are planar targets, but the present invention is not limited thereto. Furthermore, Figure 5 The projection result shown in the left half can serve as an example of a non-patterned projection result used for short-range target objects at this short distance, while Figure 5 The projection result of the right half can serve as an example of the projection result of non-patterned projection for normal / long-distance target objects at that normal or long distance, where the shaded portion can indicate that the illumination uniformity of corners or outer areas is not as good as that of the central area. When this type of TOF depth detection is required for normal / long-distance target objects, the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) can control the adaptive projector 110 (e.g., an LC lens) to change (e.g., reduce) the projection field of the non-patterned projection, so that the projection result of the non-patterned projection has a narrower distribution and higher intensity within a corresponding projection field, such as... Figure 5As shown in the right half, when this type of TOF depth detection is required for short-range target objects, the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) can control the adaptive projector 110 (e.g., an LC lens) to change (e.g., enlarge) the projection field of the non-pattern projection, so that the projection result of the non-pattern projection has a wider and more uniform distribution on a corresponding projection field, such as... Figure 5 As shown in the left half. For the sake of simplicity, similar content will not be repeated in this embodiment.

[0070] Figure 6 An adaptive pattern control scheme of the method is illustrated according to an embodiment of the present invention. The one or more predetermined patterns can be configured according to various designs, such as... Figure 6 The designs shown in cases (a), (b), (c), and (d) are implemented (e.g., by changing parameters or arrangements):

[0071] (a) High-density subpattern: Each subpattern of the predetermined pattern may have high density;

[0072] (b) Low-density subpatterns: Each subpattern of the predetermined pattern may have low density;

[0073] (c) Multi-density subpatterns: Each subpattern of the predetermined pattern may contain multiple subsets, and the density of each subset of the subpattern may contain different densities (e.g., the densities in cases (a) and (b) above); and

[0074] (d) Multi-zone multi-density pattern: The predetermined pattern may contain multiple zones, wherein some sub-patterns in one or more of the multiple zones may have a first density (e.g., the low density in case (b) above), and some sub-patterns in one or more other zones may have a second density (e.g., the high density in case (a) above).

[0075] However, the present invention is not limited thereto. For the sake of simplicity, similar content will not be repeated in this embodiment.

[0076] According to certain embodiments, image processing circuitry 130 (e.g., a second local circuit such as depth processor 131) can control adaptive projector 110 (e.g., an LC lens) to change the projection field and / or projection direction of at least a portion (e.g., a portion or all) of the predetermined pattern, such as a sub-pattern, a secondary sub-pattern of a sub-pattern (e.g., a subset), etc., to adapt to various designs such as Figure 6Switching between the designs shown in cases (a), (b), (c), and (d) is illustrated. For the sake of simplicity, similar details in these embodiments will not be repeated here.

[0077] Figure 7 An adaptive field-of-illumination (FOI) control scheme according to an embodiment of the present invention is illustrated. The FOI projected in this drawing can be adapted to various designs, such as... Figure 7 The designs shown in cases (a), (b), and (c) are implemented (e.g., by changing the FOI in one or both directions):

[0078] (a) The original FOI;

[0079] (b) FOI switching in one direction; and

[0080] (c) FOI switching in both directions;

[0081] However, the present invention is not limited thereto. For the sake of simplicity, similar content will not be repeated in this embodiment.

[0082] According to certain embodiments, image processing circuitry 130 (e.g., a second local circuit such as depth processor 131) can control adaptive projector 110 (e.g., an LC lens) to change the projection field and / or projection direction of at least a portion (e.g., a portion or all) of the predetermined pattern, such as a sub-pattern, a primary sub-pattern of a sub-pattern (e.g., a subset), etc., to adapt to various designs such as Figure 7 Switching between the designs shown in cases (a), (b), and (c) is illustrated. For the sake of simplicity, similar details in these embodiments will not be repeated here.

[0083] According to some embodiments, the image processing circuit 130 (e.g., depth processor 131) can send the 2D image to a system-on-chip (SoC) to utilize a back-end object detection algorithm running on the SoC to find the ROI (e.g., its position and size, such as width and height) surrounding a target object (e.g., a face). The back-end object detection algorithm running on the SoC then feeds back the ROI (e.g., its position and size, such as width and height) to the image processing circuit 130 (e.g., depth processor 131). The image processing circuit 130 (e.g., depth processor 131) can calculate a depth average pixel luminance (APL), such as the average luminance (e.g., pixel value) of the respective depth pixels within the ROI on the 2D image, as a depth statistics index to determine the distance information, such as the distance between the target object (e.g., a face) and the hybrid depth detection device 100, thereby determining the selected distance range. However, the invention is not limited thereto. For example, image processing circuit 130 (e.g., depth processor 131) may calculate a depth median, such as the median of the brightness (e.g., pixel value) of the respective depth pixels within the ROI on the 2D image, as a depth statistic to determine the distance information, thereby determining the selected distance range. Furthermore, image processing circuit 130 (e.g., depth processor 131) may use the depth APL or the depth median as an average depth of the depth map during depth calculation, but the invention is not limited thereto.

[0084] According to certain embodiments, image processing circuitry 130 (e.g., depth processor 131) can calculate any one of a plurality of depths in the weighted depth map (e.g., a weighted depth image) according to the following equation:

[0085] Depth WEIGHTED (x, y) = w1(x, y) * Depth SLM (x, y) + w2(x, y) * Depth TOF (x, y);

[0086] Among them Depth WEIGHTED (x, y) can represent any depth at coordinates (x, y) in the weighted depth map (e.g., the weighted depth image). SLM (x, y) and Depth TOF(x, y) can represent the corresponding depth at coordinates (x, y) in the first depth map (e.g., an SL depth map obtained from the SL type depth detection, such as an SL depth image) and the second depth map (e.g., a TOF depth map obtained from the TOF type depth detection, such as a TOF depth image), and w1(x, y) and w2(x, y) can represent the relevant weighted values, but the present invention is not limited thereto.

[0087] According to some embodiments, the image processing circuit 130 (e.g., depth processor 131) can switch alternately between the plurality of predetermined projection types and correspondingly switch between the plurality of depth detection types. Assuming that pattern projection and non-pattern projection can be referred to as speckle projection and fogging projection, respectively, they can correspond to their respective projection results. For example, when the depth statistic indicates a short distance range among the plurality of predetermined distance ranges, the image processing circuit 130 (e.g., depth processor 131) can control the adaptive projector 110 to alternately perform speckle projection and fogging projection using invisible light (for invisible light illumination), wherein the output information corresponding to the speckle projection and the fogging projection using invisible light can be the first depth map (e.g., the SL depth map) and the 2D image, respectively.

[0088] Furthermore, when the depth statistics indicate a distance range among the plurality of predetermined distance ranges, the image processing circuit 130 (e.g., depth processor 131) can control the adaptive projector 110 to alternately perform speckle projection, a cycle of invisible light fogging projection (for invisible light illumination), and another cycle of invisible light fogging projection (for invisible light illumination). The output information corresponding to the speckle projection, the aforementioned cycle of invisible light fogging projection, and the aforementioned another cycle of invisible light fogging projection can be the first depth map (e.g., the SL depth map), the second depth map (e.g., the TOF depth map), and the 2D image, respectively, but the present invention is not limited thereto. For example, when the depth statistics indicate the mid-range distance, the image processing circuit 130 (e.g., depth processor 131) can control the adaptive projector 110 to alternately perform speckle projection and fog projection using invisible light (for invisible light illumination), wherein the output information corresponding to the speckle projection and the fog projection using invisible light can be the first depth map (e.g., the SL depth map) and the second depth map (e.g., the TOF depth map), respectively.

[0089] Furthermore, when the depth statistics indicate a long distance range among the plurality of predetermined distance ranges, the image processing circuit 130 (e.g., depth processor 131) can control the adaptive projector 110 to alternately perform one cycle of invisible light-based fogging projection (for invisible light illumination) and another cycle of invisible light-based fogging projection (for invisible light illumination), wherein the output information corresponding to the aforementioned one cycle of invisible light-based fogging projection and the aforementioned another cycle of invisible light-based fogging projection can be the second depth map (e.g., the TOF depth map) and the 2D image, respectively. For the sake of simplicity, similar content in these embodiments will not be repeated here.

[0090] According to some embodiments, in addition to the hybrid depth detection device 100, the electronic product may also include a processing circuit (e.g., the SoC) for controlling the operation of the electronic product, and the processing circuit may include at least one application processor (e.g., one or more application processors) for running various program code such as operating system (OS), drivers, applications, etc. Under the control of a first application (referred to as App) running on the at least one application processor, such as a 3D face recognition App, the processing circuit may send one or more commands to the hybrid depth detection device 100 to control the hybrid depth detection device 100.

[0091] Figure 8 Illustration based on an embodiment of the present invention Figure 1 Some implementation details of the hybrid depth detection device shown. For example... Figure 8 As shown, the hybrid depth detection device 100 (e.g., its components such as the adaptive projector 110, image extraction module 120, image processing circuitry 130, and an additional sensing module 140 including a distance sensor) operates under the control of at least one application processor (collectively referred to as "AP" for simplicity) in the processing circuitry of the electronic product. The image sensor of the camera 121 in the image extraction module 120 is also illustrated for better understanding, but the invention is not limited thereto. In some embodiments, the additional sensing module 140 (e.g., the distance sensor) may be omitted. Additionally, the one or more driving circuits in the image processing circuitry 130 may include an LC lens driver for driving the at least one LC lens with one or more LC voltage signals (denoted as "LC voltage" for simplicity), and may include a laser driver for driving the at least one invisible light source, such as the VCSEL array, these edge-emitting LDs, etc., with one or more LD current signals (denoted as "LD current" for simplicity). Figure 8Some components within the illustrated architecture can communicate with each other, for example, by using certain synchronization signals and one or more predetermined communication protocols (denoted as "synchronization" and "communication protocol" respectively for simplicity). Furthermore, a core circuit within the depth processor 131 controls the operation of the depth processor 131. The depth processor 131 can acquire image data of raw images from the image sensor of the camera 121 within the image extraction module 120, perform relevant processing such as depth processing, and output relevant data such as image data and depth data to the AP. For simplicity, similar details are not repeated here in this embodiment.

[0092] Figure 9 A depth combination control scheme of the method is illustrated according to an embodiment of the present invention. The image processing circuit 130 may include a synchronization generation module for generating certain synchronization signals (denoted as "synchronization" for brevity) and communicating with certain other components using one or more predetermined communication protocols (denoted as "communication protocols" for brevity), and includes a selection and multiplexing module (denoted as "selection / multiplexing" for brevity) for acquiring and / or dispatching relevant information, such as range selection information (denoted as "range selection" for brevity) indicating the distance range (e.g., the distance range selected from the multiple predetermined distance ranges), these original images, etc., and the depth processor 131 within the image processing circuit 130 may include a TOF-type depth processor and an SL-type depth processor for performing the TOF-type depth detection and the SL-type depth detection respectively, and includes a depth data combination module for performing the depth data combination, wherein the image processor 132 may be referred to as a 2D image processor, but the present invention is not limited thereto. In some embodiments, Figure 9 The architecture shown can be modified.

[0093] According to some embodiments, the synchronization generation module may include a submodule such as a depth statistics module for performing depth statistics on multiple frames. The depth information received by the synchronization generation module may be replaced with a previous depth frame, and the synchronization generation module may perform the depth statistics based on the previous depth frame to calculate the depth APL of the previous depth frame as a predicted depth APL of the current depth frame to be generated, so as to generate the range selection information and associated synchronization signal for output to the selection and multiplexing module. In particular, the current depth frame (e.g., the weighted depth image) being generated by the image processing circuit 130 may be a depth frame F(n), while the previous depth frame may be a depth frame F(n-1) (e.g., a weighted depth frame F(n-1)). For the sake of simplicity, similar content in these embodiments will not be repeated here.

[0094] Figure 10 A workflow of the method is illustrated according to an embodiment of the present invention. The method can be applied to and executed by related devices (e.g., hybrid depth detection device 100, image processing circuit 130, depth processor 131, and adaptive projector 110).

[0095] In step S11, the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) obtains the distance information, which indicates the distance between the hybrid depth detection device 100 and the one or more target objects 11. For example, the distance information may be obtained from the distance sensor, but the invention is not limited thereto. In some examples, the source of the distance information may include one or a combination of the distance sensor and a previous depth detection operation of the hybrid depth detection device 100 (e.g., a hybrid depth detection operation for a previous frame).

[0096] In step S12, the hybrid depth detection device 100 may use an image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) to determine the distance range (e.g., the selected distance range) based on distance information (e.g., the distance indicated by the distance information), wherein the distance range is selected from a plurality of predetermined distance ranges for that distance. When the distance indicated by the distance information falls within one of the plurality of predetermined distance ranges, the image processing circuit 130 (e.g., the second local circuit such as the depth processor 131) may determine this distance range as the selected distance range. When the distance indicated by the distance information falls within another of the plurality of predetermined distance ranges, the image processing circuit 130 (e.g., the second local circuit such as the depth processor 131) may determine that other distance range as the selected distance range.

[0097] In step S13, the hybrid depth detection device 100 may use the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) to perform the projection type selection to determine at least one selected projection type from the plurality of predetermined projection types (e.g., the first projection type and the second projection type) that corresponds to the distance range (e.g., the selected distance range), wherein the at least one selected projection type is selected from the plurality of predetermined projection types.

[0098] In step S13', in response to the at least one selected projection type, the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) may trigger the execution of one of a plurality of sub-flows of the workflow. For example, if the at least one selected projection type represents the first projection type for pattern projection for use in pattern detection type depth detection such as the SL type depth detection (labeled "SL type" for better understanding), then a first sub-flow including step S14A is executed; otherwise, if the at least one selected projection type represents the second projection type for non-pattern projection for use in TOF type depth detection (labeled "TOF type" for better understanding), then a second sub-flow including step S14B is executed; otherwise, if the at least one selected projection type includes the first projection type and the second projection type (labeled "mixed" for better understanding), a third sub-flow including steps S15A, S15B, and S16 is executed.

[0099] In step S14A, the hybrid depth detection device 100 can use the adaptive projector 110 to project a pattern, specifically, to project the predetermined pattern so that at least one corresponding first image, such as one or more first images, can be extracted by the camera 121, and the first depth detection can be performed by the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) based on the one or more first images to generate the first depth map, which is then output as the result depth map.

[0100] In step S14B, the hybrid depth detection device 100 can use the adaptive projector 110 to perform non-pattern projection to extract at least one corresponding second image, such as one or more second images, by the camera 121, and use the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) to perform the second depth detection based on the one or more second images to generate the second depth map, which is then output as the result depth map.

[0101] In step S15A, the hybrid depth detection device 100 can use the adaptive projector 110 to project a pattern, in particular, to project the predetermined pattern so that at least one corresponding first image, such as one or more first images, can be extracted by the camera 121, and the first depth detection can be performed by the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) based on the one or more first images to generate the first depth map.

[0102] In step S15B, the hybrid depth detection device 100 may use the adaptive projector 110 to perform non-pattern projection to extract at least one corresponding second image, such as one or more second images, by the camera 121, and use the image processing circuit 130 (e.g., a second local circuit such as a depth processor 131) to perform the second depth detection based on the one or more second images to generate the second depth map.

[0103] In step S16, if the at least one selected projection type includes the plurality of predetermined projection types such as the first projection type and the second projection type, the hybrid depth detection device 100 may use the image processing circuit 130 (e.g., the second local circuit such as the depth processor 131) to combine the depth data based on the first depth map and the second depth map to generate the combined depth map (e.g., the weighted depth map) as the result depth map.

[0104] In step S17, the hybrid depth detection device 100 (e.g., image processing circuit 130) determines whether to stop the workflow. If yes, the workflow ends; if no, proceed to step S11.

[0105] For example, in response to a start command from one or more commands, the hybrid depth detection device 100 (e.g., image processing circuit 130) can begin operation according to the workflow to send the resulting depth map (e.g., the first depth map of the first sub-process, the second depth map of the second sub-process, or the combined depth map of the third sub-process) back to the processing circuit (e.g., the 3D face recognition app running on the at least one application processor). In step S17, the hybrid depth detection device 100 (e.g., image processing circuit 130) can check whether a stop command from one or more commands has been received. When the stop command is received from the processing circuit, the hybrid depth detection device 100 (e.g., image processing circuit 130) can determine that the check result of step S17 is yes. When no stop command is received from the processing circuit, the hybrid depth detection device 100 (e.g., image processing circuit 130) can determine that the check result of step S17 is no. For the sake of simplicity, similar content will not be repeated here in this embodiment.

[0106] To better understand, this method is available Figure 10The workflow shown is for illustrative purposes only, but the invention is not limited thereto. According to some embodiments, one or more steps may be performed... Figure 10 The workflow shown can be added, deleted, or modified. For example, in this third sub-flow containing steps S15A, S15B, and S16, the order in which steps S15A and S15B are executed can be interchanged.

[0107] According to some embodiments, the plurality of predetermined distance ranges may include a first range (e.g., the short distance range) such as the interval [D1, D1.5), a second range (e.g., the medium distance range) such as the interval [D1.5, D2), and a third range (e.g., the long distance range) such as the interval [D2, ∞), wherein distance D1.5 may represent an intermediate distance between distances D1 and D2, such as with a predetermined weighted value w D1 and w D2 Defined intermediate distance (w) D1 *D1+w D2 *D2)(e.g., w D1 >0 and w D2 >0, and w D1 +w D2 =1), but the present invention is not limited thereto. Furthermore, when the selected distance range represents the range of the interval [D1, D1.5), the at least one selected projection type can represent the first projection type to allow the first sub-process including step S14A to be executed, wherein, as Figure 3 The projection field shown in the upper part can serve as an example of a projection field for a pattern projection; when the selected distance range represents the range of the interval [D1.5, D2), the at least one selected projection type can include the first projection type and the second projection type to allow the third sub-process including steps S15A, S15B, and S16 to be executed, wherein, as Figure 3 The projection field shown in the upper part can be an example of either (e.g., each) of the respective projection fields of pattern projection and non-pattern projection; and when the selected distance range represents the range of the interval [D2, ∞), the at least one selected projection type can represent the second projection type to allow the second sub-process including step S14B to be executed, wherein, as Figure 3 The projection field shown in the lower half can be considered an example of a projection field for non-pattern projection. For the sake of simplicity, similar details in these embodiments will not be repeated here.

[0108] According to some embodiments, the plurality of predetermined distance ranges may include a first range such as the interval [D1, D2) and a second range such as the interval [D2, ∞), but the invention is not limited thereto. Furthermore, when the selected distance range represents the interval [D1, D2), the at least one selected projection type may include the first projection type and the second projection type to allow the third sub-process including steps S15A, S15B, and S16 to be executed, wherein, as Figure 3 The projection field shown in the upper part can be an example of either (e.g., each) of the respective projection fields of pattern projection and non-pattern projection; and when the selected distance range represents the range of the interval [D2, ∞), the at least one selected projection type can represent the second projection type to allow the second sub-process including step S14B to be executed, wherein, as Figure 3 The projection field shown in the lower half can be used as an example of a projection field for non-pattern projection; in which the first sub-process can be omitted. For the sake of simplicity, similar content will not be repeated here in these embodiments.

[0109] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made in accordance with the claims of the present invention shall be within the scope of the present invention.

Claims

1. A method for hybrid depth detection using an adaptive projector, the method comprising: Distance information is obtained using image processing circuitry within a hybrid depth detection device, wherein the distance information indicates the distance between the hybrid depth detection device and one or more target objects; The image processing circuit uses the distance information to determine the distance range, wherein the distance range is selected from a plurality of predetermined distance ranges for that distance; The image processing circuit is used to select a projection type to determine at least one selected projection type from a plurality of predetermined projection types corresponding to the distance range, wherein the at least one selected projection type is selected from the plurality of predetermined projection types. The adaptive projector in the hybrid depth detection device is used to project at least one selected projection type to extract at least one corresponding image by the camera, and the image processing circuit is used to perform at least one type of depth detection corresponding to the at least one selected projection type based on the at least one corresponding image to generate at least one depth map. as well as In response to the at least one selected projection type, the image processing circuit selectively outputs the at least one depth map as a resultant depth map, or, if the at least one depth map includes multiple depth maps, combines the depth data based on the multiple depth maps to generate a combined depth map as the resultant depth map.

2. The method of claim 1, wherein the plurality of predetermined projection types includes a first projection type for pattern projection of one or more predetermined patterns, and includes a second projection type for non-pattern projection of no pattern.

3. The method of claim 2, wherein the plurality of depth detection types include a first depth detection corresponding to the first projection type and a second depth detection corresponding to the second projection type.

4. The method of claim 3, wherein the first depth detection represents pattern-detection-type depth detection and the second depth detection represents time-of-flight (TOF) type depth detection.

5. The method of claim 1, wherein the at least one selected projection type corresponding to the distance range includes the plurality of predetermined projection types, and the depth detection of the at least one type corresponding to the at least one selected projection type includes the plurality of types of depth detection.

6. The method of claim 5, wherein the plurality of predetermined projection types includes a first projection type for pattern projection and a second projection type for non-pattern projection; and the step of using the adaptive projector within the hybrid depth detection device to project the at least one selected projection type to extract the at least one corresponding image by the camera and using the image processing circuit to perform depth detection of the at least one type of depth detection corresponding to the at least one selected projection type in the plurality of depth detections to generate the at least one depth map further includes: The pattern is projected using the adaptive projector to extract at least one corresponding first image using the camera, and the image processing circuit performs a first depth detection based on the at least one corresponding first image to generate a first depth map; and The adaptive projector is used to perform non-pattern projection so that at least one corresponding second image is extracted by the camera, and the image processing circuit is used to perform a second depth detection based on the at least one corresponding second image to generate a second depth map. in, When the at least one selected projection type includes the plurality of predetermined projection types, the image processing circuit is used to combine the depth data based on the first depth map and the second depth map to generate the combined depth map as the resultant depth map.

7. The method of claim 1, wherein the step of selectively outputting the at least one depth map as the resulting depth map using the image processing circuit, or, when the at least one depth map comprises multiple depth maps, combining the depth data based on the multiple depth maps to generate the combined depth map as the resulting depth map, further comprises: If the at least one selected projection type represents one of the plurality of predetermined projection types, then the image processing circuit outputs the at least one depth map as the result depth map, wherein the at least one depth map represents a depth map corresponding to the one of the plurality of predetermined projection types; otherwise, the image processing circuit combines the depth data based on the plurality of depth maps to generate the combined depth map as the result depth map, wherein the plurality of depth maps includes a first depth map corresponding to a first projection type among the plurality of predetermined projection types, and includes a second depth map corresponding to a second projection type among the plurality of predetermined projection types.

8. The method of claim 1, wherein the information source of the distance information includes one or a combination of a distance sensor within the hybrid depth detection device and a previous depth detection operation of the hybrid depth detection device.

9. The adaptive projector operated according to the method of claim 1.

10. The image processing circuit operated according to the method of claim 1.

11. A hybrid depth detection device, comprising: An image processing circuit is used to obtain distance information, determine a distance range based on the distance information, and perform projection type selection to determine at least one selected projection type from a plurality of predetermined projection types corresponding to the distance range, wherein the distance information indicates the distance between the hybrid depth detection device and one or more target objects, the distance range is selected from a plurality of predetermined distance ranges for the distance, and the at least one selected projection type is selected from the plurality of predetermined projection types; An adaptive projector, coupled to the image processing circuit, is used to project the at least one selected projection type, allowing the hybrid depth detection device to extract at least one corresponding image from the camera. as well as The camera is coupled to the image processing circuit and is used to extract the at least one corresponding image; in: The image processing circuit performs at least one type of depth detection corresponding to the at least one selected projection type among multiple types of depth detection based on the at least one corresponding image to generate at least one depth map. as well as In response to the at least one selected projection type, the image processing circuit selectively outputs the at least one depth map as a resultant depth map, or, if the at least one depth map includes multiple depth maps, combines the depth data based on the multiple depth maps to generate a combined depth map as the resultant depth map.

Images