Apparatus and method for recording projected dot pattern in display device

Abstract

An apparatus comprises a plurality of imaging optical channels tilted relative to each other, each of the imaging optical channels comprising an optical device and at least one detector pixel arrangement. The plurality of optical channels is designed to obtain an image of a point pattern from a field of view by imaging the field of view. The apparatus comprises a display device having a plurality of display pixels arranged in a display plane, the display pixels being designed to display graphical information. The optical channels pass through the display plane and are designed to image the field of view between the display pixels. The apparatus comprises an evaluation device coupled to the detector pixels of the detector pixel arrangements and designed to evaluate the point pattern based on a comparison of signal strengths of different detector pixels (which leads to super-resolution) to obtain an evaluation result, and to control the apparatus at least partially based on the evaluation result.

Description

Technical Field

[0001] This invention relates to an apparatus and a method for capturing projected dot patterns, particularly a method combined with a display device for reproducing graphic information. The apparatus for capturing projected dot patterns can be used, for example, in apparatuses for depth map production and associated face recognition. Background Technology

[0002] For example, in the terminology of face ID or face recognition, known techniques for establishing reliable depth maps for face detection within consumer areas, besides time-of-flight (ToF) methods, rely on methods such as structured illumination or active stereoscopic imaging. For this purpose, typically within the NIR wavelength range, a preferred irregular dot pattern is projected toward the face, and the dot pattern projected onto the scene (face) is recaptured using one or two imaging devices. The shape / deformation / distance between the components of the scene affects the position of each light spot in / on the imager, producing specific deformations in the captured dot pattern. This information can be used to reconstruct the depth map / shape of the observed object / scene.

[0003] A compact device and corresponding method for capturing projected dot patterns, which needs to be combined with a display device. Summary of the Invention

[0004] Therefore, the object of the present invention is to provide a compact device for capturing a projected dot pattern and a method for achieving this object.

[0005] This objective is achieved through the subject matter of the independent claims.

[0006] The core concept of this invention lies in the discovery that a detector pixel arrangement for sensing a field of view and / or for obtaining an image of a dot pattern of the field of view can be combined with a display device such that the detector pixel arrangement faces the same direction and is also illuminated by the display device for depicting graphic information. Therefore, the detector pixels can bypass, for example, the display pixels of the display device. This makes it possible to integrate the arrangement of means for capturing the projected dot pattern with the arrangement of means for displaying graphic information, so that the functionality of the already implemented means for capturing the projected dot pattern can also be used in a corresponding display device, such as a consumer area, and a very compact device can be obtained. The objective of this invention is to combine / integrate a compact means for capturing the projected dot pattern with a display device for displaying graphic information.

[0007] According to an embodiment, an apparatus includes a plurality of imaging optical channels tilted relative to each other, each of the imaging optical channels including optics and at least one detector pixel arrangement. The plurality of optical channels are configured to obtain an image of a dot pattern of a field of view by imaging the field of view. The display device of the apparatus includes a large number of display pixels arranged in a display plane, the display pixels being configured to display graphic information. The optical channels pass through the display plane and are configured to image (project) the field of view between the display pixels. An evaluation device of the apparatus is coupled to the detector pixels of the detector pixel arrangement and is configured to evaluate the dot pattern based on a comparison of the signal strength of different detector pixels that would cause super-resolution to obtain an evaluation result, to control the apparatus at least in part based on the evaluation result. This makes it possible to integrate the display device with the detector pixel arrangement provided for sensing the projected dot pattern, resulting in a compact apparatus.

[0008] According to one embodiment, the device includes a projector configured to emit a known dot pattern. This, in turn, enables the capture of dot patterns deformed by a scene / face using an image capture device. The projector and the image capture device are tuned to each other in terms of their parameters to obtain an overall system with the smallest possible length.

[0009] According to an embodiment, the projector is configured to emit a dot pattern using an infrared spectrum. The optical channels are adapted to the infrared spectrum and / or selectively sense the infrared spectrum, for example, by using corresponding filters or materials for the detector pixel arrangement. This enables the projection and / or processing of dot patterns outside the wavelength range visible to the human eye.

[0010] According to an embodiment, the projector is configured to emit a dot pattern, which is at least partially aperiodic or pseudo-random. This allows for robust evaluation of the projected dot pattern while avoiding ambiguity.

[0011] According to one embodiment, the projector is configured to emit a dot pattern having at least 5,000 points. This achieves good lateral resolution of the depth map for the field of view.

[0012] According to one embodiment, the projector and optical channels are arranged to match each other to emit and capture a dot pattern at a reference distance from the device in the field of view, such that each detector pixel senses at most one point in the dot pattern at a time. This enables a meaningful comparison of the light intensity of adjacent detector pixels.

[0013] According to one embodiment, the projector includes a surface emitter (VCSEL) array and optics including diffractive elements. For example, this enables a compact design of the projector and any arrangement of optics such as Fresnel lenses or additional beamformers (e.g., lenses).

[0014] According to an embodiment, the evaluation device is configured to obtain evaluation results based on a comparison between an image and a reference pattern. This enables accurate determination of deviations within the field of view, and thus, the determination of a depth map.

[0015] According to one embodiment, the evaluation device is configured to perform a comparison of the signal strengths of different detector pixels to determine the positions of points in a dot pattern, which is more accurate than positions that would be achieved by sampling the entire field of view using detector pixels. Therefore, a precise depth map can be generated based on precise point localization while ignoring the regions between individual points.

[0016] According to one embodiment, the evaluation device is configured to determine the location of points in an image based on an evaluation of light intensity and simultaneously considering the response function of the detector pixels, where the same point in a dot pattern is projected onto adjacent detector pixels at that light intensity. Based on a relative comparison of light intensities, precise location of the light points can thus be obtained, and accurate results can be achieved due to the relatively small number of detector pixels, the short focal length of the optics, and consequently, the small installation space.

[0017] According to an embodiment, the evaluation device is configured to determine deformations in a dot pattern and generate a depth map of the field of view based on the deformations, and obtain an evaluation result based on the depth map. Therefore, a relatively low-computation depth map can be obtained based on the deformations, which is particularly advantageous for mobile devices, where high-sampling or high-resolution optics and / or image sensors are not required.

[0018] According to an embodiment, the device is configured to determine user authorization for a user at least partially located within the field of view based on evaluation results, and to control the device in accordance with that user authorization. This enables access authorization to be determined in a manner convenient for the user, for example, by eliminating the need to enter a password or PIN. For instance, actively evaluated access authorization requires activation of the display device.

[0019] According to an embodiment, the microlenses of the optical devices and the display pixels of the optical channel are arranged in a common plane area. This generally achieves a space-saving arrangement of the display device and the capturing device for projecting the dot pattern, and in particular, it makes it possible to avoid excessive length of the optical channel of the optical devices on the one hand, and to hide the display pixels by means of the optical devices on the other hand.

[0020] According to an embodiment, the optics are arranged between adjacent display pixels. This achieves a particularly small overall lateral expansion and also results in a particularly small mounting height and / or length of the optical channel for a fixed number of apertures of the optics.

[0021] According to an embodiment, the optical channels are arranged in a first two-dimensional arrangement, and the display pixels are arranged in a second two-dimensional arrangement. This achieves a planar configuration of the display pixels and detector pixels. With at least partial overlap between the two two-dimensional arrangements, a high integration density can be further achieved.

[0022] According to an embodiment, the evaluation device is configured to evaluate structured light or for active stereo evaluation. This enables the use of robust methods or algorithms to determine the depth map.

[0023] According to an embodiment, at least one of the detector pixel arrangements includes a plurality of detector pixels. The sensing areas of the plurality of detector pixels overlap by at least 50%. This enables precise comparison of the signal strength of different (particularly adjacent) detector pixels as responses to the same captured light spot.

[0024] According to an embodiment, the detector pixel arrangement is positioned within the image plane of the optics. This enables focused projection of the light spot onto the detector pixel arrangement.

[0025] According to an embodiment, the evaluation device is configured to evaluate the field of view through multiple detector pixels in each optical channel, and subsequently evaluate the field of view relative to the multiple optical channels. This enables accurate evaluation of a portion of the field of view sensed by the optical channels through multiple detector pixels and based on a combination of information from several optical channels that take into account the entire field of view.

[0026] According to an embodiment, the total focal length of the optical channels is less than 1 mm. This achieves a small installation size for the device.

[0027] According to an embodiment, an optical device comprising lenses or lens stacks is formed. This allows the optical device to be adapted to the corresponding requirements of the optical channel.

[0028] According to the embodiments, the microlenses of different optical channels exhibit channel-specific optical properties. This allows the optics to be adapted to the channel-specific relative positions of the optical channels in the device.

[0029] According to one embodiment, in this case, the device is implemented such that the microlenses are formed in a channel-specific manner, such that a (partial) field of view is obtained in the tilted line of sight associated with the corresponding optical channel, resulting in focused projection onto the detector pixel arrangement. For this purpose, for example, morphing lenses comprising non-rotatable symmetric aspherical surfaces or freeform shapes are suitable. This achieves a particularly compact device with a small number of lens layers per optic.

[0030] According to one embodiment, an apparatus includes at least one aperture structure disposed between a detector pixel arrangement and an optical plane to provide stray light suppression for adjacent optical channels. This enables comparisons to be performed with high accuracy and without being affected by dual images.

[0031] According to one embodiment, the device includes at least one stray light suppression structure disposed between adjacent detector pixel arrangements and between their planes and the plane of the optics to provide stray light suppression for adjacent optical channels. This stray light suppression also enables comparisons to be performed with high accuracy, and, for example, when optics in different optical channels are connected to each other via a shared substrate and when the stray light suppression structure is connected to that substrate, it further enables the stray light suppression structure to be used as a spacer in a cooperative manner. And independently of this, the optics can be spaced apart by the stray light suppression structure, which can be understood as, for example, a vertical or inclined wall of the optical channel. Compared to previously presented aperture layers, in this approach, a gap is obtained between the detector pixels and the optics because the stray light suppression by the aperture structure provides a corresponding fixing device.

[0032] According to an embodiment, the device is configured for facial recognition of a user, particularly depth map-based facial recognition. This enables convenient operation of the device.

[0033] According to an embodiment, the device is configured as a smartwatch. The short optical channel implemented in this embodiment particularly collaborates in a cooperative manner with a small device having display functionality (especially the display in a smartwatch).

[0034] According to an embodiment, a method for providing the apparatus described herein is provided.

[0035] Other advantageous embodiments form the subject matter of the other dependent claims. Attached Figure Description

[0036] Other advantageous embodiments will be explained below with reference to the accompanying figures, in which:

[0037] Figure 1 A schematic cross-sectional side view of the device according to an embodiment is shown;

[0038] Figure 2 A schematic cross-sectional side view of a device including a projector according to an embodiment is shown;

[0039] Figure 3a exhibit Figure 1 and / or Figure 2 A schematic cross-sectional side view of a portion of the device;

[0040] Figure 3b exhibit Figure 3a A schematic cross-sectional side view of the optical channel is provided to further illustrate the resulting effect;

[0041] Figure 3c exhibit Figure 3b A more detailed representation of the optical channel, in which the spatial extension of the detector pixels is also taken into account;

[0042] Figure 4a A schematic cross-sectional side view showing the path (progression) of the response function along the x-direction according to an embodiment;

[0043] Figure 4b A schematic representation of two response functions is shown, where the corresponding regions are located within the pixel field of view of another detector pixel;

[0044] Figure 4c Demonstration according to embodiments Figure 4b A schematic representation of the response function, configured to precisely comply with the resolution limit;

[0045] Figure 4d A schematic representation of the response function is shown, where the response function is separable;

[0046] Figure 5 A schematic representation of the focal length problem produced by classical resolution;

[0047] Figure 6a Showing the correspondence with respect to the response function Figure 5 The configuration is an exemplary configuration, but it cannot be distinguished by classical methods; however, it can accurately locate the light spots of the dot pattern according to the embodiments.

[0048] Figure 6b Demonstrates a two-dimensional arrangement of detector pixels according to an embodiment. Figure 6a A schematic representation of the concept;

[0049] Figure 7 A schematic cross-sectional side view of a portion of a device according to an embodiment is shown, the device including a stray light suppression structure between optical channels;

[0050] Figure 8 A schematic one-dimensional representation showing the projected light spot and the necessary high sampling required for fully accurate localization of the light spot within the field of view, performed by detector pixels using classical methods;

[0051] Figure 9 Compared to Figure 8 This illustrates a schematic representation of detector pixels evaluated according to the present invention, based on an embodiment. Detailed Implementation

[0052] Before the embodiments of the present invention are explained in more detail with the aid of the accompanying drawings, it should be noted that the same, functionally identical, or action-identical components, objects, and / or structures have the same reference numerals in different figures, so that the descriptions of the components provided in different embodiments are interchangeable and / or mutually applicable.

[0053] The embodiments described below will be presented in conjunction with numerous details. However, embodiments may also be implemented with detailed features. Furthermore, for ease of understanding, embodiments will be described using block diagrams instead of detailed descriptions. Additionally, the details and / or features of the various embodiments can be readily combined with each other, provided that no explicit description to the contrary exists.

[0054] Figure 1 A schematic cross-sectional side view of a device 10 according to an embodiment is shown. The device 10 includes a plurality of optical channels 121 to 125 inclined relative to each other. Five optical channels 121 to 125 are depicted by way of example; however, it is also possible to arrange any other, particularly a larger number of, optical channels, such as at least 2, at least 4, at least 6, at least 9, at least 10, at least 20, or more. In the depicted... Figure 1 In the cross-sectional side view, the optical channels are shown in a one-dimensional array; however, embodiments also provide a device in which the optical channels 121 to 125 are arranged in a two-dimensional arrangement (e.g., a rectangular or square arrangement).

[0055] Each optical channel includes optics 141 to 145 and at least one detector pixel arrangement 161 to 165. Additional components, such as filters or the like, may also be provided. Optics 141 to 145 may also be configured to be the same or different from each other and adaptable to, for example, corresponding lines of sight. Furthermore, each of optics 141 to 145 may include a lens, as well as lens stacks, and / or additional optically effective components.

[0056] Detector pixel arrangements 161 to 165 may include the same or different numbers of detector pixels 18. Each detector pixel arrangement 16 i Includes at least one detector pixel 18 j , where i = 1...5. In Figure 1 In the depicted view, the detector pixels are arranged in a 16-pixel configuration. iThe arrangement is shown to each include three detector pixels arranged close to each other in a row. However, as described in conjunction with the arrangement of the optical channels, the detector pixel arrangement 16 may also include a two-dimensional arrangement of the detector pixels. The two-dimensional arrangement may be random, for example, arranged in a rectangle or square, and may include, for example, two, six, eight, or nine or any other number of detector pixels, especially a larger number. The detector pixel arrangement 18 may be arranged in a corresponding image plane of the optics 14, but preferably in a shared image plane. The image plane may be a plane in which the field of view and, in particular, the light spot projected therein are imaged by the optics in a focused manner.

[0057] Optical channels 121 to 125 are configured to obtain or provide an image of a dot pattern 24 of the field of view 22 by imaging the field of view 22. The dot pattern 24 may include a distribution, for example, dots 261 to 266 arranged in a pattern that is at least partially aperiodic or pseudo-random. N For example, in the context, the total number N of dots 26 of dot pattern 24 is, for example, at least 5,000, at least 7,000, or at least 10,000, such as in a 100×100 arrangement.

[0058] The apparatus 10 further includes a display device 28, which includes a plurality of display pixels 341 to 344 arranged in a display plane and configured to display graphic information. The display device 28 may be a display or the like, which includes a plurality of pixels, and may also be pixels of different colors. For example, the display pixels may include organic light-emitting diodes (OLEDs), thus being pixels or image elements of an OLED display. In its lateral extension, the pixels may be implemented to be small relative to the center-to-center spacing.

[0059] Optical channels 121 to 125 pass through display plane 32 and are configured for imaging (projecting) a field of view 22 between display pixels 341 to 344, meaning the optical channels bypass display pixels 34. The plurality of display pixels 341 to 344 and their configuration can be arbitrary, and may include, in particular, several 100×100, several 1000×1000, or more. In this context, the 2D configuration of the display pixels is, for example, square; however, it is also possible to choose different geometries, i.e., rectangular or shapes deviating from them. The number of pixels is also illustrative. By way of example, plane 32 is depicted as a plane parallel to the plane in which the detector pixel arrangement 16 is disposed. The implemented embodiments are not exclusively limited to, or not limited to, non-curved planes, but can also be implemented as curved planes 32, and particularly in planar regions within ±0.1 mm, ±0.8 mm, or ±0.5 mm or similar ranges. If display pixel 32 is placed, for example, at the apex of the lenses of optical devices 121 to 125, this should also be understood as placement within a shared plane region or within the display plane in conjunction with the embodiments described herein. If optical device 14 i Arranged in adjacent display pixels 34 j Between, such as Figure 1 The configuration described herein should also be understood as the configuration within a shared plane area.

[0060] The apparatus includes an evaluation device 36 coupled to detector pixels 18 and configured to evaluate an image of dot patterns and / or dot patterns 24 based on a comparison of signal strengths of different detector pixels 18 (achieving super-resolution) to obtain an evaluation result. The evaluation device 36 is configured to control the apparatus 10 at least in part based on the evaluation result. For example, the evaluation device 36 may be electrically and / or optically coupled to circuitry disposed on, on, or within a detector pixel substrate 38, which wholly or partially supports or carries detector pixel arrangements 161 to 165.

[0061] The detector pixel substrate 38 and the optics 141 to 145 and / or the display plane 32 may have a transparent material 42 disposed therebetween, which may serve as, for example, a substrate for the display pixels 34 and / or as a carrier or substrate for the optics. Optionally or additionally, one or more of the display pixels 34 may be disposed on or supported at the optics 14, for example because the optics 14 are configured to have a larger surface area than the display pixels and therefore the optics are only partially hidden by the display pixels. Optionally, a fluid such as a gas or a vacuum may be provided as the transparent material 42.

[0062] Optionally, substrate 44 can be used to carry or support substrate 38, transparent material 42 and / or other components.

[0063] Optionally, the device may include arrangements of detector pixels 16 i One or more aperture structures 461 and / or 462 are positioned between the plane of the optics 14 (also referred to as the optics plane). Alternatively, the aperture structures can be arranged such that the optics plane is positioned between the detector pixel arrangement 16 and one or more aperture structures 46; however, this reduces the illumination of the display pixel 34, which will be taken into account during readout. The aperture structures can be used to provide stray light suppression for adjacent optical channels. Optionally or additionally, the light illumination (emission) from the display pixel toward the detector pixel arrangement can be completely or partially blocked. However, this can remain optional, especially since the detector pixel 18 is sensitive in a wavelength range different from the wavelength range emitted by the display pixel 34. Aperture structures 461 and / or 462 can also be implemented to provide a so-called "pinhole structure" for the detector pixel arrangement 16. This achieves selective fields of view for the individual detector pixel arrangements. The lines of view 481 to 485 of the optical channels 121 to 125 can be tilted relative to each other, such that a field of view 22 is displayed; it is advantageous but not necessary for the field of view 22 to be a continuous field of view. However, the lines of sight 481 to 485 drawn by the main beam can be designed such that when the display pixel 34 is arranged in the path of the optical channels 121 to 125, the display pixel 34 can be bypassed, either because the field angle is large enough that the optical channels 121 to 125 are only partially obscured by the display pixel 34, and / or because the display pixel 34 is arranged separately from the optical channels.

[0064] In this embodiment, the total focal length of optical channels 121 to 125 is less than 1 mm, less than 0.8 mm, or less than 0.6 mm, thereby achieving a correspondingly shorter distance 52 between the detector pixel arrangement 16 and the optics 14. This can result in a smaller mounting height for the device 10, for example, when the distance 52 is configured to be parallel to the thickness direction or the mounting height direction.

[0065] Optical elements 14 of different optical channels 12, or individual microlenses derived therefrom, may include channel-specific optical properties, such as symmetry, geometry, or other properties, whether present or omitted. Microlenses may be formed such that focused imaging of the field of view 22 is obtained in a tilted line of sight associated with the optical channel. Based on the tilted line of sight 48, the problem arising therefor is that, for example, deformable lenses may include non-rotatable symmetric aspherical surface descriptions and / or free forms, especially for outer lenses. By means of channel-by-channel adaptation of the optics to their corresponding lines of sight, so-called off-axis aberrations, particularly field curvature and astigmatism, as well as coma and distortion, can be minimized. For example, if a point-symmetric array, such as a 3×3 or 5×5 array, is arranged, the central channel can readily exhibit rotationally symmetric lenses in one or more directions.

[0066] For the example, a distance 52 is drawn on one side of the optics 14 facing the detector pixel arrangement 16; however, this distance is merely illustrative. Unlike... Figure 1 In the case where each of the optical devices 121 to 125 includes its own lens, the optical devices 121 to 125 may also include a lens stack having a plurality of lenses stacked in a direction parallel to the distance 52.

[0067] Figure 2 A schematic cross-sectional side view of a device 20 according to an embodiment is shown. Device 20 is based on and includes, for example, device 10 or an embodiment thereof, and further includes a projector 54 configured to emit a dot pattern 24. The projector may be configured to emit, for example, a dot pattern 24 comprising an infrared spectrum, meaning a wavelength range that at least partially overlaps with or includes at least a portion of the infrared wavelength spectrum. Optical channels 121 to 124 may be adapted to the infrared spectrum and / or selectively sense the infrared spectrum in terms of focusing and sensitivity. Therefore, detector pixel arrangements 161 to 165 may be configured, for example, to be sensitive to the spectrum of the emitted dot pattern 24 due to their material selection. If detector pixel arrangement 16 is also sensitive to other wavelength ranges, a filter structure may be provided to attenuate or even filter out any unwanted or unintended wavelength ranges. While other wavelength ranges preferably invisible to the human eye may be used, the infrared spectrum is preferred in this embodiment. In the context, near-infrared light, comprising a wavelength range of at least 780 nm and at most 3 μm or a portion thereof, is particularly preferred. In particular, wavelengths of 850 nm and / or 940 nm are of interest because, in this case, an emitter will be established.

[0068] The projector 54 and optical channels 121-125 are configured to be tuned to each other such that at a reference distance 56 from the field of view to the device 20, the dot pattern 24 is emitted by the projector 54 and captured and / or sensed by the optical channels 121-125 in such a manner that each detector pixel 18 senses at most a single point 26 of the dot pattern 24 once. This may include setting the divergence of the individual points of the dot pattern and their mutual distances and / or setting the sampling of the field of view by the magnitude of the response functions of the detector pixels and the optical channels and / or the detector pixels. This significantly simplifies the evaluation of the evaluation device 36 because the maximum amplitude of the detector pixel 18 is triggered by a single point 26, allowing the comparison performed by the evaluation device 36 to be performed within a framework condition related to the degree to which the single point 26 introduces optical power to the detector pixel 18.

[0069] Projector 54 can be designed to emit a dot pattern that is at least partially aperiodic or pseudo-random, wherein the aperiodic or pseudo-random pattern can be selected to be large enough that repetitions in the pattern occur only in optical channels spaced apart from each other, which simplifies the evaluation of evaluation device 36. The entire dot pattern 24 can also be configured to be aperiodic or pseudo-random. The evaluation device can be set up for evaluating structured light and / or for active stereo evaluation.

[0070] Projector 54 may be designed to include a surface emitter (VCSEL) array and optics. The optics may include diffraction elements and may optionally include other beamforming optics and / or lenses.

[0071] For example, the reference distance 56 may be a distance used as the basis for calibration and, for example, a distance corresponding to the expected object distance, at least within permissible limits. As with device 10, device 20 may provide, for example, at least a portion of a display device, the access authorization of which is verified by evaluation device 36. Examples herein include electronic door locks including facial recognition, mobile phones including facial recognition, and / or smartwatches including facial recognition. The device may be tuned to a distance range within which the user's face is expected to be. The reference distance 56 may be within this range; any other implementation is also possible.

[0072] In other words, smartwatches typically feature OLED displays. Here, light emitters (display pixels) of (different colors) are located at a distance of less than 100 μm from each other. Smartwatches, even with high-powered batteries, are compact, especially thin. Smartwatches will be able to be unlocked by only specific individuals and their use will be restricted; for this purpose, facial ID or facial recognition methods are suitable. Similarly, for example, the NIR dot array emitter of projector 54 and the corresponding imaging device (i.e., including optics 14) iThe transparent substrate 42 and detector pixel arrangement 161 to 165 will be as compact as possible. The embodiments implement the concept of "imaging device below the OLED display," or the concept of placing the OLED display directly above or above the imaging device, and / or the concept of integrating or bending the OLED display of a smartwatch with the imaging device. For this purpose, the array optics 14 can be provided as the imaging device because the planar configuration of the OLED display pixels can be arranged in the middle region of each imaging channel, or possibly even directly above each lens. If the resolution and sampling of the array imaging optics will be at their maximum, then even the array imaging optics will require a very large mounting length and a very large lateral extension to capture a sufficiently dense point pattern for the depth map and to resolve the point pattern in classical scenarios (i.e., according to known resolution criteria such as Rayleigh, Sparrow, Abbe, etc.), which are based on separating the captured projected light spots on adjacent detector pixels from each other.

[0073] In contrast, the embodiments are designed based on a compound eye and / or multi-aperture imaging device model, while simultaneously using a microlens array with small to medium resolution to implement the optical channel 12 and thus a very small mounting length for the entire camera module. Such structures can be fabricated at the wafer scale and can be reminiscent of (computer) chips rather than classical objectives in their appearance. Therefore, on the one hand, they can be produced at low cost and pre-specified by their planar, integrated design for mounting in compact mobile terminals. The primary goal of this configuration of the multi-aperture imaging device has so far been to produce conventional images with resolution capabilities sufficient to accomplish the desired imaging task. The embodiments involve comparing light intensities while being aware of the so-called response function of the detector pixels, which will be explained in detail below.

[0074] The embodiment relates to an evaluation device 36 for determining and evaluating phenomena based on super-sharpness or super-resolution, which means highly accurately locating those point sources that are separated from each other by comparing the responses in adjacent detector pixels that simultaneously sense point sources with the channels of the integrated / woven / bent imaging system and the pixels of the display and / or image reproduction device. Furthermore, the embodiment makes it possible to add the explicit projection of a preferred non-isolated pattern of light spots that is sufficiently separated from each other (e.g., in near-infrared, also referred to as "NIR"), which enables the entire particular architecture, to be captured by a corresponding array of optics, which is further configured to be directly interwoven / integrated with the pixels of a flat panel display (this may be referred to as "OLED-back camera" or OLED-on-camera). Specifically, this occurs with the aid of ultra-sharp methods to effectively adapt the array imaging system to its imaging task, thereby reducing the number of detector pixels and optical channels, and particularly, reducing the mounting length of the optics to a smaller, possibly even minimal, value, so that it can be integrated into, for example, compact-driven applications, such as mobile devices, especially smartwatches, which may be based on facial recognition with depth map features, for example, by means of structured light or active stereo imaging. This means that the device described herein can be configured for user facial recognition, particularly depth map-based facial recognition.

[0075] Array imaging devices can be configured, for example, to have a number of pixels per imaging channel / lens, such as... Figure 1 and Figure 2 As depicted in [the document]. However, alternatively, it is possible to provide only one detector pixel per channel / lens, compared to this preferred embodiment, although this would result in a significantly larger lateral spread.

[0076] The individual lenses or lens groups and / or lens stacks in each optical channel may differ between channels in nature to optimally correct or adapt to the corresponding main line of sight. The optical axes of the channels are tilted differently relative to each other to sample the entire field of view as a whole. Another advantage of this arrangement is that, since the field of view is split into many optical channels, it is not necessary to optimize an optics for the entire field of view (which would increase its complexity); each channel only needs to be able to handle a clearly limited field of view of a few degrees. The optomechanical implementation of the array imaging device can be diverse. For example, what is described herein could be a so-called monolithic architecture, where the optics are connected to the detector pixels without any air gaps, but via a transparent material (“substrate”), preferably also having a medium-aperture plane or channel insulating walls. Alternatively, the optics may be arranged on the substrate and possibly inverted relative to the detector pixel arrangement and connected to the detector pixel arrangement via a grid-like spacer and / or carrier structure including optical / channel insulation, possibly tilted walls that prevent crosstalk.

[0077] In a preferred embodiment, the apparatus is configured to capture a point pattern in which the points are sufficiently small, i.e., having an angular spread (divergence) of approximately 0.1° to 0.3°, spaced apart from each other at an average distance of 0.5° to 3°, such that the classical resolution of the camera is lower than, or even significantly lower than, the required accuracy for locating the light points. Therefore, for positioning, the focal length and final mounting length can be significantly reduced if the responses of adjacent channels to the same light point are compared, while utilizing an understanding of the path of the channel response functions. In other words, the accuracy of locating the light source is significantly higher than the actual resolving power of the imaging apparatus's optical channels. The highly accurate positioning of the projected light points in the scene can be used for depth map reconstruction and can thus at least partially replace the resolution of conventional images.

[0078] Figure 3a A schematic cross-sectional side view of a portion of the display device 10 and / or 20, including, for example, segments of optical channels 122 and 123, which include main lines of sight 482 and 483. Each optical channel 122 and / or 123 may span its own (partial) field of view 582 and / or 583. The size of the partial fields of view 582 and 583 may be at least partially influenced by the size of the associated image sensor region, i.e., the size and / or configuration of detector pixels 18 in detector pixel arrangements 162 and / or 163, and the focal length of optics 142 and / or 143. Lines of sight 482 and 483 are slightly angled between the channels of the device to fully span the field of sight. Figure 1 The full field of view 22. This can be achieved, for example, by offset between the center of the optics and the center of the associated pixel group / detector pixel arrangement and / or by a specific lens shape. Partial fields of view of the device (which may be referred to as channel fields of view) may be adjacent to each other at the target object distance or may overlap each other to a small or even large extent.

[0079] Figure 3b exhibit Figure 3a Another schematic cross-sectional side view of the optical channel 122 is provided to further illustrate the resulting effect. Based on the spatial distances of detector pixels 184, 185, and 186 and the spatial distances of the shared optics 142, which can therefore provide, for example, a shared optics center 62, detector pixels 184 to 186 may include slightly different lines of sight 644, 645, and 646. In other words, each pixel within the channel faces a slightly different direction, and detector pixels 184 to 186 are able to completely span the channel's field of view 582.

[0080] Figure 3c exhibit Figure 3bA more detailed representation of the optical channel 122, wherein the spatial extension of detector pixels 184, 185, and 186 is also considered. For example, spatial extension in the x / y plane perpendicular to the distance direction 52 may result in each pixel 184 to 186 spanning its associated pixel field of view 664, 665, and 666; diffraction effects of the optics 142 are also considered for this purpose. The pixel field of view 664 to 666 may describe or provide the response function of the corresponding pixel 184, 185, or 186. The angular range on the object side associated with signal generation of the detector pixel can be defined as the response function.

[0081] Unlike Figure 3c In the descriptions, response functions typically overlap spatially and limit the effective resolving power of the imaging device by their respective sizes, even though sampling is much finer due to the use of many pixels and channels. The sensing areas of detector pixels may overlap to at least 50%, at least 70%, or at least 80%, especially relative to detector pixels with a shared detector pixel arrangement.

[0082] Regardless of any diffraction effects not described here, the response function can be reduced by decreasing the pixel size (which is detrimental to sensitivity) or by increasing the focal length and thus the mounting length (which is detrimental to compactness). The embodiment resolves this conflict by comparing the signal strengths of different detector pixels in a way that induces super-resolution and by understanding the path of their response functions. This comparison enables localization, allowing the information needed for depth map reconstruction to be obtained even if the image is not fully resolved (resolved) or sampled.

[0083] Figure 4a A schematic cross-sectional side view showing the path of the response function along the x-direction is provided. It should be noted that the response function can also be determined in the y-direction because the point response (point-like response) of the detector pixel and optics, resulting from diffraction effects and aberrations, has a two-dimensional area range. The detector pixel may include a variable signal amplitude or sensitivity 68 spanning a pixel field of view 66, which includes the maximum value in the region of maximum sensitivity 72 (e.g., the average range). A slight signal amplitude 68 may still be sensed outside the field of view 66 to be evaluated, but it may not be considered within the scope of reflections occurring there. The response function can therefore provide an amplitude value decreasing toward the margin as a function of the distance to the light spot from the optical axis of the corresponding detector pixel (while the brightness of the light spot remains the same), which can be taken into account in subsequent evaluations based on an understanding of the response function. The amplitude can represent an evaluation of light intensity; that is, the same brightness object at different locations will be evaluated differently by the detector pixel.

[0084] Figure 4bA schematic representation of two response functions 661 and 662 is shown, wherein the corresponding regions 721 and / or 722 are respectively arranged within the pixel fields of view 662 and 661 of another detector pixel. This results in the response functions being unable to be classically separated from each other or being unable to be evaluated independently for the extended object under consideration. The amplitude attenuation 74 is too small for reliable differentiation and totals, for example, only 10%, 15%, or less than 20% of the maximum amplitude value.

[0085] Figure 4c The schematic representations of response functions 661 and 662 are shown, which are configured such that they will comply with typical resolution limits, for example, regions 721 and 722 are located precisely on the marginal regions of effective response functions 661 and 662 in each case.

[0086] Figure 4d The configuration in which the response functions can be separated according to known analytical criteria such as Abbe or Reilly is depicted. Regions 721 and 722 lie outside the corresponding other response functions 661 and 662, respectively.

[0087] Figure 5 A schematic representation of the focal length problem with classical resolution is shown. Dot pattern 24 comprises numerous points 26 and is imaged by means of an optical channel, such that points 26'1 to 26'4 imaged in an image are shown as examples. The configuration of the light spots and / or focal length variations is depicted in lines A, B, and C. For simplicity, the light spots are shown in 1D in an isometric manner.

[0088] Line A illustrates a typical sample where only variations in grayscale are evaluated where the dot pattern and / or its image (photograph) are indistinguishable and / or where the location of the light spot may not be accurately pinpointed. This is because all adjacent detector pixels consistently process light spots 26'1 to 26'4, which are located in a correspondingly dense manner within this cluster map. For example, the response functions 661 and 662 for point 26'1 imply that both pixels provide correspondingly high grayscale values. Response functions 663 and 664, for instance, provide signals based on light spot 262, etc. By way of example, this is why measurable modulation of grayscale between detector pixels cannot be determined, and therefore the location of the light spot cannot be determined in a classical manner.

[0089] Line B depicts a low dot density, meaning dot pattern 24 is correspondingly adapted with a lower density, which resolves the contradiction of line A; however, on the one hand, the possible resolution of the depth map itself has been reduced by the reduced density of the dot pattern, and on the other hand, precise height localization of the dots remains impossible because the resulting dots for the same signal may be located in slightly different positions. For example, dot 261 in response functions 661 and 662 (and / or their associated detector pixels) triggers a signal in each case, response functions 664 and 665 correspond to dot 262, while response function 663, for example, does not provide a signal, meaning that dots 261 and 262 can be roughly separated from each other to the extent of alternating high and low gray levels. That is, one may indicate that some rough areas contain the dot, while other areas do not. Based on the understanding of the varying gray values, even in the case of low sampling, i.e., when the number of samples roughly corresponds to the order of magnitude of the dots to be captured, it is still impossible to determine the precise location of each dot within the pattern.

[0090] Line C uses an example to depict an increase in focal length, which causes an increase in mounting length. However, this achieves a reduction in the spatial expansion of the response function 661 to 668, and thus achieves the resolution of the pattern of line A (similar to the method in line B for a lower density dot pattern: the intermediate detector pixels do not provide a signal) and also allows for the configuration of several pixels on the same surface area, but will violate the requirement of a small mounting length.

[0091] None of the aforementioned variants achieves a compact device, requiring high-density sampling of the projected light pattern, the point density of which is generated by the assumed lateral and depth resolution of the depth map and is typically high. For this purpose, the field of view is sampled using numerous response functions (of adjacent detector pixels and subsequently, detector pixels of adjacent optical channels). By way of example, the depicted circle corresponds to an analytical criterion, such as 1 / e of the response function. 2 The length or the zero of the amplified AIRY function affected by the detector.

[0092] The embodiment includes an evaluation device coupled to detector pixels arranged in a detector pixel arrangement and configured to evaluate a point pattern based on super-resolution comparisons of the signal strengths of different detector pixels and based on an understanding of the path of the response function, in order to obtain an evaluation result. For this purpose, the detector pixels are arranged such that the sensing regions and response functions of the plurality of detector pixels overlap to at least 50%.

[0093] Figure 6a Showing the correspondence between the response function and the response function Figure 5While this exemplary configuration of configuration A does not, of course, achieve the extended resolution of the object using typical methods, it enables the relatively precise positioning of the separated light spots by means of the described method. Thus, points 26'1 to 26'4 can be sensed in each case, for example, by two detector pixels. Considering the overlapping response functions 661 to 668, these points can respectively result in two amplitude values ​​681 and 682, 683 and 684, 685 and 686, and 687 and 688, especially when it is assumed that each detector pixel senses only one point. Therefore, considering the response functions of the detector pixels, the relationship of the amplitude value 68 can lead to a precise result regarding the position of the light spot along the x-direction, the response functions of which are previously determined or otherwise (during the calibration step), even if the image itself cannot be fully discerned. Thus, the positions x1 to x4 of points 26'1 to 26'4 can be located with high precision relative to the known center of the response functions of each pixel and, given the understanding of their positions throughout the system.

[0094] The details for determining and using the response function will be described below. The response function is determined by scanning a point source across the field of view of each pixel, and the path of the response (which represents the signal) is plotted relative to the angle and distance of the pixel's optical axis. This function is necessary to determine the relative position of adjacent pixels to the optical axis of the detector pixel in question, based on their relative responses to the same point source, especially since the relative positions of the detector pixels are also known. For super-sharpness or super-resolution, overlapping response functions are advantageous, provided they are not large enough to cause several points of light to trigger the response simultaneously. This can also be related to the setting of the sampling angle.

[0095] Figure 6b A two-dimensional portion is shown in a two-dimensional arrangement of detector pixels. Figure 6a The content of this two-dimensional section is closer to the embodiment. For example, point 26 is projected onto three detector pixels such that the two-dimensional coordinates of the projected point 26' can be determined from the amplitude values ​​681, 682, and 683 of three two-dimensional distributions, respectively, by forming pairs 681 and 682 and 682 and 683. The coordinates x1 / y1 can be determined from distances 761, 762, 763, and 764 and / or from the difference between amplitude values ​​681 to 684 and the maximum amplitude. Sampling can be selected such that statistically no more than one light spot belongs to the response function of the pixel at a time, in order to avoid ambiguity.

[0096] In other words, the embodiment is about obtaining precise positioning of light spots based on the knowledge of the existence of mutually separated projected light spots and the knowledge of the accurate path of the individual response functions of pixels, which is obtained by specific comparison of the measured signals relative to the axis of adjacent pixels.

[0097] In other words, combining Figure 6a The description also applies to 2D, such as x / y coordinates and subpixel accuracy. For this purpose, the system can be entirely configured by a tuned design of the magnitudes of the response functions, their distances (sampling), and the density of the projected light spots, such that (at least based on statistical averages) no more than one light spot lies within a group of the three presented response functions at any given time, because otherwise inaccuracies and / or ambiguities could arise. Even if ambiguity occurs, it can be resolved computationally using corresponding overhead. Computational overhead can be saved by avoiding ambiguity.

[0098] The evaluation apparatus described herein can be designed to obtain the determined positions of light spots spanning several optical channels from which a pattern can be derived. This pattern can be compared with a reference pattern by the evaluation apparatus to determine deformations in the pattern from which a depth map can be derived. A well-defined depth map can be produced when the geometry between the field of view (possibly the projector) and the configuration of the optical channels is taken into account. Comparison of the signal strengths of different detector pixels can be performed by the evaluation apparatus, such that the positions of the points in the dot pattern can be determined more precisely by means of detector pixels than by sampling of the entire field of view, which is described herein as classical sampling or resolution (discrimination) of the image. Comparing signal values ​​enables the resolution of positions with sub-pixel accuracy. Evaluation based on the light intensity (signal amplitude 68) of adjacent detector pixels, while simultaneously considering the response functions of the detector pixels, can thus determine the positions of points in the image.

[0099] Therefore, the evaluation device can determine the deformation in the dot pattern and generate a depth map of the field of view based on the deformation. Based on the depth map, an evaluation result can be obtained. In short, the depth map provides information about whether an authorized user has been identified. If the device is, for example, a smartwatch, it can be unlocked, or information associated with the user can be presented via a display device, particularly those described herein, or different operating modes can be set depending on the identified user. According to an embodiment, user authorization for a user at least partially located within the field of view can be determined based on the evaluation result, and control can be performed via the evaluation device based on this authorization.

[0100] The evaluation device can be configured to evaluate the field of view 22 through multiple detector pixels in each optical channel, and subsequently evaluate the field of view 22 relative to several optical channels. This means that the positions of the light spots established in the respective optical channels can be combined to form an overall pattern.

[0101] In other words, the magnitude of the angular response function (also known as the point image response) of an imaging device depends on the diffraction limit (determined by wavelength and aperture and / or lens size) and the size of the detector pixels relative to the focal length of the optics. For extremely short optics (focal lengths of a few 100 μm), the second aspect can play a relatively dominant role; however, it is often neglected for typical micro-objects. For example, if approximately 10,000 tiny light spots distributed across a typical diagonal field of view of 70° to 90° were to be distinguished / separated in a conventional manner and imaged by detector pixels to a size of approximately 3 μm to 5 μm, this would also result in an array of optics with a mounting length significantly greater than 1 mm, which could prevent integration into mobile devices such as smartwatches or the like.

[0102] However, due to the known nature of the patterns in which the positions of many light sources to be determined with high precision lie—because, in practice, when the design of the optics is adapted, the light spots are sufficiently separated from each other, statistically there will not be several light spots at any given time within the response region of a detector pixel, but a light spot can simultaneously lie within the response regions of several adjacent detector pixels—without needing to discern the dot pattern according to the Rayleigh, Spylow, and / or Abbe criteria, for this purpose, the response functions must be significantly closer, and therefore the focal length, and thus the mounting length, will need to be significantly larger than the values ​​applicable to the envisioned application. In contrast, this embodiment utilizes largely overlapping response functions, and the number of response functions sampled from the field of view is chosen to be sufficiently large to ensure that there will not be several light spots simultaneously present in the response functions (at least based on statistical averages), and the relative signals of adjacent response functions at the same light spot are used to determine the precise position of this light spot and, in the same manner, to determine the precise positions of all other projected light spots within the field of view with high precision.

[0103] A substrate having a lens or multiple aperture layers thereon to prevent crosstalk between channels can also be provided as a wall that can be used to insulate the optical channels, and is at least partially opaque and may be tilted.

[0104] Figure 7 A schematic cross-sectional side view of a portion of the display device 70 is shown, which is modified from devices 10 or 20 to include stray light suppression structures 781, 782, and 783, as alternatives to the chosen apertures 461 and 462 in devices 10 and 20. In each case, stray light suppression structures 781, 782, and 783 are arranged between adjacent detector pixel arrangements 182 to 183 and / or other detector pixel arrangements. Although stray light suppression structures 781, 782, and 783 are depicted as having the same thickness and tilted relative to the main substrate side, this allows for adaptation to the optical path of the optical channel. However, it is also possible, alternatively, to provide variable thickness and / or a vertical configuration towards the main substrate side without necessarily sacrificing these advantages.

[0105] exist Figure 7 In the representation, transparent material 42 is disposed between display pixel 34 and detector pixel substrate 38. In some embodiments, distance 52 may be set wholly or partially by stray light suppression structures 781, 782 and / or 783, meaning that transparent material 42 may also be non-solid, such as liquid or gaseous materials, such as air. Stray light suppression structures 781, 782 and 783 may be individual structures, but may also be provided as shared and interconnected structures implied as a grid structure in top view. For example, optics 14 with different optical channels to be connected to each other via a shared substrate and stray light suppression structures to be connected to the substrate and, for example, to the detector pixel substrate 38 may be defined. Alternatively, optics may also be placed and thus fixed in the gaps between stray light suppression structures. In this case, a gap between the lens and the image sensor may also be obtained. For example, transparent material 42 may be configured to be relatively thin and spaced apart from substrate 38 by stray light suppression structures.

[0106] In one exemplary embodiment, the projected pattern displays, for example, 10,000 dots, presented in a simplified manner within a 100×100 pattern spanning the field of view (FOV), resulting in 100 dots representing one dimension. It should be noted that regardless of the regularity of the pattern's representation, this pattern is typically irregular. In typical imaging methods of conventional cameras or even conventional array cameras (which differ from the embodiments described herein), one would proceed to use an image sensor of approximately 1 megapixel (i.e., 1000×1000 pixels) in the case of 1000 pixels in 1D, to achieve a ratio of approximately 10 pixels per dot, thus obtaining sufficiently accurate positioning or resolution, in order to confirm the displacement of the dots described above within the FOV.

[0107] This led to Figure 8 The comparison depicted in the text is between the cross-field pixel density (non-shadow) and the cross-field projection point density (shadow). Even with only a small point density, the pixel density needs to be significantly higher to achieve the required accuracy in locating points projected onto the surface (approximately 10 times higher in this example), due to the established methods, such as whether points are discretely present on this pixel or that pixel. To achieve the resolution required for the optical detector configuration for this sampling step, the system becomes extremely long and provides a large number of channels, because each pixel can or will only be associated with an extremely narrow response function to obtain the required accuracy.

[0108] In comparison, Figure 9 Display detector pixels 341 to 34 nInventive evaluation. Even if detector pixels 34 are displayed in two rows, the instance can also be a single-row configuration, depicted only in two rows for illustrative purposes, for example. Figure 4b , Figure 4c , Figure 4d And especially Figure 6a and Figure 6b The overlapping of the response functions depicted. For example, points 26'1 to 26'4 sensed by two or more detector pixels 34 are numerically depicted in their respective detector pixels as amplitude values. For instance, projection point 26'1 provides a signal value of 70 in detector pixel 341 and a signal value of 100 in detector pixel 342; by example, 8-bit quantization with signal values ​​from 0 to 255 is implemented. Any other values ​​are possible in the embodiments, such as 4-bit quantization, 6-bit quantization, 8-bit quantization, 12-bit quantization, or any other value. By comparing the values ​​of adjacent detector pixels, it is possible to accurately assess the position of each point 26'1 to 26'4 based on an understanding of the path of their response functions, even if the provided values ​​are less than those of adjacent detector pixels. Figure 8 The configuration has a significantly smaller number of detector pixels.

[0109] In other words, the method of the present invention enables the use of significantly fewer pixels for each point to be sensed because the positional accuracy of the point is determined by the relative signals of adjacent pixels rather than by the pixel positions themselves. This is particularly effective because the scene is previously known and because it is known that the upcoming point is a non-intersecting point. Otherwise, ambiguity would arise. The response function can, should, or must be ready, and therefore, the system can have a shorter mounting length, and / or can implement a shorter focal length. Figure 9 It is combined with what has been combined Figures 4a to 4d , Figure 5 and Figures 6a to 6b The representation shown is similar to that shown, where, for example, 1 / e is defined. 2 A circle of width is depicted in a 2D field, for example, spanning the width of the response function.

[0110] This representation can also be drawn by the path of curves depicting response functions that are adjacent to each other. (See reference) Figure 8 This will produce many very narrow and separate circles, and Figure 9 This invention provides several overlapping response functions, so that each pixel drawn will obtain one response function in each case. An illustrative grayscale can be depicted as a signal value from 0 to 255 in each pixel. For comparison... Figure 8 and Figure 9It is assumed that the point density in the field of view is uniform. However, the significantly smaller pixel density and wider response function (larger angular spread (in object space) and / or region spread (in image space)) achieve the advantages of this invention. The position of a point relative to the optical axis of the corresponding pixel can be determined from the signal strength of each pixel and based on an understanding of the path of the response function. For simplicity only, a representation of an equidistant point pattern is chosen. In this context, in principle, the number of pixels per channel can be arbitrary. The pixel groups per channel, the number of channels, and therefore the space requirements can be kept small or even minimized.

[0111] According to an embodiment, a method of providing an apparatus includes: providing a plurality of imaging optical channels tilted relative to each other, each of the plurality of imaging optical channels including an optical element and at least one detector pixel arrangement, such that the plurality of optical channels are configured to obtain an image of a dot pattern of a field of view by imaging the field of view. The method further includes providing a display device including a plurality of display pixels arranged in a display plane, such that the display pixels are configured to display graphic information. The method is performed such that the optical channels pass through the display plane and are configured to image a field of view between the display pixels. The method further includes coupling an evaluation device to the detector pixels of the detector pixel arrangement and configuring the evaluation device such that it is configured to evaluate the dot pattern based on a comparison of the signal strengths of different detector pixels (causing super-resolution), while knowing their response functions, to obtain an evaluation result, and controlling the apparatus at least in part based on the evaluation result.

[0112] The embodiments achieve a flat or ultra-flat design for the imaging unit. It can be thin enough to have a thickness of only a few tenths of a millimeter. This is particularly advantageous for applications in smartwatches or other smart devices, where, for example, additional thickness or additional installation space presents even greater problems compared to conventional smartphones, but both problems can be avoided in this application. The embodiments can be manufactured at low cost through wafer-scale production. The embodiments enable the array optics to be adapted for imaging tasks, particularly for positioning projected light spots that do not intersect each other, and allow for a further reduction in mounting length compared to array optics that require distinguishing dot patterns in a conventional manner.

[0113] Even though some aspects have been described within the context of the apparatus, it should be understood that these aspects also represent a description of the corresponding method, such that blocks or structural components of the apparatus can also be understood as corresponding method steps or features of method steps. Similarly, aspects described in connection with or described as method steps also represent a description of corresponding blocks or details of features of the corresponding apparatus.

[0114] The embodiments described above are merely illustrative of the principles of the invention. It should be understood that those skilled in the art will recognize any modifications and variations to the configurations and details described herein. This is because it is intended that the invention be limited only by the scope of the following claims, and not by the specific details presented herein through the description and discussion of the embodiments.

Claims

1. An apparatus for capturing a projected dot pattern, comprising: A plurality of imaging optical channels (12) tilted relative to each other, each of the plurality of imaging optical channels (12) including an optical element (14) and at least one detector pixel arrangement (16); the plurality of imaging optical channels (12) are configured to obtain an image of a dot pattern (24) of a field of view (22) by imaging the field of view (22); A display device (28) includes a plurality of display pixels (34) arranged in a display plane (32), the display pixels (34) being configured to display graphic information; The imaging optical channel (12) passes through the display plane (32) and is configured to image the field of view (22) between the display pixels (34); An evaluation device (36), coupled to detector pixels (18) of the detector pixel arrangement (16) and configured to evaluate the dot pattern (24) based on a comparison of the signal strengths of different detector pixels (18) that would cause super-resolution, to obtain an evaluation result, and to control the device at least in part based on the evaluation result. The evaluation device (36) is configured to determine the location of points in the image based on an evaluation of light intensity and simultaneously taking into account the response function of the detector pixels (18), wherein the same point of the point pattern is projected onto different adjacent detector pixels (18) at the light intensity.

2. The apparatus of claim 1, wherein the imaging optical channel (12) is implemented as an imaging device, and the display pixels (34) have a planar arrangement disposed in the middle region of the imaging optical channel (12) or on the lens of the imaging optical channel (12).

3. The apparatus of claim 1, comprising a projector configured to emit the dot pattern (24).

4. The apparatus of claim 3, wherein the projector is configured to emit the dot pattern (24) with an infrared spectrum; wherein the imaging optical channel (12) is adapted to the infrared spectrum and / or selectively sense the infrared spectrum.

5. The apparatus of claim 3, wherein the projector is configured to emit the dot pattern (24), the dot pattern (24) being at least partially aperiodic or pseudo-random.

6. The apparatus of claim 3, wherein the projector is configured to emit the dot pattern (24) having at least 5,000 dots.

7. The apparatus of claim 3, wherein the projector and the imaging optical channel (12) are arranged to be matched to emit and capture the dot pattern (24) at a reference distance (56) from the apparatus in the field of view (22), such that each detector pixel (18) senses at most one dot of the dot pattern (24) on average once.

8. The apparatus of claim 3, wherein the projector comprises an array of surface emitters and an optical device (14) including a diffractive element.

9. The apparatus of claim 1, wherein the evaluation device (36) is configured to obtain the evaluation result based on a comparison of the image with a reference pattern.

10. The apparatus of claim 1, wherein the evaluation device (36) is configured to perform the comparison of the signal strengths of different detector pixels (18) to determine the position of the points in the dot pattern (24) more accurately than the position that would be achieved by sampling the entire field of view (22) by means of the detector pixels (18).

11. The apparatus of claim 1, wherein the evaluation device (36) is configured to determine deformation in the dot pattern (24) and generate a depth map of the field of view (22) based on the deformation, and obtain the evaluation result based on the depth map.

12. The apparatus of claim 1, configured to determine, based on the evaluation results, a user use authorization for a user at least partially located within the field of view (22), and to control the apparatus in accordance with the user use authorization.

13. The apparatus of claim 1, wherein the microlens of the optical device (14) of the imaging optical channel (12) and the display pixel (34) are arranged in a common plane region.

14. The apparatus of claim 1, wherein the optical element (14) is arranged between adjacent display pixels (34).

15. The apparatus of claim 1, wherein the imaging optical channel (12) is arranged in a first two-dimensional arrangement and the display pixels (34) are arranged in a second two-dimensional arrangement.

16. The apparatus of claim 1, wherein the evaluation device (36) is configured for evaluating structured light or for active stereo evaluation.

17. The apparatus of claim 1, wherein at least one detector pixel arrangement (16) comprises a plurality of detector pixels (18), and the sensing regions of the plurality of detector pixels (18) overlap by at least 50%.

18. The apparatus of claim 1, wherein the detector pixel arrangement (16) is arranged in the image plane of the optics (14).

19. The apparatus of claim 1, wherein the evaluation device (36) is configured to evaluate the field of view (22) by a plurality of detector pixels (18) of each imaging optical channel (12), and subsequently evaluate the field of view (22) relative to the plurality of imaging optical channels (12).

20. The apparatus of claim 1, wherein the total focal length of the imaging optical channel (12) is less than 1 mm.

21. The apparatus of claim 1, wherein an optical device (14) comprising a lens or a stack of lenses is formed.

22. The apparatus of claim 1, wherein the microlenses of different imaging optical channels exhibit channel-specific optical properties.

23. The apparatus of claim 22, wherein the microlens is formed in a channel-specific manner such that, in a tilted line of sight associated with the corresponding imaging optical channel (12), the field of view (22) is focused onto the detector pixel arrangement (16).

24. The apparatus of claim 1, comprising at least one aperture structure disposed between the detector pixel arrangement (16) and the plane of the optics (14) to provide stray light suppression for adjacent imaging optical channels.

25. The apparatus of claim 1, comprising at least one stray light suppression structure (46) disposed between adjacent detector pixel arrangements (16) and between its plane and the plane of the optics (14) to provide stray light suppression for adjacent imaging optical channels.

26. The apparatus of claim 1, comprising at least one stray light suppression structure (78) arranged between adjacent detector pixel arrangements (16), and setting a distance (76) between the detector pixel arrangements (16) and at least a portion of the optics (14).

27. The apparatus of claim 26, wherein the optical devices (14) of different imaging optical channels are connected to each other via a shared substrate, and the stray light suppression structure (78) is connected to the substrate.

28. The device of claim 1, wherein it is configured for facial recognition of a user.

29. The device as claimed in claim 1, wherein it is configured as a smartwatch.

30. A method of providing an apparatus for capturing a projected dot pattern, comprising: A plurality of imaging optical channels (12) tilted relative to each other are provided, each of the plurality of imaging optical channels (12) including an optical element (14) and at least one detector pixel arrangement (16); such that the plurality of imaging optical channels (12) are configured to obtain an image of a dot pattern (24) of a field of view (22) by imaging the field of view (22); A display device (28) is provided, comprising a plurality of display pixels (34) arranged in a display plane (32), such that the display pixels (34) are configured to display graphic information; The imaging optical channel (12) passes through the display plane (32) and is configured to image the field of view (22) between the display pixels (34); An evaluation device (36) is coupled to the detector pixels (18) of the detector pixel arrangement (16) and configured such that it is set to evaluate the dot pattern (24) based on a comparison of the signal strengths of different detector pixels (18) that would cause super-resolution, to obtain an evaluation result, and to control the device at least in part based on the evaluation result. The evaluation device (36) is configured to determine the location of points in the image based on an evaluation of light intensity and simultaneously taking into account the response function of the detector pixels (18), wherein the same point of the point pattern is projected onto different adjacent detector pixels (18) at the light intensity.

Images