Sensor device, sensor module, imaging system and method for operating a sensor device
By grouping the photodetector array into macropixels and using a multiplexing structure to connect adjacent photodetector signals, the detector area is dynamically adjusted, solving the problems of high noise and reduced resolution caused by misaligned light spots, and achieving efficient and accurate 3D stereoscopic image sensing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 에이엠에스오스람아게
- Filing Date
- 2025-01-17
- Publication Date
- 2026-07-14
Smart Images
Figure CN122397263A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to optical detector units, particularly optical detector units for mobile devices. Specifically, it relates to optical detector units comprising an array of photodetectors grouped into multiple macropixels, such that each macropixel is equipped with a time-to-digital converter, the output signal of the photodetector of the corresponding macropixel being delivered to the time-to-digital converter via a series of compression trees. Furthermore, this invention relates to optical sensor devices and imaging systems comprising such detector units, as well as methods for operating optical sensor units. Background Technology
[0002] Optical sensors are increasingly being used in various technological fields, such as smartphones and mobile devices, smart homes and buildings, industrial automation, medical technology, and connected vehicles. In particular, the protection of personal mobile phone data is becoming increasingly important in smartphones and mobile devices. Advanced technologies for locking and unlocking devices are of paramount importance for user convenience.
[0003] As one of several options, this can be achieved through 3D stereoscopic image sensing technology. In particular, electronic products with facial recognition capabilities are increasingly appearing on the market. Taking mobile phones as an example, mobile phones with facial recognition capabilities include at least a flood illuminator, a dot projector, and an infrared camera. The facial recognition process in a mobile phone sequentially involves three steps: proximity sensing (determining whether an object is close to the mobile phone), flood illuminator sensing, and dot projection sensing. Notably, the method of sensing via a flood illuminator involves: the flood illuminator emitting a light source (e.g., infrared light) at a large illumination angle and projecting it onto the surface of an object (e.g., a human face). Subsequently, the infrared camera receives the infrared light reflected from the object, and then a processor, etc., performs calculations to roughly determine whether the object is a human face. If the object is determined to be a face, the dot projector emits multiple light spots projected onto the face, and the infrared camera receives changes in the reflected light spots to calculate the virtual facial surface contour. This can then be used to accurately determine whether the detected face belongs to the mobile phone user or another authenticated person.
[0004] In such applications, and in other applications suitable for measuring 3D information of a target, the time of flight (TOF) of a signal emitted from a dot projector and received in an associated detector in a sensor system after reflection at the target can be measured. The measured time can then be converted into a digital value characterizing the distance of the target from the sensor system in a time-to-digital converter (“TDC”). Compared to floodlighting systems, dot projector-based systems focus light into points, which are projected onto the scene or target using a dot projector on the illuminator side. A major advantage of such systems is that the signal generated in the emitter is concentrated into a point due to focusing, resulting in a higher intensity level (= higher signal level) at the detector and thus better performance.
[0005] The detector associated with such a dot matrix projector can be based on the use of a single-photon avalanche diode, or SPAD for short. SPADs are solid-state photodetectors that are increasingly used in optical sensors, including those for spectroscopy, medical technology, consumer and security applications. SPAD arrays combine high sensitivity and spatial resolution, for example, for high-accuracy distance measurement in time-of-flight sensors. In a SPAD array, groups of individual SPADs can be formed to collectively deliver detector signals to the detector. For example, a given region (also referred to as a "macropixel") embedded in a direct time-of-flight system within a SPAD array can be assigned to a region in an image to create 3D spatial image data. In other words, a macropixel can be configured with an associated time-to-digital converter and is a SPAD array such that the output signal of the individual photodetector of the macropixel can be delivered to the time-to-digital converter of that macropixel, thus belonging to the detector portion. In such a system, a corresponding macropixel can be associated with a 2D portion of an image of a target or scene, and then a third-dimensional parameter for that 2D portion can be added via the TOF value detected by the corresponding macropixel, thereby completing a 3D dataset for that portion of the scene.
[0006] In the dot projector-based systems mentioned above, a significant advantage is that unilluminated SPADs within individual macropixels can be disabled, reducing the system's noise level (lower noise level). Additionally, focusing light on the illuminated points increases the intensity level within those points, which in turn increases the signal level of the enabled pixels. However, to avoid undesirable degradation of sensor resolution and to provide high sensor accuracy, it is desirable to ensure that each reflective point correctly illuminates the area of its associated macropixel. However, in practical applications, points or their reflections are often not uniformly distributed across the scene at the sensor level; for example, distances may be closer at the center of the scene than at the edges. Furthermore, parallax effects can cause point displacement based on distance from the target. Summary of the Invention
[0007] Therefore, the object of the present invention is to provide an improved optical detector unit comprising an optical sensor having an array of detector elements or pixels, which helps to overcome the aforementioned defects and, in particular, enables reliable compensation for effects caused by the displacement of the incident light spot detected on the sensor array. Furthermore, an improved optical sensor device and an improved method for operating such a sensor unit should also be provided.
[0008] Regarding the optical detector unit, which includes an array of photodetectors grouped into multiple macropixels, such that each macropixel is equipped with a time-to-digital converter (TDD), the output signal of the photodetector of the corresponding macropixel can be delivered to the TDD via a series of compression trees. To achieve the objective according to the invention, the photodetectors of each macropixel are also grouped into multiple regions, such that the signal outputs of the photodetectors of each region are commonly connected to a first-level compression tree associated with that region. The first-level compression tree can be connected to the TDD of the corresponding macropixel via multiple subsequent higher-order compression trees. In one aspect of the invention, the first-level compression tree of a region of a macropixel is connected to subsequent compression trees of its macropixel via an associated multiplexer. To also account for incident events detected in adjacent macropixels and thereby compensate for misalignment, in one aspect of the invention, the first-level compression tree of a region of a macropixel can also be switched to multiple multiplexers associated with regions of other macropixels.
[0009] The preferred embodiments are the subject of the dependent claims.
[0010] This invention is based on the consideration that, in order to compensate for misalignment effects, the effective detector region of a macropixel can be dynamically adjusted to include, to the extent possible, individual pixels of adjacent macropixels, if these pixels are also hit by the incident radiation spot. Specifically, this can be achieved by making the pixels of adjacent macropixels available to additionally contribute to the signal directly detected at the original macropixel.
[0011] According to one aspect of the invention, and for reliable connection and short signal transmission time, the photodetectors of each zone may be hardwired or “fixed-wired” connected to their first-level compression tree with respect to their signal output.
[0012] As mentioned above, in one aspect and basic concept of the invention, a multiplexing structure is provided to connect photodiodes of adjacent macropixels to a common TDC channel. This specifically makes it possible to consider the respective readings of the photodiodes that originally belonged to adjacent macropixels. To achieve this in a particularly efficient manner, in a preferred aspect of the invention, all or some macropixels in a macropixel can be divided into four regions, each of which can then correspond to a quarter of the corresponding macropixel. According to one aspect of the invention, the desired expansion of the effective area of the macropixel can be achieved by considering the intersection region, in which the corners of adjacent macropixels are arranged adjacent to each other. Preferably, the macropixels are arranged in a square pattern such that any four of the macropixels are adjacent to each other in the region around a common intersection.
[0013] In one aspect, for each macropixel in this arrangement, the adjacent regions and intersections within the corresponding macropixel can be considered. Therefore, in one aspect of the invention, the outputs of these regions of adjacent macropixels directly adjacent to their common intersections should be provided as additional inputs to the TDC of each of the adjacent macropixels.
[0014] In a preferred embodiment, for each region of the macropixel directly adjacent to the common intersection, its first-level compression tree is switchably connected to a multiplexer associated with the other regions of the macropixel directly adjacent to the common intersection. Specifically, in one aspect of the invention, this connection between regions adjacent to the intersection can be considered as forming a second-level compression tree based on a ring structure formed by these regions and their corresponding interconnections. In other words, in this aspect of the invention, the ring structure comprises four adjacent macropixels adjacent to their common intersection. One-quarter of the ring structure is located at the corresponding corner of each macropixel. The ring structure can be established by connecting various photodiodes, for example, by adjacency. The resulting configuration can essentially be considered similar to a bus interface with a single driver and four receivers.
[0015] In one aspect of the invention, a ring structure is provided so that the output of the first-level compression tree of a region is available as an additional input signal for each of the adjacent macropixels. Therefore, in one aspect of the invention, the ring structure includes a signal ring for each region of interest. Thus, the switchable connection of the first-level compression tree of the corresponding region of a macropixel with multiplexers associated with their respective regions of other macropixels constitutes a second-level compression tree for connection to the corresponding time-to-digital converter.
[0016] Regarding the optical sensor device, the objective mentioned above is to provide an optical detector unit of the type described above for the sensor device. In a preferred embodiment, the sensor device may further include an optical transmitter unit. In yet another preferred embodiment, the optical transmitter unit may include an optical transmitter disposed in a cavity with a hole within a housing, the housing also including a cavity in which the detector unit is disposed. In one aspect of the invention, the optical transmitter unit and the detector unit may be arranged as a time-of-flight module.
[0017] Regarding the method of operating the optical sensor unit, the objective mentioned above according to the present invention can be achieved by a method comprising the following steps:
[0018] - Photons received through the aperture of the cavity are detected by means of an optical sensor arranged in the cavity of the housing.
[0019] The optical sensor includes an array of photodetectors, which are grouped into multiple macropixels. Each macropixel is equipped with a time-to-digital converter, and the output signal of the photodetector of the corresponding macropixel can be delivered to the time-to-digital converter via a series of compression trees.
[0020] - When the incident photon reaches the photodetector, a sensor signal is generated and delivered to a time-to-digital converter associated with a macropixel of the photodetector.
[0021] - The sensor signal is also delivered to a time-to-digital converter associated with another macropixel.
[0022] Preferably, the method may further include the step of using a switchable multiplexer to optionally deliver the sensor signal to the time-to-digital converter associated with the additional macropixel.
[0023] The main advantage achieved by this invention is that, due to the basic concept of using the detector signals of each photodetector as additional inputs to adjacent macropixels when needed, higher data detection efficiency and accuracy can be achieved even when the incident radiation spot is misaligned with the assigned macropixel. In principle, with the concept of this invention, the effective sensor area assigned to an individual macropixel can be modified or shifted by additionally considering the output signals of adjacent macropixels or their corresponding adjacent regions. A particularly applicable use case for this operating mode according to one aspect of the invention is that one or more spots fall on the intersections or intermediate boundaries between adjacent macropixels. In response, offset compensation can be achieved using the concept of this invention. Specifically, all adjacent regions can be connected to the TDC of a macropixel. Any kind of spatial offset is possible. The spots of various incident lights can be unevenly distributed on the scene and therefore on the focal plane region of the sensor device. By rearranging the interconnection of the macropixels to the TDC in the manner described above, the effective detector area of the macropixel can be aligned with the pattern of spot 62. Attached Figure Description
[0024] Preferred embodiments and aspects of the present invention will be further described with reference to the accompanying drawings. In these drawings,
[0025] Figure 1 An embodiment of the optical sensor device is shown;
[0026] Figure 2 yes Figure 1 A top view of the array of photoelectric sensors in the sensor device;
[0027] Figure 3 is Figure 1 Top view of the macropixels of the sensor device ( Figure 3a Top floor Figure 3b (bottom layer)
[0028] Figure 4 It is a block diagram that reveals the details of the circuit.
[0029] Figure 5 This is a schematic diagram of a multiplexing structure;
[0030] Figure 6 It is a top view of a combination of four adjacent macro pixels;
[0031] Figure 7 It is a block diagram of the multiplexer structure; and
[0032] Figure 8 yes Figure 1 An array of photoelectric sensors for a sensor device.
[0033] Identical parts are labeled with the same reference numerals. Detailed Implementation
[0034] Figure 1 An example cross-section of the optical sensor device 1 is shown. The illustrated embodiment of the sensor device 1 is intended for use as a 3D sensor in mobile devices, particularly smartphones, especially for facial recognition purposes. However, it should be noted that device 1 can also be used in other suitable applications, and this disclosure should not be construed as limited to the use cases shown. Figure 1 The optical sensor device 1 shown includes both a detector unit 2 and a transmitter unit 4. In the illustrated embodiment, the detector unit 2 and the transmitter unit 4 can be disposed in a common sensor package of the sensor device 1. Note that the present invention relates only to the design of the optical detector unit 2, and therefore, within the scope of the present invention, this design can be used very well alone for the optical detector unit 2.
[0035] In at least one embodiment, the imaging system includes at least one sensor device according to the aspects discussed below and a host system in which at least one sensor device 1 is embedded. For example, the host system includes a mobile device, a 3D camera device, or a spectrometer. For example, the mobile device may be a mobile phone, a smartphone, a computer, a tablet computer, etc. The sensor device 1 can be implemented in the mobile device using a detector unit 2. Thus, the sensor device can be used as an optical sensor, such as a rangefinder, a proximity sensor, a color sensor, or a time-of-flight sensor.
[0036] The optical sensor device 1 includes an opaque housing 6 having two chambers: a first chamber 8 as part of a transmitter unit 4 and a second chamber 10 as part of a detector unit 2. The opaque housing 6 is disposed on a substrate or carrier 12 and includes a light barrier 14 dividing the housing 6 into the first chamber 8 and the second chamber 10, and the light barrier 14 can therefore be interpreted as a dividing line between the detector unit 2 and the transmitter unit 4. The first chamber 8 and the second chamber 10 are also laterally confined by a frame body 16 disposed within the housing 6. A cover or cap 18, also part of the housing 6, is located opposite the carrier 12 and thus covers the chambers 8 and 10. The cover 18, the frame body 16, and the light barrier 14 are made of a continuous, integral material, such as a molding material.
[0037] The carrier or substrate 12 provides mechanical support and electrical connection to the electronic components integrated into the multispectral sensor 1. For example, the carrier 12 includes a printed circuit board (PCB) (not shown). However, in other embodiments (not shown), the carrier 12 may also be part of the housing 6, and the electronic components may be embedded in the housing 6, for example, by molding.
[0038] An optical emitter 20 is located inside the first chamber 8. The optical emitter 20 is disposed on and electrically connected to the carrier 12, for example, electrically connected to a PCB. For example, the optical emitter 20 is a laser diode, such as a VCSEL or VECSEL. These types of lasers are configured to emit light at a specific wavelength, for example, in the UV, visible, or infrared portions of the electromagnetic spectrum. In some embodiments, the optical emitter 20 is tunable to emit within a specific wavelength range. The specific emission wavelength or emission spectrum is located in IR or UV / Vis. Emission can be narrowband or broadband. For example, a vertical-cavity surface-emitting laser (VCSEL) or a vertical-external-cavity surface-emitting laser (VECSEL) emits primarily in IR or NIR, for example, at 940 nm. In the illustrated embodiment, considering its intended use as a 3D detector device, the emitter 20 in one aspect of the invention is designed as an NIR emitter 20 with primary emission at approximately 940 nm. It can be seen that the advantage of this wavelength selection is that although it is invisible to the human eye, light of this wavelength can be detected well in Si-based detectors, and sunlight is partially absorbed, thus reducing the influence of ambient light.
[0039] For example, in a direct time-of-flight detector system, such a VCSEL can operate in pulse mode (with a pulse width of <500 ps and high peak power). Therefore, the VCSEL can be directly coupled to the driver IC to avoid excessively long interconnects (which can have excessively high inductance for the steep edges of the VCSEL's drive signal). Above the VCSEL, a microlens array can be mounted, which generates points and defines the illumination field.
[0040] As part of the optical detector unit 2, the optical sensor 22 is disposed inside the second chamber 10 and on the carrier 12. In this particular embodiment, the optical sensor 22, along with other electronic devices, is integrated into a single semiconductor sensor die 24. The optical sensor comprises an array 26 of individual optical detector elements or pixels 28, which will be discussed in more detail below. In at least one embodiment, the photodetector 28 comprises a photodiode, a single-photon avalanche diode (SPAD), and / or an avalanche photodiode (APD). The proposed sensor device can be used with different types of photodetectors, particularly photodetectors compatible with time-to-digital converters. In the illustrated embodiment and according to one aspect of the invention, the pixel 28 is implemented as a single-photon avalanche diode or a SPAD. Furthermore, on the receiver side, an imaging lens, an interference filter (bandpass 940 nm), and optional microlenses may be provided above the sensor surface. The CMOS die may be connected to the SPAD die via hybrid bonding.
[0041] As another part of the optical detector unit 2, an array 30 of filters 32 is arranged in the second chamber 10 above the optical sensor 22. The filters 32 are attached to the optical sensor 22 and all have the same transmission characteristics to match the spectrum emitted by the emitter 20 for accurate detection of reflected signals from the emitter 20. The filters 32 can be interference filters, such as optical cutoff filters, bandpass filters, long-pass or short-pass filters, dielectric filters, Fabry-Perot filters, and / or polymer filters. However, since a single wavelength of approximately 940 nm is used in the illustrated embodiment, the filter arrangement could alternatively be a continuous coating.
[0042] To allow light or radiation to pass through appropriately, in each of the two sub-units, detector unit 2 and transmitter unit 4, the cover or cap 18 of housing 6 is provided with a first hole 36 and a second hole 38. The first hole 36 and the second hole 38 are located above optical transmitter 20 and optical sensor 22, respectively. In fact, holes 36 and 38 are located within the emission cone of optical transmitter 20 and the field of view (FOV) of optical sensor 22, respectively. For example, for a fixed transmitter position and orientation, the emission cone includes all points in space that can be illuminated by optical transmitter 20, at least theoretically. Similarly, for example, for a fixed detector position and orientation, the field of view of optical sensor 22 includes all points in space from which light reflected from external target 40 can pass toward optical sensor 22, at least theoretically.
[0043] Optionally, the optical systems can be arranged within the first chamber 8 and the second chamber 10, respectively. For example, the first optical stack 42 includes lenses or lens systems and is attached to an optical emitter 20 inside the first chamber 8. The emitter 20 and its optical stack are combined to form a dot projector system 44. Thus, the optical stack 42 is designed to focus the light emitted by the emitter 20 into a point that is projected onto the scene or target 40. Additionally, for example, the optical stack 42 may have filters or protective glass layers or windows. The first chamber 8 and the second chamber 10 may be sealed with optical windows, respectively.
[0044] Furthermore, the second optical stack 48 may include lenses or lens systems and be attached to the optical sensor 22 inside the second chamber 10. For example, the second optical stack 48 may include a microlens array, in which each microlens is associated with a corresponding pixel 28 of the optical sensor 22. The microlens array may have an additional black aperture layer and define the field of view (FOV) of the optical sensor 22. The optical characteristics of the microlenses (focal length, lens diameter, distance between the lens and the pixel 28, etc.) can be adjusted such that each associated pixel 28 of the optical sensor 22 detects only light from a defined area of the target 40. The optical characteristics of the microlenses can be adjusted so that the defined areas of the target 40 do not overlap. Additionally, the second optical stack 48 may have additional optical layers, such as filters, angle filters, or protective glass layers or windows. Furthermore, the second optical stack 48 may have additional lenses to reduce the FOV of the optical sensor 22, for example, to 10 degrees or less. Such optical devices, such as single lenses or objectives, are used to collect light reflected from the external target 40 and image it onto the sensor device 2, such as a CMOS or CCD photoelectric sensor. Optical bandpass filters typically allow light with the same wavelength as the emitter unit 20 to pass through.
[0045] The control unit 52, along with the optical sensor 22, is integrated into the semiconductor sensor die 24. The control unit 52 includes a driver for the optical emitter 20. For example, the control unit 52 initiates the light emission of the dot matrix projector system 44. The emission of the dot matrix projector system 44 can be modulated, for example, by pulse modulation or by continuous wave modulation. In operation, the optical emitter 20 emits light of a specific wavelength that illuminates the target 40.
[0046] Furthermore, the control unit 52 is also the control unit for the optical sensor unit 2. For example, it provides the sensor signal generated by the optical sensor 22. The control unit 52 can be implemented as control logic, a state machine, a microprocessor, etc. The control unit 52 may also include additional components, such as analog-to-digital converters, time-to-digital converters, and amplifiers, also located in the semiconductor sensor die 24. The semiconductor die 24 may have a printed circuit board (PCB) providing electrical communication to the various components of the multispectral sensor. In particular, the control unit 52 includes means for time-of-flight measurement (i.e., measuring the time difference between the signal emitted by the transmitter 20 and the incident signal reflected at the sensor 22 as the target 40).
[0047] For example, in operation, the external target 40 is located within the field of view (FOV) of the optical sensor device 1. The control unit 52 initiates pulsed light emission through the first aperture 36 and toward the external target 40 by means of the optical emitter 20. Typically, the illumination unit 20 emits modulated light at a high speed of up to about 100 MHz. Alternatively, a single pulse per frame, for example, 30 Hz, can be used. The emitted light has a specific wavelength. In response, the external target 40 will reflect the incident light signal, and the reflected signal can be detected in the sensor 22.
[0048] In one aspect of the invention, the optical sensor device 1 may be intended for use with or in conjunction with a 3D camera device. In a 3D camera device imaging system, two types of images can be generated: a conventional 2D image and an additional 1D image with distance information. These two images can be combined to produce a 3D image. In this context, the sensor device 1 can be used to provide distance information to a 1D image, particularly based on time-of-flight measurements. To properly match this image with the "conventional" 2D image, the sensor device 1 is designed to provide 1D distance information in a manner that can be combined with 2D image information. For this purpose, in one aspect of the invention, the photodetector 28 of the detector unit 2 is provided in the form of an array 26. Array 26 (in...) Figure 2 The top view (showing a portion of it) includes individual photodetectors 28 arranged in a regular, horizontal manner.
[0049] To provide the desired information, photodiodes 28 or pixels 28 are arranged and grouped into so-called macropixels 60. In one aspect of the invention, regarding the space requirements of the evaluation electronics located below, each macropixel in the macropixels 60 comprises 8 × 8 = 64 individual photodiodes 28 arranged regularly. Regarding the lateral size, the size of the pixels 28 is chosen such that, under normal operating conditions, the spot 62 of incident light or incident radiation covers 2 × 2 = 4 individual pixels 28. Therefore, under normal operation, only 4 of the 64 individual pixels 28 will be needed to correctly detect the incident radiation, while the remaining pixels 28 can remain inactive. Within each macropixel 60, the activated photodiodes 28 or pixels 28 of the corresponding macropixel 60 are logically connected in an "OR" mode such that all pixels 28 struck by the incident radiation will be triggered and thus contribute to the generation of the output signal.
[0050] The control unit 52 also includes a readout circuit 64, which in turn includes at least one control terminal. The readout circuit 64 provides a readout path for each of the photodetectors 28. An array of time-to-digital converters 66 (TDCs) can be electrically connected to the converter output terminal of the readout circuit 64, such that the output signals of the photodetectors 28 can be delivered to the time-to-digital converter 66. Specifically, an associated time-to-digital converter 66 is provided for each macropixel 60, such that the output signals of the photodetectors 28 of the corresponding macropixel 60 can be delivered to the associated TDC 66.
[0051] Typically, the accuracy and reliability of the output signal provided by the optical sensor 22 can be limited and reduced by a number of factors. In particular, both static and dynamic sources of potential error in the signal can be correlated. As an example of such a static source of error, geometric factors can become correlated. Furthermore, however, misreadings inherent in the measurement concept, as well as misreadings caused by effects such as the incident light spot 62 not being uniformly distributed to the scene or target 40 (the distance at the center is closer than at the edges), or parallax effects that can cause a specific shift of the light spot 62 relative to the array 26, can occur depending on the distance to the target.
[0052] In a preferred embodiment and in one aspect of the invention, such as in Figure 2 As can be clearly seen, all or a selected number of macropixels 60 can be arranged in a square pattern, such that any four of these macropixels 60 are adjacent to each other in the area around a common intersection 70. In this layout, a relatively large area can be reliably covered by the regular arrangement of individual photodetectors 28, and low production costs can be maintained due to the inherent symmetry of the design. By way of example, for such a square pattern, Figure 2 The diagram shows some possible positions of each spot 62 relative to array 26 and therefore relative to its corresponding macropixel 60:
[0053] - Spots 62a to 62c fall into different areas entirely within macropixel 60.
[0054] - The light spot 62d falls on the vertical boundary 68 between two adjacent macro pixels 60.
[0055] - The light spot 62e falls on the horizontal boundary 68 between two adjacent macro pixels 60.
[0056] - The light spot 62f falls on the center intersection 70 of four adjacent macro pixels 60.
[0057] In the latter case, the light energy of the corresponding spots 62d, 62e, and 62f is distributed across multiple macropixels 60, causing underrepresentation of the light energy in the original "correct" macropixel 60. This may lead to a decrease in the accuracy of the system.
[0058] To overcome this, in one aspect of the invention, a multiplexing scheme is provided that allows photodiodes 28 of adjacent macropixels 60 to be connected to a common TDC channel, thereby enabling consideration of the original readings of the photodiodes 28 belonging to adjacent macropixels 60. Therefore, the sensor device 1 enables device-basis compensation for optical offset. Consequently, the 2D image and the accompanying 1D image can be aligned with higher accuracy.
[0059] In one aspect of the invention, this is achieved by a design in which the array 26 of one or each of the photodetectors 28 of the macropixel 60 is logically arranged as a plurality of subgroups or regions 72 of the pixel 28. Figure 3 shows the macropixel 60 in a top view. The readout architecture associated with the macropixel 60 can be implemented as a separate layer and arranged above or below the array 26, which can be arranged as another layer. However, as shown in the figures, the readout architecture can also be implemented next to the array 26 or between the photodetectors 28 or on the back side of the array 26. The readout architecture includes the array 26 and the readout circuitry 64.
[0060] from Figure 3a As can be seen from the top view of the macropixel 60 shown, according to one aspect of the invention, the array 26 of photodiodes 28 (or, in the illustrated embodiment, SPADs) is subdivided into four subgroups or regions 72a to 72d. Due to the square shape of the macropixel 60, each region 72a to 72d is shaped as a sub-quadrant of the macropixel 60 and is located in one of the corners of the macropixel 60. Each region 72a to 72d includes 4 × 4 = 16 photodiodes 28. By convention, in the following text, for a square structure, for each macropixel 60, its upper right region 72 should be named region 72a, its lower right region should be named region 72b, its upper left region 72 should be named region 72c, and its lower left region should be named region 72d. Relative to the readout circuit 64, a first-level compression tree 76 with a 16:1 ratio is used to group the 16 photodiodes 28 in each region 72a to 72d. In one aspect of the invention, the first-level compression tree 76 can be hardwired in the readout circuit 64 with a fixed wiring scheme.
[0061] Accordingly, such as Figure 3bThe bottom layer of macropixel 60 shown in the top view includes a front end 74 for each region 72a to 72d of photodiode 28, implemented in readout circuit 64. Each front end 74 further includes a quench circuit for the corresponding region 72a to 72d and a first-level compression tree 76. Therefore, the photodetectors 28 of macropixel 60 are grouped into regions 72, such that the photodetectors 28 of each region 72 are commonly connected to the first-level compression tree 76 associated with that region 72 regarding their signal output. Furthermore, readout circuit 64 includes a TDC 66 for the corresponding macropixel 60 and data processing logic 78. In the illustrated embodiment, when photodiode 28 is implemented as a SPAD, the SPAD anode can be connected to the quench circuit via hybrid bonding.
[0062] Figure 4 A block diagram detailing the readout circuitry 64 for the corresponding zones 72a to 72d is shown. As shown, the individual photodiodes 28 or SPADs (16 per zone 72) are hardwired in their outputs to a common 16:1 first-stage compression tree 76.
[0063] As mentioned above, in one aspect and basic concept of the invention, a multiplexing structure is provided to connect the photodiodes 28 of adjacent macropixels 60 to a common TDC channel, particularly for making it possible to take into account the respective readings of the photodiodes 28 that originally belong to the adjacent macropixels 60. Specifically, according to one aspect of the invention, the outputs of those regions 72 directly adjacent to their common intersection 70 in the adjacent macropixels 60 shall be provided as additional inputs to the TDC 66 of each of the adjacent macropixels 60. Figure 5 The schematic diagram illustrates this multiplexing structure, which, in one aspect of the invention, is designed to overcome the effects of undesirable misaligned spot 62 on array 26. In this multiplexing structure, in one aspect of the invention, a second-level compression tree 80 based on a ring structure 82 is implemented in the readout circuit 64.
[0064] Figure 5 A top-down view shows the region near the common intersection 70 of four adjacent macropixels 601, 602, 603, and 604. For macropixel 601, region 72b1 is shown, while for macropixels 602, 603, and 604, their adjacent regions 72d2, 72a3, and 72c4 are shown. Figure 5In the illustrated embodiment, the ring structure 82 surrounds four adjacent macropixels 601, 602, 603, and 604 immediately adjacent to their common intersection 70. A quarter of the ring structure 82 is located at the corresponding corner 84 of each macropixel 60. The ring structure 82 can be established by connecting various photodiodes 28, for example, through adjacency. The resulting configuration can essentially be considered similar to a bus interface with a single driver and four receivers.
[0065] In one aspect of the invention, a ring structure 82 is provided such that the output of the first-level compression tree 76 of regions 72b1, 72d2, 72a3, and 72c4 can be used as an additional input signal for each of the adjacent macropixels 601, 602, 603, and 604. Therefore, in one aspect of the invention, the ring structure 82 includes a signal ring 86 for each region of interest (i.e., regions 72b1, 72d2, 72a3, and 72c4). Each signal ring 86 is connected via an output terminal 88 to the input terminal 90 of the TDC 66 of each of the adjacent macropixels 601, 602, 603, and 604. In other words, a multiplexer is implemented in each region 72b1, 72d2, 72a3, and 72c4, which allows each signal ring 86 in the rings to be connected to the TDC 66 of one of the macropixels 601, 602, 603, and 604. The multiplexer can be controlled by the USE_ZX_X signal. Furthermore, each signal loop 86 is connected via loop input 92 to the output 94 of one of the first-stage compression trees 76 in zones 72b1, 72d2, 72a3, and 72c4.
[0066] In the preferred embodiment shown, corner regions 72b1, 72d2, 72a3, and 72c4 connect four macropixels 601, 602, 603, and 604. In one aspect of the invention, the ring structure 82 correspondingly includes four separate rings 86, each of which is driven by the output terminal 94 of exactly one of regions 72b1, 72d2, 72a3, and 72c4. In the illustrated embodiment, the first-level compression tree 76 of region 72a3 drives ring interconnect 86a, region 72b1 drives ring interconnect 86b, region 72c4 drives ring interconnect 86c, and region 72d2 drives ring interconnect 86d. In one aspect of the invention, the layout of the ring structure 82 can be implemented very symmetrically to avoid timing errors on each line. In each of zones 72b1, 72d2, 72a3, and 72c4, a multiplexer 96 is provided, which enables the activation of the connection between each signal ring 86 in the signal ring 86 and the TDC 66 of the specific macropixels 601, 602, 603, and 604. This multiplexer 96 can be controlled by a USE control signal. Specifically, in one aspect of the invention, each signal ring 86 in the signal ring 86 can be activated individually and independently by the associated multiplexer 96.
[0067] As a result of this design, and according to one aspect of the invention, each of the regions 72b1, 72d2, 72a3, and 72c4 is capable of driving a dedicated ring 86. Therefore, for each macropixel 60 in the macropixels 60, each of its three adjacent regions 72 can be signal-connected to its corresponding TDC channel. Thus, one-quarter of each adjacent macropixel 60 (since each macropixel 60 is divided into four regions 72) can be connected to the TDC channel of the corresponding macropixel 60. In one aspect of the invention, this interconnection is implemented in a highly balanced manner to avoid timing errors in the signals.
[0068] Figure 6A top-down view shows a combination of four adjacent macropixels 60, illustrating in more detail how the interconnects described above are positioned between the four corners 84 of the four adjacent macropixels 60. Four individual macropixels 60 are shown, each subdivided into four regions (72a to 72d). Each macropixel 60 includes a TDC 66 (located on the bottom layer; the TDC 66 of the top-left macropixel 60 is shown only by indication). In normal operation, the four regions 72a to 72d of each macropixel 60 are connected to its TDC 66. A multiplexer ring structure 82 in the region of the adjacent corners 84 enables the connection of regions 72 from adjacent macropixels 60 to the TDC 66 of each macropixel 60. In the example shown here, the multiplexer 96 of the top-left macropixel 60 is activated by the corresponding USE control signal, and thus the connection of the adjacent regions 72 of the three adjacent macropixels 60 to the top-left TDC 66 is activated.
[0069] Therefore, one of the signal loops 86 implemented by each drive in the corresponding region 72 in this state feeds its signal as an additional signal contribution to the TDC 66 of the upper left macropixel 60. In other words, as a result of activating the multiplexer 96 on the upper left macropixel 60, the SPAD events of the adjacent region 72 are directed to the upper left TDC 66. Thus, the effective detection surface of the upper left macropixel 60 is increased through the corresponding adjacent region 72, thereby effectively providing a quarter-offset in the x and y directions. This is due to... Figure 6 The area 98 indicated is shown. An example of this mode of operation according to one aspect of the invention is that a spot 62 of light falls on the intersection 70 of the four macropixels 60 shown. Of course, other use cases in which other offsets can be implemented can be solved. In particular, all adjacent areas 72 can be connected to the TDC 66 of a certain macropixel 60. Any kind of spatial offset is possible.
[0070] Figure 7A block diagram of a multiplexer structure provided for a sensor device according to one aspect of the invention is shown. As described above, in a preferred embodiment, quadrant or region 72 comprises 4 × 4 = 16 individual photodiodes 28 or SPADs, which are grouped by 16:1 (first-level) compression trees 76 to form respective regions 72. Each output of the first-level compression tree 76 of each region 72 is connected to the corresponding region multiplexer 96. Additionally, the output 100 can be connected to a ring 86 via its corresponding line 102, which is connected to adjacent regions 72. As a result of this interconnection, the region multiplexer structure 96 can be considered as compression trees 80 for regions 72a to 72d. The USE signal control signal for the multiplexer 96 then enables the activation of an event signal connection with any of the sixteen SPADs of the respective region 72. Within each macropixel 60, two additional 2:1 compression trees 104 (third-level) are provided, which complete the connection to the TDC 66. In other words, in the illustrated embodiment and in one aspect of the invention, the first-level compression tree 76 of each region 72 with respect to its signal output is connected to the time-to-digital converter 66 of its respective macropixel 60 via a second-level or first higher-order compression tree 80 (as provided by the multiplexing ring structure 82) and further via a subsequent higher-order (third-level) compression tree 104. In detail, and according to one aspect of the invention, the first-level compression tree 76 of each region 72 of the macropixel 60 with respect to its output signal is connected to the subsequent compression tree 104 of its macropixel 60 via an associated multiplexer 96, and it can also be switchably connected (due to the ability to activate the functionality of the multiplexer 96) to a plurality of multiplexers 96, each multiplexer 96 associated with a region 72 of other macropixels 60.
[0071] In particular, and according to one aspect of the invention, for the regular square pattern shown above, wherein a ring structure 82 is provided between adjacent areas 72, the first-level compression tree 76 of each area 72 directly adjacent to the common intersection 70 can be switchably connected to a multiplexer 96 associated with the other areas 72a, 72b, 72c, 72d of the macropixel 60 directly adjacent to the common intersection 70.
[0072] Figure 8 In Figure 2The representations similar to those shown illustrate several possible use cases according to aspects of the invention. Various light spots 62 of the incident light can be unevenly distributed on the scene, and therefore unevenly distributed on the focal plane region of the sensor device 1. By rearranging the interconnection of the macropixels 60 to the TDC 66 in the manner described above, the effective detector region of the macropixels 60 can be aligned with the pattern of the light spots 62. Specifically (the shifts are indicated by the framed areas 106 respectively):
[0073] The 62g light spot fell into the correct macropixel area and did not require rearrangement.
[0074] For light spots 62h and 62i, a spatial offset to the right in one quadrant is provided.
[0075] For a 62k light spot, a spatial offset to the lower quadrant is provided.
[0076] For light spots 62l and 62m, a spatial offset of one quadrant downwards and to the right is provided.
[0077] Note that embodiments of the optical sensor device 1 discussed herein have been disclosed for the purpose of familiarizing the reader with the novel aspects of the concept. Although preferred embodiments have been shown and described, many changes, modifications, equivalents, and substitutions can be made to the disclosed concept by those skilled in the art without unnecessarily departing from the scope of the claims.
[0078] In particular, this disclosure is not limited to the disclosed embodiments, and provides examples of as many alternatives as possible to the features included in the discussed embodiments. However, it is intended that any modifications, equivalents, and substitutions of the disclosed concept be included within the scope of the claims appended herein.
[0079] Features listed in a separate dependent claim can be advantageously combined. Furthermore, the reference numerals used in the claims are not intended to limit the scope of the claims.
[0080] Furthermore, as used herein, the term "comprising" does not exclude other elements. Additionally, as used herein, the article "a" is intended to include one or more parts or elements, and is not limited to being interpreted as meaning only one.
[0081] Unless otherwise expressly stated, no method described herein is intended to be construed as requiring its steps to be performed in a particular order. Therefore, no specific order is intended to be inferred unless the method claims actually describe the order to be followed by its steps or unless the claims or description otherwise specifically state that the steps should be limited to a particular order.
[0082] List of reference numerals
[0083] 1 Optical sensor device
[0084] 2 Optical Detector Unit
[0085] 4 Optical transmitter units
[0086] 6. Opaque casing
[0087] 8 and 10 chambers
[0088] 12 substrate
[0089] 14 Light Barrier
[0090] 16. Main Framework
[0091] 18 lids
[0092] 20 transmitters
[0093] 22 sensors
[0094] 24 Sensor Chips
[0095] 26 array
[0096] 28. Photodetector
[0097] 30 array
[0098] 32 Filter
[0099] 36 and 38 holes
[0100] 40 Targets
[0101] 42, 48 Optical Stacking
[0102] 44 Dot Matrix Projector System
[0103] 52 Control Unit
[0104] 60 macropixels
[0105] 62 light spots
[0106] 64 Readout Circuit
[0107] 66 Time to Digital Converter
[0108] 68 Boundaries
[0109] 70 intersections
[0110] District 72
[0111] 74 Frontend
[0112] 76 Compressed Tree
[0113] 78 Data Processing Logic
[0114] 80 Compressed Tree
[0115] 82-ring structure
[0116] 84 jiao
[0117] 86 signal loop
[0118] 88 Output Terminal
[0119] 90 Input Terminal
[0120] 92 ring input terminal
[0121] 94 Output terminal
[0122] 96 Multiplexer
[0123] 98 Shaded Area
[0124] 100 output
[0125] Line 102
[0126] 104 Compressed Tree
[0127] 106 Framed Areas
Claims
1. An optical detector unit (2) comprising an array (26) of photodetectors (28), the array (26) of photodetectors (28) being grouped into a plurality of macropixels (60), such that each macropixel (60) is provided with a time-to-digital converter (66), wherein the output signal of the photodetector (28) of the corresponding macropixel (60) can be delivered to the time-to-digital converter (66) via a series of compression trees (76, 80, 104), wherein, The photodetector (28) of each macropixel (60) is further grouped into multiple zones (72) such that the signal output of the photodetector (28) of each zone (72) with respect to them is commonly connected to a first-level compression tree (76) associated with the zone (72), wherein the first-level compression tree (76) with respect to its signal output is connected to the time-to-digital converter (66) of the corresponding macropixel (60) via multiple subsequent higher-order compression trees (80, 104), and wherein the first-level compression tree (76) of the zone (72) of the macropixel (60) with respect to its output signal is connected to the subsequent compression tree (104) of its macropixel (60) via an associated multiplexer (96), and may also be switchably connected to multiple multiplexers (96) associated with each other's zones (72) of the macropixel (60).
2. The optical detector unit (2) according to claim 1, wherein, The photodetectors (28) of each zone (72) are hardwired with their signal outputs to their first-level compression tree (76).
3. The optical detector unit (2) according to claim 1 or 2, wherein, The first-level compression tree (76) of the region (72) of the macropixel (60) and the switchable connection of the plurality of multiplexers (96) associated with the respective regions (72) of other macropixels (60) constitute a second-level compression tree (80) for connection with the corresponding time-to-digital converter (76).
4. The optical detector unit (2) according to any one of claims 1 to 3, wherein, Multiple macro pixels (60) are each divided into four regions (72a, 72b, 72c, 72d).
5. The optical detector unit (2) according to any one of claims 1 to 4, wherein, Each zone (72) includes 16 individual photodetectors (28).
6. The optical detector unit (2) according to any one of the preceding claims, wherein, Multiple macro pixels (60) are arranged in a square pattern such that any four macro pixels in the macro pixels (60) are adjacent to each other in the area around a common intersection (70).
7. The optical detector unit (2) according to claim 6, wherein, For each region (72a, 72b, 72c, 72d) of the macropixel (60) directly adjacent to the common intersection (70), its first-level compression tree (76) is switchably connected to a multiplexer (96) associated with the other regions (72a, 72b, 72c, 72d) of the macropixel (60) directly adjacent to the common intersection (70).
8. An optical sensor device (1) comprising an optical detector unit (2) according to any one of claims 1 to 7.
9. The optical sensor device (1) according to claim 8 further includes an optical transmitter unit (4).
10. The optical sensor device (1) according to claim 9, wherein, The optical transmitter unit (4) includes an optical transmitter (20) arranged in a cavity (8) with a hole (36) in a housing (6), and the housing (6) also includes a cavity (10) in which the detector unit (2) is arranged.
11. The optical sensor device (1) according to claim 9 or 10, wherein, The optical transmitter unit (4) and the detector unit (2) are arranged as a time-of-flight module.
12. An imaging system comprising an optical sensor device (1) according to any one of claims 8 to 11 and a host system wherein said sensor device (1) is embedded, wherein, The host system is one of a mobile device, a 3D camera device, or a spectrometer.
13. A method for operating an optical detector unit (2) according to any one of claims 1 to 7, the method comprising the following steps: - Photons received through the aperture (38) of the cavity are detected by means of an optical sensor (22) arranged in the cavity (10) of the housing (6). The optical sensor (22) includes an array (26) of photodetectors (28), which is grouped into multiple macropixels (60), such that each macropixel (60) is provided with a time-to-digital converter (66). The output signal of the photodetector (28) of the corresponding macropixel (60) can be delivered to the time-to-digital converter via a series of compression trees (76, 80, 104). - When the incident photon reaches the photodetector (28), a sensor signal is generated and said sensor signal is delivered to a time-to-digital converter (66) associated with a macropixel (60) of said photodetector (28), and - The sensor signal is also delivered to a time-to-digital converter (66) associated with another macropixel (60).
14. The method of claim 13, further comprising the step of: - A switchable multiplexer (96) is used to optionally deliver the sensor signal to the time-to-digital converter (66) associated with the additional macropixel (60).