Optical sensing devices and their sensing methods, light receivers and electronic devices

Abstract

This application provides an optical sensing device, including a light emitter and a light receiver. The light emitter sequentially emits light pulses; the light receiver includes multiple effective pixels, each of which includes multiple physical pixels. Each effective pixel is configured such that it is activated only in one corresponding time partition out of multiple time partitions. The physical pixels are used to receive light pulses reflected back from an object from the emitted light pulses and convert the received light pulses into electrical signals. Each light pulse period corresponds to one of the multiple time partitions. This application can improve the tolerance of the optical sensing device to ambient light, thereby increasing the maximum outdoor ranging distance. Compared with traditional time gating schemes, it can also improve the frame rate.

Description

Technical Field

[0001] This application relates to the field of sensing technology, and in particular to an optical sensing device, an electronic device, a light receiver, and a sensing method for the optical sensing device. Background Technology

[0002] Currently, ranging sensing technology using Time-of-Flight (ToF) is rapidly developing and is widely used in devices such as mobile phones for laser autofocus and presence detection. In the future, it will be widely used in augmented reality (AR), 3D modeling, and augmented reality navigation. Current direct Time-of-Flight (dToF) measurement schemes use Time-correlated Single Photon Counting (TCSPC) to perform histogram counting and then extract the ToF from the histogram. Because TCSPC can only count the first photon signal per cycle, the counts of other photons are discarded, resulting in a so-called pile-up effect. In situations with strong ambient light, the pile-up effect severely limits the maximum measurement distance. Common methods to suppress the pile-up effect include reducing the amount of ambient light received, error correction, and time gating. The time gating method divides the total measurement range into many time windows, identifies the window with the highest number of counts as the fine measurement window, and performs histogram counting only within this fine measurement window. If the stacking effect is not suppressed, in strong outdoor ambient light conditions, the stacking effect becomes more and more severe as the measurement distance increases, eventually causing the signal to be submerged in noise and making it impossible to obtain TOF, thus greatly limiting the outdoor ranging capability. Summary of the Invention

[0003] This application provides an optical sensing device, electronic device, light receiver, and optical sensing method that can improve the tolerance to ambient light.

[0004] In a first aspect, embodiments of this application provide an optical receiver including a plurality of effective pixels, wherein at least one effective pixel includes a plurality of physical pixels, the physical pixels being used to receive optical signals and convert them into corresponding electrical signals for output, the physical pixels having a preset correspondence with a plurality of time partitions in which the optical receiver operates, and the effective pixels being configured such that the physical pixels are turned on only in one corresponding time partition among the plurality of time partitions and turned off in other time partitions.

[0005] Secondly, embodiments of this application provide an optical sensing device, the optical sensing device comprising:

[0006] A light emitter, used to sequentially emit light pulses; and

[0007] As described above, in the optical receiver, the plurality of time partitions are configured correspondingly according to a light pulse period and

[0008] The processing module is used to process the electrical signal to generate sensing data.

[0009] Thirdly, embodiments of this application provide an electronic device, the electronic device including a main body and an optical sensing device as described above disposed on the main body.

[0010] Fourthly, embodiments of this application provide an optical sensing method, including:

[0011] Control the light emitter to emit light pulses sequentially;

[0012] The physical pixels of each effective pixel in the optical receiver are controlled to be turned on only in one corresponding time partition out of multiple time partitions, and turned off in other time partitions, so as to receive the reflected light pulse formed by the emitted light pulse being reflected back by the object and convert it into an electrical signal for output. The effective pixel includes multiple physical pixels, and the physical pixels have a preset correspondence with the multiple time partitions in which the optical receiver operates. The multiple time partitions are set according to a light pulse cycle.

[0013] The electrical signal is processed to generate sensing data.

[0014] The aforementioned optical sensing devices, electronic devices, light receivers, and optical sensing methods, since the effective pixels are configured such that the physical pixels are turned on only in one corresponding time partition and turned off in other time partitions, the turned-on physical pixels only receive the light signal arriving in the current time partition and are not affected by the noise photons arriving in the previous time partitions. This significantly reduces the stacking effect of ambient light and improves the tolerance to ambient light. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the structure of an optical sensing device provided in an embodiment of this application.

[0017] Figure 2 A schematic diagram of an electronic device provided in an embodiment of this application.

[0018] Figure 3 This is a schematic diagram of an optical receiver provided in an embodiment of this application.

[0019] Figure 4 A flowchart of a sensing method provided in an embodiment of this application.

[0020] Figure 5 A schematic diagram of time partitioning provided for the first embodiment of this application.

[0021] Figure 6 Another schematic diagram of time partitioning provided for the first embodiment of this application.

[0022] Figure 7 A schematic diagram of time partitioning provided for the second embodiment of this application.

[0023] Figure 8 for Figure 4 The diagram shows the correspondence between physical pixels and time partitions.

[0024] Figure 9 for Figure 4 Another schematic diagram showing the correspondence between physical pixels and time partitions.

[0025] Figure 10 for Figure 4 The diagram shows a sub-process of the sensing method.

[0026] Figure 11 This is a schematic diagram comparing the light pulses detected by existing optical sensing devices and the light pulses detected by the optical sensing device of this application.

[0027] Component Symbol Explanation

[0028] Label Name Label Name

[0029] 100 Electronic devices 13 Data processing modules

[0030] 10 Optical Sensing Devices 131 TDC Module

[0031] 11. Optical emitter 132. Statistical module

[0032] 12 light receivers 20 main body

[0033] 120 effective pixels T per light pulse cycle

[0034] 121 physical pixels t time partition

[0035] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0037] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar planned objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data are interchangeable where appropriate; in other words, the described embodiments are implemented according to a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, may also include other content; for example, a process, method, system, product, or apparatus that comprises a series of steps or apparatuses is not necessarily limited to those steps or apparatuses explicitly listed, but may include other steps or apparatuses not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0038] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" and "second" may explicitly or implicitly include one or more of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0039] Please refer to Figure 1 This is a schematic diagram of the structure of the optical sensing device provided in an embodiment of this application. In this embodiment, the optical sensing device is a device that collects time-of-flight measurements to determine the distance between the optical sensing device and an external object, such as a direct time-of-flight (dToF) sensor. The optical sensing device 10 includes a light emitter 11, a light receiver 12, and a data processing module 13.

[0040] The light emitter 11 is configured to emit light pulses sequentially. The light emitter 11 includes, but is not limited to, a laser emitter, a light-emitting diode emitter, etc. A portion of the light pulses emitted by the light emitter 11 is reflected by an object to form reflected light pulses.

[0041] Please refer to the following: Figure 3The optical receiver 12 includes multiple effective pixels, which are used to receive optical signals and convert them into corresponding electrical signals for output. The optical signals include reflected light pulses and photons of ambient light. Optionally, in some embodiments, the multiple effective pixels can be arranged in an array, and reflected light pulses can be incident on the effective pixels. The spot of one reflected light pulse covers at least one effective pixel. In this embodiment, the spot of one reflected light pulse covers one effective pixel, such as... Figure 3 As shown by the circles in the diagram. An effective pixel 120 includes a plurality of physical pixels 121, which can be arranged in an array. Optionally, in some embodiments, each effective pixel of the light receiver 12 includes the same number of physical pixels 121. It is understood that in other embodiments, different effective pixels of the light receiver 12 may also include different numbers of physical pixels 121. Physical pixels 121 include, for example, single-photon avalanche diodes for receiving optical signals and converting them into corresponding electrical signal outputs. However, optionally, in other embodiments, single-photon avalanche diodes may be replaced with other suitable photoelectric sensing elements, such as avalanche photodiodes. For example, as... Figure 3 The light receiver 12 shown has nine physical pixels 121 in one effective pixel, arranged in a 3x3 array. Understandably, in the above embodiments, each physical pixel 121 corresponds one-to-one with a time partition. Optionally, in some embodiments, the number of physical pixels 121 included in one effective pixel can be set according to actual conditions and is not limited here. In some embodiments, multiple effective pixels can also be arranged irregularly. Alternatively, multiple physical pixels 121 can also be arranged irregularly. Optionally, only some of the multiple effective pixels 120 in the light receiver 12 have multiple physical pixels 121, while other effective pixels 120 include only one physical pixel 121. The following description still uses the example of an effective pixel 120 including multiple physical pixels 121.

[0042] The physical pixel 121 has a preset correspondence with multiple time zones in which the light receiver 12 operates. The effective pixel 120 is configured such that the physical pixel 122 is only active in one corresponding time zone and active in the others. These multiple time zones are set based on a single light pulse cycle. For example: Figure 5 As shown, a light pulse period has a duration of T. A light pulse period comprises multiple time segments t, and the duration of each time segment t can be the same or different. Understandably, the sum of the durations of multiple time segments t equals the duration of a light pulse period. That is, a time segment t is equivalent to a fraction of a light pulse period.

[0043] In this embodiment, taking each effective pixel as comprising 9 physical pixels 121 as an example, the number of time partitions t is 9, each time partition being equivalent to one-ninth of a light pulse. The 9 time partitions t1 to t9 are respectively the first T / 9, the second T / 9, ... the ninth T / 9. The 9 physical pixels 121 are respectively configured to be turned on in time partitions t1 to t9. For example, the 9 physical pixels 121 are represented by the first physical pixel, the second physical pixel, ... the ninth physical pixel. The first physical pixel is configured to be turned on in time partition t1, the second physical pixel is configured to be turned on in time partition t2, and so on, with the ninth physical pixel being configured to be turned on in time partition t9. In this case, the third physical pixel is only turned on in time partition t3 and will not receive photons arriving in time partitions t1 and t2. Therefore, photons arriving in time partitions t1 and t2 will not cause the third physical pixel to generate a count. Understandably, when a time partition t begins, the corresponding physical pixel 121 is activated to receive and sense photons; when a time partition t ends, the corresponding physical pixel 121 is deactivated and stops receiving and sensing photons. Optionally, the correspondence between the physical pixels 121 and the time partition t is the same in each effective pixel 120, and the corresponding physical pixel 121 activated in each effective pixel 120 in different time partitions is located at the same position in its respective effective pixel array. That is, the physical pixels 121 located at the same position in each effective pixel are activated simultaneously at the beginning of the same time partition t in one light pulse cycle, and deactivated simultaneously at the end of the same time partition t.

[0044] More specifically, multiple physical pixels 121 arranged in an array can be traversed in the order of left to right in the same row and top to bottom in different rows (e.g., Figure 8 As shown, they operate in a one-to-one correspondence across multiple time partitions t. Alternatively, the multiple physical pixels 121 arranged in an array can also be traversed in the order of top to bottom in the same column and left to right in different columns (e.g., ...). Figure 9 As shown, each of the multiple physical pixels 121 arranged in an array works in a one-to-one correspondence with the time partition t. It is understood that the correspondence between the multiple physical pixels 121 arranged in an array and the time partition t can also be set in other ways according to the actual situation, and this application does not make specific limitations in this regard. The correspondence between the physical pixels 121 at different positions in each effective pixel 120 and the time partition t can also be different, or can be changed according to the actual situation.

[0045] Since each physical pixel can only avalanche once in the same light pulse cycle, if all the physical pixels of each effective pixel are activated at the beginning of the light pulse cycle to receive the reflected light pulse, when the intensity of the reflected light pulse is high or the ambient light is too strong, most of the physical pixels of each effective pixel will be avalancheed quickly and a sufficient number of physical pixels will not be retained to receive the reflected light pulse that arrives in the later part of the light pulse cycle. This results in the loss of the reflected light pulse count in the later part of the light pulse cycle and the stacking effect where most of the count values ​​are concentrated in the earlier part of the light pulse cycle.

[0046] Specifically, such as Figure 11 As shown, the optical pulse signal S0 is detected by the optical sensing device when a stacking effect occurs, and the optical pulse signal S1 is detected by the optical sensing device provided in the above embodiment. In the optical sensing device experiencing a stacking effect, physical pixels avalanche due to earlier-arriving photons, resulting in the inability to detect later-arriving photons, leading to photon count loss. Consequently, light pulses reflected from a distance cannot be detected. For example, the optical pulse signal S0 cannot detect photons starting at time Tp, before the end of a light pulse period T, resulting in the inability to count photons reflected after time Tp, i.e., the inability to measure photons reflected from a distance. However, in the optical sensing device provided in the above embodiment, different physical pixels 121 of the effective pixels only receive photons within their corresponding time partitions and are not affected by previously arriving noise photons. The tolerance to ambient light is significantly improved later in a light pulse period T. Therefore, photons reflected from a distance can still be detected by the corresponding physical pixels in that time partition, and the corresponding signal peak can also be measured, such as the peak of the 8th time partition of the optical pulse signal S1. Therefore, the stacking effect of ambient light in the sensing device provided in the above embodiments is greatly reduced, which can improve the tolerance of the optical sensing device to ambient light.

[0047] The data processing module 13 is configured to process the electrical signal generated by the optical receiver 12 to generate sensing data. In this embodiment, the sensing data may be a timestamp, a histogram, or a distance, etc. In this embodiment, the sensing data is a histogram. Specifically, the data processing module 13 includes a Time-to-Digital Converter (TDC) module 131 and a statistics module 132.

[0048] The TDC module 131 processes the electrical signal generated by the light receiver 12 to obtain a timestamp. Specifically, the TDC module 131 calculates the time interval of the electrical signal generated for each physical pixel 121 and converts the time interval into a timestamp. The TDC module 131 can calculate the time interval of the electrical signal based on the difference between the time of the emitted light pulse and the time of the received light pulse for each physical pixel 121, and calculate the flight time of each light pulse, i.e., the timestamp, based on the time interval. Each effective pixel 120 is electrically connected to at least one TDC module 131. In this embodiment, the effective pixels 120 and TDC modules 131 are electrically connected in a one-to-one correspondence. Optionally, the physical pixels 121 and TDC modules 131 are electrically connected in a one-to-one correspondence. In this embodiment, the effective pixels 120 can be stacked with the corresponding TDC modules 131, i.e., the TDC modules 131 are placed below the corresponding effective pixels 120, thereby increasing the effective photosensitive area of ​​each effective pixel 120. Optionally, in some embodiments, each TDC module 131 is only turned on within its corresponding time partition, that is, it is turned on at the beginning of the corresponding time partition and turned off at the end of the corresponding time partition. In this embodiment, the TDC module 131 can be implemented using a time-to-digital converter (TDC) with time-to-digital conversion function.

[0049] The statistics module 132 counts the number of each physical pixel 121 in each time unit within the corresponding time partition according to the timestamp; and obtains the count distribution of the corresponding effective pixels throughout the entire light pulse cycle based on the combination of the counts of each physical pixel 121 in each time unit within the corresponding time partition. Specifically, the statistics module 132 is implemented using a histogram circuit, and the sensing result is a histogram. The histogram includes multiple bins, and the time unit corresponds to the bin of the histogram, which is the shortest time interval that the TDC module 131 can distinguish. The horizontal axis of the histogram represents the timestamp, and the vertical axis of the histogram represents the count of the light pulse. When a physical pixel 121 receives a light pulse, the statistics module 132 counts in the corresponding bin based on the corresponding timestamp. In this embodiment, each physical pixel 121 corresponds to a sub-histogram. When a physical pixel 121 receives a light pulse, it counts in the corresponding bin of the corresponding sub-histogram according to the timestamp of the light pulse. Based on the time sequence of time partition t, the histograms corresponding to multiple physical pixels 121 in each effective pixel 120 are concatenated and combined to generate a total histogram of effective pixels 120, which is the sensing result of each effective pixel 120. The sensing result is used to obtain depth information.

[0050] Since each TDC module 131 is only turned on within its corresponding time partition, that is, turned on at the beginning of the corresponding time partition and turned off at the end of the corresponding time partition, the timestamp corresponding to each valid pixel can be obtained directly, and thus the histogram can be obtained directly.

[0051] Common methods for suppressing the stacking effect in existing technologies include reducing ambient light reception, error correction by counting, and time gating. The conventional time gating method involves selecting a window sequentially from front to back, repeatedly transmitting a certain number of pulse cycles, counting the total photons within that window, and then selecting the next pulse cycle, repeating the same number of pulse cycles and counting photons until all windows are counted. The photon count of the time window containing the return pulse signal will be higher than other windows; therefore, finding the window with the highest count determines the window containing the return signal. Only this window is then selected for histogram counting to obtain an accurate Time of Flight (TOF). While the previous time gating method can improve tolerance to ambient light (suppressing the stacking effect), it significantly sacrifices frame rate. The frame rate reduction mainly stems from two aspects: first, the measurement process needs to be divided into a coarse measurement process for finding the window and a fine measurement process for histogram counting; second, when finding the window, counting needs to be performed sequentially, with each window requiring the same number of pulse cycles to be repeated. For scenarios with high frame rate requirements, such as automotive dToF LiDAR, the maximum ranging range must be reduced, thus limiting the application scope of optical sensing devices, i.e., dToF. In the embodiments of this application described above, the physical pixels in the optical sensing device are activated only at the beginning of each time partition. Therefore, the physical pixels are not affected by photons arriving in previous time partitions within the corresponding time partition. Consequently, the optical sensing device can measure multiple time partitions simultaneously, eliminating the need to measure partitions sequentially one by one, thereby increasing the frame rate. For example, dividing the device into N partitions can increase the frame rate by N times.

[0052] In the above embodiments, multiple physical pixels of each effective pixel are controlled to be activated in the corresponding time partitions according to multiple time partitions, and a sensing result is generated based on the light pulse received by the multiple physical pixels in each effective pixel. In the above embodiments, since each physical pixel is not affected by previously arrived photons (i.e., noise photons) in the corresponding time partition within a light pulse cycle, the tolerance to ambient light is greatly improved. Therefore, a corresponding peak value can still be measured for light pulses reflected from a distance. In addition, the optical sensing device can measure multiple time partitions simultaneously, eliminating the need to measure partitions one by one in sequence. For example, dividing into N partitions can increase the frame rate by N times, thereby achieving the goal of increasing the frame rate. That is, the optical sensing device can sense objects at a greater distance while maintaining the same frame rate. Furthermore, each effective pixel is electrically connected to a TDC module, which can effectively reduce the number of TDC modules, thereby reducing the size of the optical sensing device.

[0053] Optionally, the sensed data can be distance data. That is, the data processing module 13 also includes a measurement module (not shown), which directly obtains depth / distance information based on the histogram. Optionally, the sensed data can also be a timestamp, that is, the data processing module 13 can omit the statistics module 132, and the timestamp can be processed by another data processing module to obtain the measurement data required by the user. Alternatively, the sensed data can also be an electrical signal, that is, the data processing module 13 can omit the TDC module 131 and the statistics module 132, and the electrical signal can be processed by another data processing module to obtain the measurement data required by the user.

[0054] Optionally, the durations of the multiple time partitions included in a single optical pulse cycle can also be different, and correspondingly, the resolution of the TDC module 131 corresponding to each time partition can also be different. It should be understood that the resolution of the TDC module 131 refers to the shortest time interval that the TDC module can resolve, i.e., the fineness of the binning or time unit in the corresponding histogram. The higher the resolution of the TDC module 131, the shorter the time interval that the TDC module 131 can resolve, and the finer the binning or time unit in the corresponding histogram. Figure 6As shown, the duration of time partition t9 is different from that of time partitions t1...t8. The time partitions at the beginning are shorter than those at the end. The duration of the time partitions in a light pulse cycle gradually increases from beginning to end. That is, the duration of time partition t1 is shorter than that of time partition t2, the duration of time partition t2 is shorter than that of time partition t3, and so on, with the duration of time partition t8 being shorter than that of time partition t9. For example, the duration of time partition t1 is 10 ns, the duration of time partition t2 is 20 ns, and so on, with the duration of time partition t9 being 90 ns. Correspondingly, the resolution of the corresponding TDC also gradually decreases. Optionally, the multiple time partitions with different durations can increase in stages rather than gradually. For example, a light pulse cycle can be divided into a beginning stage, a middle stage, and an end stage according to time, with each stage including multiple time partitions. Multiple time partitions within the same stage have the same duration, while time partitions in different stages have different durations. Specifically, for example: the initial stage includes time partitions t1 to t2, with a total duration of 20 ns for each partition and a duration of 10 ns for each partition; the TDC resolution is 100 ps. The intermediate stage includes time partitions t3 to t7, with a total duration of 100 ns for each partition and a duration of 20 ns for each partition; the TDC resolution is 200 ps. The final stage includes time partitions t8 to t9, with a total duration of 200 ns for each partition and a duration of 100 ns for each partition; the TDC resolution is 400 ps. Understandably, the absolute accuracy requirement for ranging at long distances is lower. Therefore, the partition time after one light pulse cycle can be lengthened, and a lower-precision TDC can be used. Understandably, when sensing objects at a long distance, a lower-precision TDC module can be used to calculate the timestamp of the electrical signal corresponding to each physical pixel, thereby reducing the power consumption of the optical sensing device and the requirement for a smaller number of physical pixels while maintaining the ranging range.

[0055] Optionally, in some embodiments, the sum of the durations of the multiple time partitions is greater than the duration of one optical pulse cycle. Specifically, two adjacent time partitions partially overlap, such that the sum of the durations of the multiple time partitions is greater than the duration of one optical pulse cycle; that is, in two adjacent time partitions, the start time of one time partition falls within the start time of the other time partition, such as... Figure 7As shown. Accordingly, each valid pixel is electrically connected to multiple TDC modules 131. Further, each physical pixel 121 among the valid pixels is electrically connected to one TDC module 131. At the start of a time partition t, the corresponding physical pixel 121 is activated, and the TDC module 131 electrically connected to the corresponding physical pixel 121 is turned on; at the end of a time partition t, the corresponding physical pixel 121 is turned off, and the TDC module 131 electrically connected to the corresponding physical pixel 121 stops working. Specifically, as... Figure 7 As shown, the second time partition t2 begins before the first time partition t1 ends, meaning the beginning of the second time partition t2 falls within the first time partition t1. For example, the second time partition t2 begins 2ns before the first time partition t1 ends, meaning the first time partition t1 and the second time partition t2 have a 2ns overlap.

[0056] Since the two adjacent time partitions partially overlap, and the photons received by the physical pixel 121 can be counted through the corresponding TDC, the situation that the measurement results may be incorrect when the light pulse falls exactly at the junction of different time partitions can be avoided.

[0057] Please refer to the following: Figure 2 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 100 includes a main body 20 and an optical sensing device 10 disposed on the main body 20. The electronic device 100 includes, but is not limited to, mobile phones, laptops, tablets, electronic watches, smart glasses, etc. In this embodiment, a mobile phone is used as an example to illustrate the electronic device 100. The specific structure of the optical sensing device 10 is as described in the above embodiment. Since the electronic device 100 uses the optical devices of all the above embodiments, it possesses at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be elaborated further here.

[0058] Please refer to the following: Figure 1 , Figure 4 and Figure 5 The sensing method of the optical sensing device 10 utilizes sensing data of an object (not shown) sensed by the optical sensing device 10. This sensing data can be the result data required by the user, or it can be process data used to calculate the result data required by the user. Process data can be a timestamp, a histogram, or an electrical signal generated by the light receiver 12. Result data includes, but is not limited to, the depth image of the object, the distance between the object and the optical sensing device 10, and a two-dimensional image of the object. The aforementioned sensing data can be applied to devices such as mobile phones for laser autofocus, biometrics, augmented reality (AR), 3D modeling, and augmented reality navigation. The sensing method of the optical sensing device 10 specifically includes the following steps.

[0059] Step S102: Control the light emitter 11 to sequentially emit light pulses. In this embodiment, the light emitter 11 of the optical sensing device 10 is controlled to emit light pulses. The light emitter 11 includes, but is not limited to, laser emitters, light-emitting diode emitters, etc. A portion of the light pulses emitted by the light emitter 11 is reflected by an object to form reflected light pulses.

[0060] Step S104: In the light receiver 12, each effective pixel 120 has one physical pixel 121 activated only in one corresponding time partition among multiple time partitions to receive reflected light pulses converted into electrical signals. The effective pixel 120 includes multiple physical pixels 121, and one light pulse period corresponds to one of the multiple time partitions. In this embodiment, the sensing result is generated based on the electrical signal, which will be specifically described in step S106. More specifically, the multiple effective pixels 120 are arranged in an array, and reflected light pulses can enter the effective pixels 120. The light spot of one light pulse covers at least one effective pixel 120. In this embodiment, the light spot of one light pulse covers one effective pixel 120, such as... Figure 3 As shown by the circles in the diagram. An effective pixel 120 includes a plurality of physical pixels 121. The plurality of physical pixels 121 are arranged in an array. Optionally, the number of physical pixels 121 included in each effective pixel 120 of the light receiver 12 may be the same or different. The physical pixels 121 include, for example, a single-photon avalanche diode. However, optionally, in other embodiments, the single-photon avalanche diode may be replaced with other suitable photosensitive elements, such as avalanche photodiodes, etc. For example, as Figure 3 The light receiver 12 shown has each effective pixel 120 comprising nine physical pixels 121 arranged in a 3x3 array. Understandably, the number of physical pixels 121 in each effective pixel 120 can be set according to actual conditions and is not limited here. In some embodiments, the multiple effective pixels 120 may also be irregularly arranged. Alternatively, the multiple physical pixels 121 may also be irregularly arranged. Optionally, only some of the multiple effective pixels 120 in the light receiver 12 may have multiple physical pixels 121, while the other effective pixels 120 may only include one physical pixel 121. The following description still uses the example of an effective pixel 120 comprising multiple physical pixels 121.

[0061] Optionally, in some embodiments, the effective pixel 120 is configured to enable one physical pixel 122 for each of a plurality of time partitions. In this embodiment, the plurality of time partitions are set according to a light pulse period. For example: Figure 5As shown, the duration of one optical pulse cycle is T, and one optical pulse cycle corresponds to multiple time partitions t, with each time partition t having the same duration. It can be understood that the sum of the durations of multiple time partitions t equals the duration of one optical pulse cycle. That is, one time partition t is equivalent to a fraction of one optical pulse cycle.

[0062] In this embodiment, taking each effective pixel 120 as comprising 9 physical pixels 121 as an example, the number of time partitions t is 9, and each time partition is equivalent to one-ninth of a light pulse. When a light pulse has a pulse period T, the 9 time partitions t1 to t9 are respectively the first T / 9, the second T / 9, ..., the ninth T / 9. The 9 physical pixels 121 are configured to be active in time partitions t1 to t9. For example, the 9 physical pixels 121 are represented by the first physical pixel, the second physical pixel, ..., the ninth physical pixel. The first physical pixel is configured to be active in time partition t1, the second physical pixel is configured to be active in time partition t2, and so on, with the ninth physical pixel being configured to be active in time partition t9. In this case, the third physical pixel is only active in time partition t3 and will not receive photons arriving in time partitions t1 and t2. Therefore, photons arriving in time partitions t1 and t2 will not cause the third physical pixel to generate a count. Understandably, when a time partition t begins, the corresponding physical pixel 121 is activated to receive and sense photons; when a time partition t ends, the corresponding physical pixel 121 is deactivated and stops receiving and sensing photons. Optionally, the correspondence between the physical pixels 121 and the time partition t is the same in each effective pixel 120, and the physical pixel 121 activated in each effective pixel 120 in different time partitions is located at the same position in its respective effective pixel array. That is, the physical pixels 121 located at the same position in each effective pixel are activated simultaneously at the beginning of the same time partition t in one light pulse cycle, and deactivated simultaneously at the end of the same time partition t.

[0063] More specifically, multiple physical pixels 121 arranged in an array can be traversed in the order of left to right in the same row and top to bottom in different rows (e.g., Figure 8 As shown, they operate in a one-to-one correspondence across multiple time partitions t. Alternatively, the multiple physical pixels 121 arranged in an array can also be traversed in the order of top to bottom in the same column and left to right in different columns (e.g., ...). Figure 9As shown, the physical pixels 121 arranged in an array and their corresponding time partitions t operate in a one-to-one correspondence. It is understood that the correspondence between these physical pixels 121 and time partitions t can also be configured differently depending on the actual situation, and this application does not impose specific limitations on this. The correspondence between physical pixels 121 at different positions within each effective pixel and time partition t can also be different, or can be changed according to the actual situation.

[0064] Since each physical pixel can only avalanche once in the same light pulse cycle, if all physical pixels of each effective pixel are activated at the beginning of the light pulse cycle to receive reflected light pulses, under conditions of high reflected light pulse intensity or strong ambient light, most of the physical pixels of each effective pixel will be quickly avalanced, leaving insufficient physical pixels to receive reflected light pulses arriving later in the light pulse cycle. This results in the loss of reflected light pulse counts in the later part of the light pulse cycle, with most counts concentrated in the early part of the light pulse cycle due to a stacking effect. In the optical sensing device 10 provided in the above embodiments of this application, the effective pixels 120 are configured to activate a portion of the corresponding physical pixels 121 for sensing according to the divided time partitions in a light pulse cycle. Therefore, the activated physical pixels 121 only receive photons arriving in the current time partition and are not affected by noise photons arriving in the previous time partitions, thereby significantly reducing the stacking effect of ambient light and improving the tolerance of the optical sensing device 10 to ambient light.

[0065] Step S106, the optical sensing method further includes processing the electrical signal to generate sensing data. Specifically, the sensing data can be a timestamp, a histogram, or distance, etc. In this embodiment, the sensing data is a histogram. Please refer to... Figure 10 Specifically, step S106 includes the following steps.

[0066] Step S1060: The electrical signal generated by the optical receiver 12 is processed to obtain a timestamp. Specifically, step S1060 is implemented by the TDC module 131. Further, the TDC module 131 processes the electrical signal corresponding to each physical pixel 121 to obtain a timestamp. Specifically, the TDC module 131 calculates the time interval of the electrical signal corresponding to each physical pixel 121 and converts the time interval into a timestamp. The TDC module 131 can calculate the time interval of the electrical signal based on the difference between the time of transmitting the light pulse and the time of receiving the light pulse by each physical pixel 121, and calculate the flight time of each light pulse, i.e., the timestamp, based on the time interval. Each valid pixel 120 is electrically connected to at least one TDC module 131. In this embodiment, valid pixels 120 and TDC modules 131 are electrically connected in a one-to-one correspondence. Optionally, physical pixels 121 are electrically connected in a one-to-one correspondence with TDC modules 131. In this embodiment, the effective pixels 120 can be stacked with the corresponding TDC modules 131, that is, the TDC modules 131 are positioned below the corresponding effective pixels 120, thereby increasing the effective photosensitive area of ​​each effective pixel. Optionally, in some embodiments, each TDC module 131 is only turned on within its corresponding time partition, that is, it is turned on at the beginning of the corresponding time partition and turned off at the end of the corresponding time partition. In this embodiment, the TDC module 131 can be implemented using a time-to-digital converter (TDC) with time-to-digital conversion function.

[0067] Step S1062: Based on the timestamp, count the number of each physical pixel 121 in the corresponding time unit. Specifically, the counting module 132 is implemented using a histogram circuit, and the sensing result is a histogram. The histogram includes multiple bins, and the time unit corresponds to the bin of the histogram. The horizontal axis of the histogram represents the timestamp, and the vertical axis represents the count of light pulses. When a physical pixel 121 receives a light pulse, the counting module 132 counts in the corresponding bin based on the corresponding timestamp.

[0068] Step S1064: The count of the corresponding effective pixels 120 is obtained based on the count combination of each physical pixel 121 in the corresponding time unit. In this embodiment, each physical pixel 121 has a corresponding sub-histogram. When a physical pixel 121 receives a light pulse, the corresponding bin count is performed in the corresponding sub-histogram according to the timestamp of the light pulse. The histograms corresponding to multiple physical pixels 121 in each effective pixel 120 are spliced ​​and combined according to the time order of time partition t to generate a total histogram of effective pixels 120, that is, the sensing result of each effective pixel 120. The sensing result is used to obtain depth information.

[0069] Since each TDC module 131 is only turned on within its corresponding time partition—that is, turned on at the beginning of the corresponding time partition and turned off at the end of the corresponding time partition—the timestamp corresponding to each valid pixel 120 can be obtained directly, thus directly obtaining the histogram. Therefore, the optical sensing device 10 provided in the above embodiment can also measure multiple time partitions simultaneously, eliminating the need to measure each partition sequentially. For example, dividing into N partitions can increase the frame rate by N times, thereby achieving the goal of increasing the frame rate.

[0070] In the above embodiments, multiple physical pixels 121 of each effective pixel 120 are controlled to be in the on state in the corresponding time partition according to multiple time partitions, and a sensing result is generated based on the light pulse received by the multiple physical pixels 121 in each effective pixel 120. In the above embodiments, since each physical pixel 121 is not affected by previously arrived photons (i.e., noise photons) in the corresponding time partition, the tolerance to ambient light is greatly improved. Therefore, a corresponding peak value can still be measured for light pulses reflected from a distance. At the same frame rate, the optical sensing device 10 can sense objects at a greater distance. In addition, the optical sensing device 10 can measure multiple time partitions simultaneously, eliminating the need to measure partitions one by one in sequence. For example, dividing into N partitions can increase the frame rate by N times, thereby achieving the purpose of increasing the frame rate. Furthermore, each effective pixel 120 is electrically connected to a TDC module, which can effectively reduce the number of TDC modules, thereby reducing the size of the optical sensing device 10.

[0071] Optionally, the sensing data can be distance. Specifically, the sensing method further includes directly obtaining depth / distance information based on a histogram. Optionally, the sensing data can also be a timestamp, and the sensing method can omit step S1064, as the timestamp can be processed by another device to obtain the measurement data required by the user. Alternatively, the sensing data can also be an electrical signal, that is, the sensing method can also include steps S1062 and S1064, and the electrical signal can be processed by another device to obtain the measurement data required by the user.

[0072] Optionally, the durations of the multiple time partitions included in one optical pulse cycle can also be different, and correspondingly, the resolution of the TDC module 131 corresponding to each time partition can also be different. For example... Figure 6As shown, the duration of time partition t9 is different from that of time partitions t1...t8. The time partitions at the beginning are shorter than those at the end. The duration of the time partitions in a light pulse cycle gradually increases from beginning to end. That is, the duration of time partition t1 is shorter than that of time partition t2, the duration of time partition t2 is shorter than that of time partition t3, and so on, with the duration of time partition t8 being shorter than that of time partition t9. For example, the duration of time partition t1 is 10 ns, the duration of time partition t2 is 20 ns, and so on, with the duration of time partition t9 being 90 ns. Correspondingly, the resolution of the corresponding TDC also gradually decreases. Optionally, the multiple time partitions with different durations can increase in stages rather than gradually. For example, a light pulse cycle can be divided into a beginning stage, a middle stage, and an end stage according to time, with each stage including multiple time partitions. Multiple time partitions within the same stage have the same duration, while time partitions in different stages have different durations. Specifically, for example: the initial stage includes time partitions t1 to t2, with a total duration of 20 ns for each partition and a duration of 10 ns for each partition; the TDC resolution is 100 ps. The middle stage includes time partitions t3 to t7, with a total duration of 100 ns for each partition and a duration of 20 ns for each partition; the TDC resolution is 200 ps. The final stage includes time partitions t8 to t9, with a total duration of 200 ns for each partition and a duration of 100 ns for each partition; the TDC resolution is 400 ps. Understandably, the absolute accuracy requirement for ranging at long distances is lower. Therefore, the partition time after one light pulse cycle can be lengthened, and a lower-precision TDC can be used. Understandably, when sensing objects at a long distance, a lower-precision TDC module can be used to calculate the timestamp of the electrical signal corresponding to each physical pixel, thereby reducing the power consumption of the optical sensing device and reducing the requirement for the number of physical pixels while ensuring the ranging range.

[0073] Optionally, in some embodiments, the sum of the durations of the multiple time partitions is greater than the duration of one optical pulse cycle. Specifically, two adjacent time partitions partially overlap, such that the sum of the durations of the multiple time partitions is greater than the duration of one optical pulse cycle; that is, in two adjacent time partitions, the start time of one time partition falls within the start time of the other time partition, such as... Figure 7As shown. Accordingly, each valid pixel is electrically connected to multiple TDC modules 131. Further, each physical pixel 121 among the valid pixels is electrically connected to one TDC module 131. At the start of a time partition t, the corresponding physical pixel 121 is activated, and the TDC module 131 electrically connected to the corresponding physical pixel 121 is turned on; at the end of a time partition t, the corresponding physical pixel 121 is turned off, and the TDC module 131 electrically connected to the corresponding physical pixel 121 stops working. Specifically, as... Figure 7 As shown, the second time partition t2 begins before the first time partition t1 ends, meaning the beginning of the second time partition t2 falls within the first time partition t1. For example, the second time partition t2 begins 2ns before the first time partition t1 ends, meaning the first time partition t1 and the second time partition t2 have a 2ns overlap.

[0074] Since the two adjacent time partitions partially overlap, and the photons received by the physical pixel 121 can be counted through the corresponding TDC, the situation that the measurement results may be incorrect when the light pulse falls exactly at the junction of different time partitions can be avoided.

[0075] Many of the functional units described in this specification have been labeled as modules to more specifically emphasize their independent implementation. For example, modules can be implemented as hardware circuits, including custom-designed very large-scale integrated circuits (VLSI) or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. Modules can also be implemented within programmable hardware devices such as field-programmable gate arrays, programmable array logic, programmable logic devices, and so on.

[0076] Modules can also be implemented in software so that they can be executed by various types of processors. A module of identifiable executable code can, for example, comprise a physical or logical block of one or more computer instructions, wherein the physical or logical block can be organized, for example, as an object, program, or function. However, the executable files of an identifiable module do not need to be physically located together, but can include different instructions stored in different locations, which, when logically combined, constitute a module and implement the module's defined objectives.

[0077] Executable code modules can be single instructions or many instructions, and can even be distributed across numerous different code segments in different programs, spanning many storage devices. Similarly, operational data will be identified and presented within this module, and can be represented in any suitable form and organized within any suitable data structure. Operational data can be collected as a single dataset or can be distributed across different locations, including across different storage devices, and can exist at least in part solely as electronic signals on a system or network.

[0078] Those skilled in the art will recognize that modifications can be made to the described exemplary embodiments within the scope of the claimed invention, and that many other embodiments are also possible.

[0079] The above-listed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. An optical receiver, characterized by, The effective pixel includes a plurality of physical pixels, at least one of which is configured to receive a light signal and convert it into an electrical signal output, and the physical pixels have a preset correspondence with a plurality of time partitions in which the light receiver works, and the time partitions are set according to a light pulse period, and the effective pixel is configured such that the physical pixels thereof are only turned on in a corresponding time partition among the plurality of time partitions and turned off in other time partitions; wherein the light receiver uses time-correlated single photon counting to count the electrical signal histogram, and then extracts the time of flight from the histogram.

2. The optical receiver of claim 1, wherein, The plurality of physical pixels are arranged in an array, and the physical pixels in the same row are sequentially turned on in the corresponding time partitions from left to right, and the physical pixels in different rows are sequentially turned on in the corresponding time partitions from top to bottom.

3. The optical receiver of claim 1, wherein, The plurality of physical pixels are arranged in an array, and the physical pixels in the same column are sequentially turned on in the corresponding time partitions from top to bottom, and the physical pixels in different columns are sequentially turned on in the corresponding time partitions from left to right.

4. The optical receiver of claim 1, wherein, The physical pixels at the same position in each effective pixel work in the same time partition.

5. An optical sensing device, characterized by The optical sensing device comprises: a light emitter configured to sequentially emit light pulses; The light receiver according to any one of claims 1-4, wherein the plurality of time partitions are set according to a light pulse period, and a processing module configured to process the electrical signal to generate sensing data.

6. The optical sensing device of claim 5, wherein, The sum of the time lengths of the plurality of time partitions is equal to or greater than the time length of a light pulse period.

7. The optical sensing device of claim 5, wherein, The time lengths of the plurality of time partitions can be different.

8. The optical sensing device of claim 5, wherein, Two adjacent time partitions partially overlap.

9. The optical sensing device of claim 5, wherein, The processing module includes a TDC module and a statistical module, the TDC module is configured to calculate the time interval of the electrical signal and convert the time interval into a timestamp, each physical pixel is connected to a corresponding TDC module, and each TDC module is only turned on in the time partition in which the corresponding connected physical pixel works, and the statistical module is configured to count the number of time units corresponding to each physical pixel according to the timestamp, and combine the counts of each physical pixel in the corresponding time units to obtain the count of the corresponding effective pixel.

10. The optical sensing device of claim 9, wherein, Each physical pixel is stacked with a corresponding TDC module.

11. The optical sensing device of claim 5, wherein, A part of the emitted light pulses is reflected by an object to form reflected light pulses, and a light spot of one of the reflected light pulses covers at least one of the effective pixels.

12. An electronic device, comprising: The electronic device comprises a main body and an optical sensing device according to any one of claims 5-11 arranged on the main body.

13. An optical sensing method, characterized by, The optical sensing method comprises: controlling the light emitter to sequentially emit light pulses; controlling the physical pixels of each effective pixel in the light receiver to be turned on only in a corresponding time partition among a plurality of time partitions and turned off in other time partitions, so as to receive reflected light pulses formed by the emitted light pulses reflected by an object and convert them into electrical signal outputs, wherein the effective pixel includes a plurality of physical pixels, the physical pixels have a preset correspondence with a plurality of time partitions in which the light receiver works, and the time partitions are set according to a light pulse period; and processing the electrical signal to generate sensing data. The sum of the time lengths of the plurality of time partitions is equal to or greater than the time length of a light pulse period. The time lengths of the plurality of time partitions can be different. Two adjacent time partitions partially overlap. The processing module includes a TDC module and a statistical module, the TDC module is configured to calculate the time interval of the electrical signal and convert the time interval into a timestamp, each physical pixel is connected to a corresponding TDC module, and each TDC module is only turned on in the time partition in which the corresponding connected physical pixel works, and the statistical module is configured to count the number of time units corresponding to each physical pixel according to the timestamp, and combine the counts of each physical pixel in the corresponding time units to obtain the count of the corresponding effective pixel. Each physical pixel is stacked with a corresponding TDC module. A part of the emitted light pulses is reflected by an object to form reflected light pulses, and a light spot of one of the reflected light pulses covers at least one of the effective pixels. The electronic device comprises a main body and an optical sensing device according to any one of claims 5-11 arranged on the main body. The optical sensing method comprises: controlling the light emitter to sequentially emit light pulses; controlling the physical pixels of each effective pixel in the light receiver to be turned on only in a corresponding time partition among a plurality of time partitions and turned off in other time partitions, so as to receive reflected light pulses formed by the emitted light pulses reflected by an object and convert them into electrical signal outputs, wherein the effective pixel includes a plurality of physical pixels, the physical pixels have a preset correspondence with a plurality of time partitions in which the light receiver works, and the time partitions are set according to a light pulse period; and processing the electrical signal to generate sensing data.

14. The optical sensing method of claim 13, wherein, The time durations of the plurality of time partitions can be different.

15. The optical sensing method of claim 13, wherein, The sum of the plurality of time partitions is equal to one optical pulse period or greater than one optical pulse period.

16. The optical sensing method of claim 13, wherein, Two adjacent time partitions partially overlap.

Images