Control method of image sensor, image sensor, and electronic device

Abstract

The application discloses a control method of an image sensor, an image sensor and an electronic device. The control method of the image sensor comprises the following steps: one SPAD pixel unit in M SPAD pixel units acquires a clock signal and a control signal; the SPAD pixel unit generates a digital signal based on the clock signal; the SPAD pixel unit outputs the digital signal to a target digital counter based on the control signal; the target digital counter performs counting processing on the digital signal in a digital counting period allocated to the SPAD pixel unit, and obtains a digital counting signal of the SPAD pixel unit; and the digital counting signal of the SPAD pixel unit is used for generating a target image; wherein the target digital counter allocates mutually staggered digital counting periods to different SPAD pixel units in the M SPAD pixel units.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of electronics, and particularly relates to a control method of an image sensor, an image sensor and an electronic device. BACKGROUND

[0002] In a CMOS image sensor (CIS), a SPAD (Single-Photon Avalanche Diode) device can be used as a pixel photosensitive element of the image sensor. The SPAD pixel unit has a single-photon sensing level of ultra-high sensitivity, and the imaging capability is very significant.

[0003] In the related art, the CMOS image sensor can use a global shutter working mode to synchronously control and operate each SPAD pixel unit in the SPAD pixel array of the CMOS image sensor. However, in the case where there are multiple SPAD pixel units in the CMOS single-photon image sensor, the existing global shutter working mode can not be effectively operated. SUMMARY

[0004] Embodiments of the present application provide a control method of an image sensor, an image sensor and an electronic device, which solve the technical problem that the existing global shutter working mode can not be effectively operated in the related art.

[0005] In a first aspect, embodiments of the present application provide a control method of an image sensor, the image sensor comprising a SPAD pixel array and a target digital counter, there being M SPAD pixel units in the SPAD pixel array that share the target digital counter, the M SPAD pixel units being connected to the target digital counter via the same output wire, the control method comprising:

[0006] one of the M SPAD pixel units acquires a clock signal and a control signal; M is a positive integer greater than 1;

[0007] the SPAD pixel unit generates a digital signal based on the clock signal;

[0008] the SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal;

[0009] the target digital counter performs counting processing on the digital signal within a digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image.

[0010] The target digital counter assigns different SPAD pixel units in the M SPAD pixel units with mutually staggered digital counting time periods.

[0011] In a second aspect, an embodiment of the present application provides an image sensor, which is configured to implement the control method in the first aspect.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, which includes the image sensor in the second aspect.

[0013] In the embodiment of the present application, the image sensor includes a single photon avalanche transistor (SPAD) pixel array and a target digital counter, the SPAD pixel array includes M SPAD pixel units sharing the target digital counter, the M SPAD pixel units are connected to the target digital counter via the same output wire, one of the M SPAD pixel units acquires a clock signal and a control signal; M is a positive integer greater than 1; the SPAD pixel unit generates a digital signal based on the clock signal; the SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; the target digital counter performs counting processing on the digital signal in a digital counting time period assigned to the SPAD pixel unit, to obtain a digital counting signal of the SPAD pixel unit; and the digital counting signal of the SPAD pixel unit is used to generate a target image; wherein the target digital counter assigns different SPAD pixel units in the M SPAD pixel units with mutually staggered digital counting time periods. In this way, in the scenario where M SPAD pixel units share one target digital counter for counting in the image sensor, compared with the working mode of the global shutter in the related art, since the target digital counter in the embodiment of the present application assigns different SPAD pixel units in the M SPAD pixel units with mutually staggered digital counting time periods, it is ensured that the M SPAD pixel units sharing the target digital counter for counting can perform counting processing using the time of the target digital counter individually, thereby realizing effective operation of each SPAD pixel unit in the image sensor. BRIEF DESCRIPTION OF DRAWINGS

[0014] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings in which:

[0015] Figure 1 FIG. 1 is a schematic diagram of an image sensor provided by an embodiment of the present application;

[0016] Figure 2 FIG. 2 is a schematic diagram of another image sensor provided by an embodiment of the present application;

[0017] Figure 3 is a schematic flow chart of a control method of an image sensor provided by an embodiment of the present application;

[0018] Figure 4 is a schematic diagram of a running mode timeline of an image sensor in a unit of time provided by an embodiment of the present application;

[0019] Figure 5 is a schematic diagram of a running mode timeline of an image sensor in a unit of time provided by another embodiment of the present application;

[0020] Figure 6 is a schematic diagram of a running mode timeline of an image sensor in a unit of time provided by another embodiment of the present application;

[0021] Figure 7 is a schematic diagram of a running mode timeline of an image sensor in a unit of time provided by another embodiment of the present application;

[0022] Figure 8 is a schematic diagram of an electronic device provided by an embodiment of the present application.

[0023] Legend of reference signs:

[0024] 10 - image sensor; 100 - pixel layer; 200 - logic layer; 300 - connection point; 400 - input / output pin; 110 - SPAD pixel array; 111 - SPAD pixel unit; 120 - digital counter; 130 - clock and control circuit; 140 - signal processing circuit; 150 - port circuit. DETAILED DESCRIPTION

[0025] Embodiments of the present application will be described in detail below, examples of which are shown in the drawings, wherein the same or similar notations represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary only, and are used only for explanation of the present application, and cannot be understood as a limitation of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts, fall within the scope of protection of the present application.

[0026] The terms "first", "second" in the description and claims of the present application can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally means that the front and rear associated objects are in an "or" relationship.

[0027] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "connected", "connection" should be understood in a broad sense, for example, it can be directly connected, or indirectly connected through an intermediate medium, or the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0028] In the related art, each SPAD pixel unit of the image sensor contains a clock and control circuit, a digital counter and a digital readout circuit. When the excitation signal reaches the clock and control circuit, the clock and control circuit controls the quenching and charging of the SPAD device in the SPAD pixel unit and counts by the digital counter, and the counting result is read by the digital readout circuit and output as a pixel output signal. Based on the working principle of the above-mentioned SPAD pixel unit, the CMOS single-photon image sensor based on SPAD adopts a global shutter working mode to synchronously control and operate each SPAD pixel unit in the SPAD pixel array.

[0029] Referring to Figure 1 The image sensor 10 provided by the present application removes the digital counter 120 from the SPAD pixel unit 111, and adopts a column-parallel digital counter on the logic layer 200. Each column of SPAD pixel units of the pixel layer 100 shares an output wire, and the output wire is electrically connected to the corresponding digital counter 120 located on the logic layer 200. The output wire adopts Cu-Cu bonding or TSV (Trans-Silicon Via, through silicon via) process at the connection point 300 of the chip cross-layer area.

[0030] Among them, at least one digital counter 120, clock and control circuit 130, signal processing circuit 140, port circuit 150 and other auxiliary circuits are arranged on the logic layer 200. Each row of SPAD pixel units 111 of the pixel layer 100 shares a clock and control signal. The clock and control signal generated by the clock and control circuit 130 reaches the corresponding row of SPAD pixel units of the pixel layer through the wire. Similarly, the wire adopts Cu-Cu bonding or TSV process at the connection point 300 of the chip cross-layer area. The clock and control circuit 130 can control and transmit signals to any one or more rows of SPAD pixel units.

[0031] Among them, the output signal (multi-bit digital signal) of each digital counter is scanned or transmitted to the signal processing circuit 140 for subsequent image signal processing and buffering. Finally, it is transmitted to the outside of the image sensor 10 through the port circuit 150 and the input / output pin 400.

[0032] In the embodiment of the present application, the digital counter 120 is removed from the SPAD pixel unit 111 and is arranged separately on the logic layer 200, so that the complexity of the circuit in the SPAD pixel unit 111 is greatly reduced and the unit pixel area is reduced. At the same time, the digital counter 120 on the logic layer 200 adopts a column parallel design, and each digital counter 120 can be shared by multiple SPAD pixel units 111 in the same column, thereby reducing the number of digital counters 120 required by the image sensor 10. Further, more SPAD pixel units can be arranged on a unit chip area, thereby improving the effective resolution of the image sensor.

[0033] In addition, with reference to Figure 2 , the present application provides an image sensor, in which the SPAD pixel array 110 composed of multiple SPAD pixel units 111 is arranged on the pixel layer 100. Compared with the related art, the pixel circuit array corresponding to the SPAD array is cancelled on the logic layer, and the saved space can be used to arrange the digital counter and auxiliary circuits such as signal processing circuit and port circuit, so that the layout area of the logic layer and the pixel layer is consistent, and the optical performance of the image sensor in the CCM module is improved.

[0034] It should be noted that, in the image sensor 10 provided by the embodiment of the present application, multiple SPAD pixel units 111 in the same column need to share one digital counter 120 for counting, and it is difficult for multiple SPAD pixel units 111 sharing one digital counter 120 to run synchronously (i.e., it is difficult for multiple SPAD pixel units 111 sharing one digital counter to count independently), which causes the existing global shutter to be unable to effectively work.

[0035] Therefore, the present application provides a control method of an image sensor, which adopts a rolling shutter-like operation mode to control the M rows of SPAD pixel units of the image sensor row by row, and allocates mutually staggered digital counting time periods to each row of SPAD pixel units, so that the digital counting time periods of the SPAD pixel units in the same column and different rows are staggered, and each SPAD pixel unit in the same column but different rows can use the time of the digital counter for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor.

[0036] Figure 3 is a schematic diagram of the control method of the image sensor provided by the embodiment of the present application.

[0037] As Figure 3 shown, the control method of the image sensor provided by the embodiment of the present application can include:

[0038] Step 310: One of the M SPAD pixel units acquires a clock signal and a control signal;

[0039] Step 320: The SPAD pixel unit generates a digital signal based on the clock signal;

[0040] Step 330: The SPAD pixel unit outputs the digital signal to a target digital counter based on the control signal;

[0041] Step 340: The target digital counter performs counting processing on the digital signal in a digital counting period allocated to the SPAD pixel unit, to obtain a digital counting signal of the SPAD pixel unit;

[0042] The target digital counter allocates mutually staggered digital counting periods to different SPAD pixel units of the M SPAD pixel units.

[0043] The digital counting signal of the SPAD pixel unit is used to generate a target image.

[0044] In the embodiments of the present application, the image sensor includes a SPAD pixel array and a target digital counter, the SPAD pixel array has M SPAD pixel units sharing the target digital counter, and the M SPAD pixel units are connected to the target digital counter via the same output wire.

[0045] For example, referring to Figure 1 and Figure 2 , the M SPAD pixel units can be M SPAD pixel units 111 in the same column of the SPAD pixel array 110, and M is a positive integer greater than 1. The M SPAD pixel units in the same column are connected to one digital counter 120 and share the digital counter 120. The target digital counter can be the digital counter 120 connected to the M SPAD pixel units in the same column.

[0046] The digital counting periods allocated to the M SPAD pixel units by the target digital counter are mutually staggered, so that the M SPAD pixel units use the target digital counter separately.

[0047] In step 310, referring to Figure 1 , the clock and control circuit 130 can provide a clock signal and a control signal for each row of SPAD pixel units 111, respectively.

[0048] In actual applications, the SPAD pixel unit 111 can include a PMOS switch transistor, a SPAD, an inverter, and a pixel selection switch transistor SEL. The working principle of the SPAD pixel unit 111 can include: when the clock signal S CLKUnder modulation, the SPAD device continuously cycles between quenching and charging during exposure, generating an analog pulse signal; the inverter can be considered a 1-bit analog-to-digital converter, converting the analog pulse signal into a digital signal (i.e., a 1-bit digital signal). When the pixel selection switch transistor SEL receives the control signal S... SEL (This can be understood as the pixel output selection signal) After conversion, the resulting digital signal will be used as the pixel output signal S. PIX The signal is transmitted to the corresponding digital counter 120 via the output wire, so that the digital counter 120 can perform digital counting processing on the photons sensed by the SPAD device to obtain a digital counting signal (multi-bit digital signal).

[0049] In step 320, at clock signal S CLK Under the modulation, the SPAD device of the SPAD pixel unit 111 continuously cycles between quenching and charging under exposure, forming an analog pulse signal, and an inverter is used to convert the analog pulse signal into a digital signal.

[0050] In step 330, when the SPAD pixel unit 111 receives the control signal S SEL Then, the pixel selection switch transistor SEL is turned on to output the digital signal as the pixel output signal to the target digital counter.

[0051] In step 340, the target digital counter performs counting processing on the digital signal during the digital counting period allocated to the SPAD pixel unit to obtain the digital counting signal of the SPAD pixel unit.

[0052] The digital counting signal reflects the number of photons received by the SPAD pixel unit, and can be used to generate the target image.

[0053] The digital counting signal of the SPAD pixel unit output by the target digital counter can be scanned or transmitted to the signal processing circuit 140 for subsequent image signal processing and buffering, and finally transmitted to the outside of the image sensor through the port circuit 150.

[0054] In this embodiment, the target digital counter allocates staggered digital counting periods to different SPAD pixel units among the M SPAD pixel units. Furthermore, the staggered counting periods of the target digital counter for different SPAD pixel units among the M SPAD pixel units ensure that each SPAD pixel unit can independently use the digital counter's time for counting. Therefore, in application scenarios where the M SPAD pixel units of an image sensor share a single digital counter, it can be guaranteed that each SPAD pixel unit can independently use the digital counter's time for counting, achieving efficient operation of each SPAD pixel unit in the image sensor.

[0055] According to the method for controlling the image sensor provided in the embodiments of the present application, the image sensor comprises a single photon avalanche transistor (SPAD) pixel array and a target digital counter, M SPAD pixel units in the SPAD pixel array share the target digital counter, the M SPAD pixel units are connected with the target digital counter via the same output wire, one of the M SPAD pixel units acquires a clock signal and a control signal; M is a positive integer greater than 1; a SPAD pixel unit generates a digital signal based on the clock signal; the SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; the target digital counter performs counting processing on the digital signal in a digital counting period allocated to the SPAD pixel unit, and obtains a digital counting signal of the SPAD pixel unit; and the digital counting signal of the SPAD pixel unit is used to generate a target image; wherein the target digital counter allocates mutually staggered digital counting periods to different SPAD pixel units of the M SPAD pixel units. In this way, in the scenario where M SPAD pixel units share one target digital counter for counting in the image sensor, compared with the working mode of the global shutter in the related art, since the target digital counter of the embodiments of the present application allocates mutually staggered digital counting periods to different SPAD pixel units of the M SPAD pixel units, it is ensured that the M SPAD pixel units sharing the target digital counter for counting can perform counting processing using the target digital counter individually, and the effective operation of each SPAD pixel unit in the image sensor is realized.

[0056] In a specific embodiment, the method for controlling the image sensor provided in the embodiments of the present application can be applied in an image sensor with row parallel counting processing. For example, referring to Figure 1 , the image sensor 10 comprises a SPAD pixel array, the SPAD pixel array comprises M*N SPAD pixel units 111, M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; in the step 310, the M SPAD pixel units are located in the same column of the SPAD pixel array.

[0057] In other words, the M SPAD pixel units located in the same column of the SPAD pixel array share the same one of the N digital counters 120.

[0058] The digital counter assigns the digital counting time periods staggered with each other to the M different SPAD pixel units in the same column. Furthermore, the M different SPAD pixel units in the same column stagger with each other using the digital counting time period of the target digital counter, so as to ensure that each SPAD pixel unit can use the time of the digital counter alone for counting processing.

[0059] In this way, in the application scenario that the same column of SPAD pixel units of the image sensor share one digital counter, it can be ensured that each SPAD pixel unit can use the time of the digital counter alone for counting processing, and the effective operation of each SPAD pixel unit in the image sensor is realized.

[0060] In another specific embodiment, in order to improve the control efficiency of the SPAD pixel array, the embodiments of the present application can adopt a running mode similar to a rolling shutter, and control the M rows of SPAD pixel units row by row, and the N SPAD pixel units in the same row of the SPAD pixel array can operate synchronously.

[0061] For example, in a specific embodiment, before step 310, the control method of the image sensor provided by the embodiments of the present application can further include:

[0062] The clock signal and the control signal are provided to the M rows of SPAD pixel units in the SPAD pixel array row by row.

[0063] Reference Figure 1 The image sensor 10 can include M rows of SPAD pixel units, and the N SPAD pixel units in the same row share one clock signal and one control signal, so that the N SPAD pixel units in the same row of the SPAD pixel array can operate synchronously.

[0064] In the embodiments of the present application, the SPAD pixel units of each row (Row) operate synchronously in a unit time. For example, referring to Figure 4 The plurality of SPAD pixel units of the first row (Row 1) can operate synchronously in a unit time, the plurality of SPAD pixel units of the second row (Row 2) can operate synchronously in a unit time, and so on, and the plurality of SPAD pixel units of the Mth row (Row M) can operate synchronously in a unit time.

[0065] In this way, in the embodiments of the present application, the plurality of SPAD pixel units in the same row share one clock signal and one control signal, so as to ensure that the plurality of SPAD pixel units of each row (Row) operate synchronously in a unit time, and improve the control efficiency of the SPAD pixel array.

[0066] In a specific embodiment, in a rolling shutter-like operation mode, the image sensor can output the image signal required for outputting one frame of image in a unit time, which can refer to the operation time of each SPAD pixel unit 111 in the process of generating the image signal required for generating one frame of image by the image sensor.

[0067] For example, there are a first SPAD pixel unit and a second SPAD pixel unit in the M SPAD pixel units; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and the starting time of the unit time of the first SPAD pixel unit is different from the starting time of the unit time of the second SPAD pixel unit.

[0068] The unit time includes K cycles, and K is a positive integer; each cycle includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same.

[0069] The time difference is greater than or equal to the length of the digital counting period.

[0070] For example, for one SPAD pixel unit 111, the unit time (one frame time) can include K cycles, and K is a positive integer; each cycle includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same. Figure 4 Figure 5 For example, for one SPAD pixel unit 111, the unit time (one frame time) can include K cycles, and K is a positive integer; each cycle includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same.

[0071] In order to ensure that the digital counting periods of the M SPAD pixel units sharing the target digital counter are staggered, the unit times of the M SPAD pixel units sharing the target digital counter have a time difference in operation.

[0072] For example, there are a first SPAD pixel unit and a second SPAD pixel unit in the M SPAD pixel units; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and the starting time of the unit time of the first SPAD pixel unit is different from the starting time of the unit time of the second SPAD pixel unit.

[0073] The time difference is greater than or equal to the length of the digital counting period.

[0074] For example, in the SPAD exposure period, the SPAD pixel unit is in an exposure state and generates a digital signal under the modulation of a clock signal.

[0075] ​In the digital counting period, the digital signal output by the SPAD pixel unit is counted by the target digital counter, and a digital counting signal of the SPAD pixel unit is obtained.

[0076] In the embodiments of the present application, the time length of each period in the unit time is the same, and each period is independent of each other.

[0077] The time length of each period and the number K of periods can be adjusted according to different application scenarios, and the present application does not make specific limitations.

[0078] The length of the unit time of the SPAD pixel unit of each row can be the same, and the unit time of the SPAD pixel unit of adjacent rows has a time difference T (unit: second) in operation. The time difference is greater than or equal to the length of the digital counting period. For example, the SPAD pixel unit of the first row (Row 1) and the SPAD pixel unit of the second row (Row 2) have a time difference T in operation in a unit time, and this time difference T is used to ensure that the SPAD pixel unit of each row of the pixel layer uses the digital counter separately in each period.

[0079] For example, referring to Figure 5 , the first SPAD pixel unit can be the SPAD pixel unit of the first row (Row 1), and the second SPAD pixel unit can be the SPAD pixel unit of the second row (Row 2). The length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and the starting time of the unit time of the first SPAD pixel unit has a time difference T from the starting time of the unit time of the second SPAD pixel unit. The time difference is greater than or equal to the length of the digital counting period.

[0080] And. With the time difference T between the rows rolling in turn until the Mth row of SPAD pixel units completes the digital counting, all the rows of SPAD pixels complete the entire operation of one period.

[0081] For each period in the unit time, the digital counting periods of the M rows of SPAD pixel units in each period are staggered. For example, for the first period in the unit time, the digital counting periods of the M rows of SPAD pixel units in the first period can be staggered. Referring to Figure 5, the SPAD pixel unit of the first row (Row 1) runs in the first period, first exposes in the SPAD exposure period, generates a digital signal in the case of receiving a photon under exposure, and then enters the digital counting period of the SPAD pixel unit of the first row (Row 1) in the case of receiving a control signal, outputs the digital signal to the target counter in the digital counting period of the SPAD pixel unit of the first row (Row 1), and performs digital counting processing. The SPAD pixel unit of the first row (Row 1) is simultaneously waiting for the exposure of the SPAD pixel unit of the second row (Row 2) in the digital counting period of the SPAD pixel unit of the first row (Row 1), and after the digital counting period of the SPAD pixel unit of the first row (Row 1), enters the digital counting period of the SPAD pixel unit of the second row (Row 2), and performs the digital counting process of the SPAD pixel unit of the second row (Row 2). In this way, the SPAD pixel unit of the Mth row completes the digital counting in the first period, and all the SPAD pixel rows complete the entire operation of the first period.

[0082] In the embodiment of the present application, due to the time difference in the operation of each SPAD pixel unit per unit time, it is ensured that the target digital counter counts the SPAD pixel unit of a row alone without being occupied by the SPAD pixel unit of other rows.

[0083] In this way, all the SPAD pixel units can be effectively operated and output the counted photon number by one digital counter per unit time. Moreover, the number K of periods per unit time and the length of the SPAD exposure period in the period can be flexibly adjusted according to application requirements to meet the shooting requirements of complex scenes.

[0084] In a specific example, in order to ensure that the target digital counter counts the SPAD pixel unit of a row alone without being occupied by the SPAD pixel unit of other rows per period, the first SPAD pixel unit and the second SPAD pixel unit can be located in different rows of the SPAD pixel array, and the digital counting period of the first SPAD pixel unit in the i th period of the K periods is staggered with the digital counting period of the second SPAD pixel unit in the i th period of the K periods; 1≤i≤K.

[0085] For example, the digital counting period of the SPAD pixel unit of the first row (Row 1) in the first cycle is staggered with the digital counting period of the SPAD pixel unit of the second row (Row 2) in the first cycle, the digital counting period of the SPAD pixel unit of the second row (Row 2) in the first cycle is staggered with the digital counting period of the SPAD pixel unit of the third row (Row 3) in the first cycle, and so on, the digital counting period of the SPAD pixel unit of the (M-1)th row (Row M) in the first cycle is staggered with the digital counting period of the SPAD pixel unit of the Mth row (Row M) in the first cycle.

[0086] For example, the digital counting period of the SPAD pixel unit of the first row (Row 1) in the second cycle is staggered with the digital counting period of the SPAD pixel unit of the second row (Row 2) in the second cycle, the digital counting period of the SPAD pixel unit of the second row (Row 2) in the second cycle is staggered with the digital counting period of the SPAD pixel unit of the third row (Row 3) in the second cycle, and so on, the digital counting period of the SPAD pixel unit of the (M-1)th row (Row M) in the second cycle is staggered with the digital counting period of the SPAD pixel unit of the Mth row (Row M) in the second cycle.

[0087] In this way, since the digital counting periods of the SPAD pixel units in each cycle are staggered in operation, it is ensured that the target digital counter counts the SPAD pixel units of one row in each cycle without being occupied by the SPAD pixel units of other rows.

[0088] In addition, in a specific example, the digital counting periods in two adjacent cycles are staggered in operation, so as to ensure that the target digital counter counts the SPAD pixel units of one row in two adjacent cycles without being occupied by the SPAD pixel units of other rows.

[0089] For example, the first SPAD pixel unit is located in the first row of the SPAD pixel array, and the second SPAD pixel unit is located in the last row of the SPAD pixel array; the digital counting period of the first SPAD pixel unit in the (j+1)th cycle of K cycles is located in time after the digital counting period of the second SPAD pixel unit in the jth cycle of K cycles; 1≤j

[0090] For example, the first SPAD pixel unit is located in the first pixel row, and the second SPAD pixel unit is located in the last pixel row. The SPAD pixel unit in the first row (Row 1) is in the digital counting period in the second cycle after the digital counting period in the first cycle of the SPAD pixel unit in the last row (Row M).

[0091] Specifically, referring to Figure 5 When the digital counter completes the digital counting period in the first cycle of the SPAD pixel unit in the last row (Row M), the SPAD pixel unit in the first row (Row 1) can still be running in the SPAD exposure period in the second cycle, and there is a counting blanking time. After the counting blanking time, the SPAD pixel unit in the first row (Row 1) runs in the digital counting period in the second cycle. The counting blanking time is located after the completion of the digital counting period in the first cycle of the SPAD pixel unit in the last row (Row M) and before the start of the digital counting period in the second cycle of the SPAD pixel unit in the first row (Row 1).

[0092] In other embodiments, if the SPAD exposure period of each row of SPAD pixel units is adjusted to be short enough, this counting blanking time can not exist, ensuring that the digital counting period in the second cycle of the SPAD pixel unit in the first row (Row 1) is just after the digital counting period in the first cycle of the SPAD pixel unit in the last row (Row M).

[0093] In this way, the digital counting periods of the SPAD pixel units in different pixel rows in adjacent two cycles are staggered in operation, ensuring that the target digital counter separately counts the digital counts of a row of SPAD pixel units in adjacent two cycles without being occupied by other rows of SPAD pixel units.

[0094] In actual applications, for some large light ratio shooting scenes, the image sensor can implement high dynamic range (High Dynamic Range, HDR) related technologies, such as a multi-frame fusion technology. In the operation method of the image sensor adopted in the present application, the multi-frame fusion technology is realized by coordination between different cycles in unit time.

[0095] The embodiments of the present application can adjust the position of the digital counting period in each cycle in the entire cycle to achieve the length modulation of the exposure period of the SPAD pixel unit in unit time in at least two cycles as a group.

[0096] For example, in a unit time, there are a first period and a second period in K periods, the first period includes a first digital counting period, and the second period includes a second digital counting period, the position of the first digital counting period in the first period is different from the position of the second digital counting period in the second period.

[0097] The first period further includes a first SPAD exposure period, and the second period further includes a second SPAD exposure period, the length of the first SPAD exposure period is different from the length of the second SPAD exposure period.

[0098] In a specific embodiment, referring to Figure 5 , in the K periods, the positions of the digital counting periods in the respective periods are the same, and are all located at the end of the period.

[0099] In the embodiment of the present application, the position of the first digital counting period in the first period is adjusted, so that the position of the first digital counting period in the first period is different from the position of the second digital counting period in the second period. Alternatively, the position of the second digital counting period in the second period is adjusted, so that the position of the first digital counting period in the first period is different from the position of the second digital counting period in the second period. Furthermore, the length of the SPAD period in the first period and the second period is modulated to implement the HDR technology.

[0100] In a specific embodiment, the first period includes odd periods in the K periods, and the second period includes even periods in the K periods, the first digital counting period is located at a first position of the first period, and the second digital counting period is located at a second position of the second period, the first position is one of a tail position and a middle position, and the second position is the other of the tail position and the middle position.

[0101] In the embodiment of the present application, by controlling the position of the digital counting period in the first period / second period, the SPAD pixel unit can implement different long-short exposure combinations, and the length of each period remains consistent.

[0102] In this way, two adjacent periods in a unit time can be taken as a group, and the position of the digital counting period in the period is adjusted to achieve the long-short modulation of the exposure period of the SPAD pixel unit.

[0103] For example, as Figure 7 shown, in the odd period (1st, 3rd, 5th...) period, the digital counting period is located at the end of the period. In the even period (2nd, 4th, 6th...), the digital counting period is located in the middle of the period. Figure 7As shown, the number counting period of Row M of the odd cycle immediately enters the number counting period of Row 1 of the even cycle after completion, which is one of the ways. If necessary, there can also be a certain counting blank period.

[0104] The exposure period of the SPAD pixel unit is configured in length and short, including a long exposure period and a short exposure period. The SPAD long exposure period is composed of an odd cycle and a part of an even cycle period. The SPAD short exposure period is composed of a part of an even cycle period. In this way, the long and short exposure modulation of the SPAD pixel unit per unit time can be achieved under the condition that the long and short of all cycles are consistent, so as to achieve the implementation of the HDR technology.

[0105] Of course, in addition to Figure 7 As shown, two cycles are taken as a group, and the position of the number counting period in each cycle is adjusted to achieve the long and short modulation of the exposure period of the SPAD pixel unit in the whole cycle. The embodiment of the present application can also take more than two cycles as a group to form a multi-exposure time modulation to achieve more complex and refined HDR technology.

[0106] In this way, by modulating the position of the number counting period in each cycle, the long and short exposure modulation of the SPAD pixel unit per unit time can be achieved to implement the HDR technology to cope with complex shooting scenes.

[0107] In the embodiment of the present application, as Figure 6 As shown, the unit time of the SPAD pixel unit can also include a signal processing period, which is located after the K cycles; the signal processing period is used for processing the digital counting signal output by the number counting period in the K cycles;

[0108] After step 340, the control method provided by the embodiment of the present application can further include:

[0109] In the signal processing period, the digital counting signal output by the number counting period in the K cycles is processed to obtain an image signal, and the image signal is output.

[0110] In this embodiment, the unit time may further include a signal processing period. The unit time begins with the first cycle and enters the signal processing period after the Kth cycle is completed. The unit time ends and the next unit time begins after the signal processing period is completed. In one embodiment, for a SPAD pixel array with M rows of SPAD pixel units, each unit time of the image sensor includes K cycles. Each cycle consists of a SPAD exposure period and a digital counting period. During the SPAD exposure period, the SPAD devices within the SPAD pixel units are exposed and detect incident photons, generating a digital signal. During the digital counting period, the SPAD pixel units located in the pixel layer output digital signals to a digital counter located in the logic layer for counting, obtaining a digital counting signal. The digital counting signal can be buffered in the subsequent signal processing circuit 140 for image signal processing, or it can be directly scanned and output by the port circuit 150.

[0111] For example, after completing K cycles, the SPAD pixel unit can enter the signal processing period. This signal processing period can be used by the signal processing circuit 140, port circuit 150, or other auxiliary circuits to perform image signal processing (such as digital magnification, bad pixel screening, signal output processing, etc.), circuit reset, and preparation for the start of the next frame.

[0112] Specifically, within a unit of time, the signal processing period begins after the Kth cycle is completed. For example... Figure 6 As shown, the signal processing period does not overlap with the digital counting period of the Kth cycle. During the signal processing period, the signal processing circuit located in the logic layer completes signal processing, signal transmission, and necessary reset steps, preparing for the start of the first cycle of the next unit of time. Similarly, the signal processing period for each SPAD pixel unit also has a time difference T, and the size of T is at least one digital counting period. The signal processing circuit may process multiple lines of image signals during the signal processing period of a single line.

[0113] In this way, image signal processing is performed by the signal processing circuitry of the logic layer, and its signal processing phase is placed after all cycles and is independent of the operation within each cycle. Therefore, interference and crosstalk between the signal processing circuitry and the SPAD pixel unit operation in the pixel layer can be effectively reduced. Furthermore, the signal processing circuitry can be ignored, improving flexibility.

[0114] Based on the same concept as the control method of the image sensor provided in any of the above embodiments, this application also provides an image sensor that can be used to implement the control method of the image sensor provided in any of the above embodiments.

[0115] For example, refer to Figure 1 andFigure 2 The image sensor 10 provided by the embodiment of the present application can include a pixel layer 100 and a logic layer 200 arranged in a stack;

[0116] The pixel layer 100 is provided with an SPAD pixel array 110, the SPAD pixel array 110 including M*N SPAD pixel units 111, M being the number of rows, N being the number of columns, M being a positive integer greater than 1, and N being a positive integer;

[0117] The logic layer 200 is provided with N digital counters 120, a clock and control circuit 130, a signal processing circuit 140, and a port circuit 150;

[0118] The M SPAD pixel units 111 in the same column in the SPAD pixel array are connected to the same digital counter 120 of the N digital counters.

[0119] The SPAD pixel unit 111 can include a PMOS switch transistor, a SPAD, an inverter, and a pixel selection switch transistor SEL. The working principle of the SPAD pixel unit 111 includes: under the modulation of a clock signal S CLK , the SPAD device continuously cycles between quenching and charging under exposure, forming an analog pulse signal; the inverter converts the analog pulse signal into a digital signal. When the pixel selection switch transistor SEL receives a control signal S SEL (which can be understood as a pixel output selection signal), the converted digital signal is output as a pixel output signal S PIX to the corresponding digital counter 120 through an output wire, so that the digital counter 120 performs digital counting processing on the photons sensed by the SPAD device within the digital counting period allocated to the SPAD pixel unit.

[0120] Each photon received by the SPAD is counted once by the digital counter, and X photons reaching the SPAD are counted X times by the digital counter.

[0121] The PMOS switch transistor can be equivalent to a variable resistor, which controls the size of the analog pulse signal flowing through the SPAD to prevent the peak value of the analog pulse signal from being too large or too small.

[0122] In the embodiment of the present application, the digital counter assigns different SPAD pixel units in the same column with mutually staggered digital counting periods. Furthermore, different SPAD pixel units in the plurality of SPAD pixel units use the digital counting period of the target digital counter to stagger with each other, so as to ensure that each SPAD pixel unit can use the time of the digital counter for counting processing. Furthermore, in the application scenario where the plurality of SPAD pixel units of the image sensor share one digital counter, it can be ensured that each SPAD pixel unit can use the time of the digital counter for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor.

[0123] It should be noted that in actual application, the number and type of the target digital counter are not specifically limited in the embodiment of the present application. The type of the target digital counter can be a single-input digital counter or a multi-input digital counter, and the number of the target digital counter can be one or more. The type and number of the target digital counter can be set according to actual needs in the embodiment of the present application.

[0124] For example, in other embodiments (not shown in the figure), the target digital counter can be a multi-input multi-output digital counter, and the number of the target digital counter can be one. For example, the target digital counter can include N input terminals, and the target digital counter can be used to count the pixel output signals input by the N input terminals at the same time. The SPAD pixel array can include M*N SPAD pixel units, M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; wherein the M SPAD pixel units in the first column are connected to the first input terminal of the target digital counter through the same output wire, the M SPAD pixel units in the second column are connected to the second input terminal of the target digital counter through the same output wire, and so on, and the M SPAD pixel units in the Nth column are connected to the Nth input terminal of the target digital counter through the same output wire.

[0125] The target digital counter can be used to count the photons sensed by the M SPAD pixel units in the first column, count the photons sensed by the M SPAD pixel units in the second column, and so on, and count the photons sensed by each SPAD pixel unit in the Nth column, respectively.

[0126] Furthermore, the target digital counter can be used to count the pixel output signals from different input terminals synchronously, for example, the target digital counter can be used to count the photons sensed by the N SPAD pixel units in the first row synchronously, count the photons sensed by the N SPAD pixel units in the second row synchronously, and so on, and count the photons sensed by the N SPAD pixel units in the Mth row synchronously.

[0127] In this way, by means of the M SPAD pixel units in the same column accessing the same input end of the target digital counter through the same output wire, and the SPAD pixel units in different columns accessing different input ends of the target digital counter, the M*N SPAD pixel units can share one target digital counter with N input ends for counting, and effective operation of the M*N SPAD pixel units is realized.

[0128] It should be noted that the image sensor can be used to implement the control method of the image sensor provided in any of the above embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0129] Based on the same concept as the image sensor provided in any of the above embodiments, the embodiments of the present application also provide an electronic device.

[0130] Reference Figure 8 The embodiments of the present application provide an electronic device 20, which can include the image sensor 10 provided in any of the above embodiments.

[0131] In actual application, the electronic device 20 can include at least one CCM (Compact Camera Module, compressed camera module) module, each CCM module can include one image sensor 10, and the image sensor performs bidirectional communication with an ISP (Image Signal Processor, image signal processing) module in an AP (Application Processor, application processor) or SoC (System on Chip, system chip) chip through a port link. The AP / SoC sends a control signal to the image sensor in each CCM module, and the image sensor generates an image signal and returns it to the ISP module in the AP / SoC for backend processing.

[0132] In the embodiments of the present application, the electronic device 20 can be a device with a camera function. The electronic device can be a terminal, or can be other devices than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), etc., and the embodiments of the present application are not limited in this way.

[0133] In the description of the present specification, the description referring to the terms "one embodiment", "some embodiments", "exemplary embodiment", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the exemplary description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0134] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the claims and their equivalents.

Claims

1. A control method of an image sensor, characterized by, The image sensor comprises a single-photon avalanche transistor (SPAD) pixel array and a target digital counter, M SPAD pixel units in the SPAD pixel array share the target digital counter, the M SPAD pixel units are connected with the target digital counter via the same output wire, and the method comprises the following steps: One of the M SPAD pixel units acquires a clock signal and a control signal; M is a positive integer greater than 1; The SPAD pixel unit generates a digital signal based on the clock signal; The SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; The target digital counter performs counting processing on the digital signal within a digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image; and the digital counting signal of the SPAD pixel unit is used to reflect the number of photons received by the SPAD pixel unit; The target digital counter allocates mutually staggered digital counting periods to different SPAD pixel units of the M SPAD pixel units.

2. The method of claim 1, wherein, The SPAD pixel array comprises M*N SPAD pixel units, M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; the M SPAD pixel units are located in the same column of the SPAD pixel array.

3. The method of claim 2, wherein, Before one of the M SPAD pixel units acquires a clock signal and a control signal, the method further comprises the following steps: The clock signal and the control signal are provided to the M rows of SPAD pixel units in the SPAD pixel array in a row-by-row manner.

4. The method of claim 3, wherein, The number of target digital counters is N, all SPAD pixel units located in the same column of the SPAD pixel array share the same target digital counter, and SPAD pixel units in different columns share different target digital counters; the SPAD pixel units located in the same row operate synchronously within a unit time.

5. The method of controlling an image sensor according to any one of claims 1 to 4, characterized by, The M SPAD pixel units comprise a first SPAD pixel unit and a second SPAD pixel unit; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and there is a time difference between the starting time of the unit time of the first SPAD pixel unit and the starting time of the unit time of the second SPAD pixel unit; The unit time comprises K periods, and K is a positive integer; each period of the K periods comprises a SPAD exposure period and a digital counting period, and the length of the digital counting period in each period is the same; The time difference is greater than or equal to the length of the digital counting period.

6. The method of controlling an image sensor according to claim 5, wherein The first SPAD pixel unit and the second SPAD pixel unit are located in different rows of the SPAD pixel array, and a digital counting period of the first SPAD pixel unit in an i-th period of the K periods is staggered with a digital counting period of the second SPAD pixel unit in the i-th period of the K periods; 1≤i≤K.

7. The method of controlling an image sensor according to claim 5, wherein The first SPAD pixel unit is located in a first row of the SPAD pixel array, and the second SPAD pixel unit is located in a last row of the SPAD pixel array; a digital counting period of the first SPAD pixel unit in a j+1-th period of the K periods is located in time after a digital counting period of the second SPAD pixel unit in a j-th period of the K periods; 1≤j<K.

8. The method of controlling an image sensor according to claim 5, wherein The K periods include a first period and a second period, the first period includes a first digital counting period, and the second period includes a second digital counting period; a position of the first digital counting period in the first period is different from a position of the second digital counting period in the second period.

9. The method of controlling an image sensor according to claim 8, wherein The first period includes an odd period of the K periods, the second period includes an even period of the K periods, the first digital counting period is located at a first position of the first period, and the second digital counting period is located at a second position of the second period; the first position is one of a tail position and a middle position, and the second position is the other of the tail position and the middle position.

10. The method of controlling an image sensor according to claim 5, wherein The unit time further includes a signal processing period, and the signal processing period is located after the K periods; The signal processing period is used for processing the digital counting signal output by the digital counting period in the K periods; The method further includes: In the signal processing period, the digital counting signal output by the digital counting period in the K periods is processed to obtain an image signal, and the image signal is output.

11. An image sensor, comprising: The image sensor is used to implement the control method according to any one of claims 1-10.

12. The image sensor of claim 11, wherein, The image sensor includes a pixel layer and a logic layer arranged in a stack; The pixel layer is provided with a SPAD pixel array, the SPAD pixel array includes M*N SPAD pixel units, M is a row number, N is a column number, M is a positive integer greater than 1, and N is a positive integer; The SPAD pixel unit includes a P-channel metal oxide semiconductor (PMOS) switch transistor, a SPAD, an inverter, and a pixel selection switch transistor; The logic layer is provided with N digital counters, a clock and control circuit, a signal processing circuit, and a port circuit; M SPAD pixel units located in the same column in the SPAD pixel array are connected to the same digital counter of the N digital counters through the same output wire.

13. An electronic device, comprising: The image sensor includes the image sensor according to claim 11.

Images