Control method of image sensor, image sensor and electronic equipment
By allocating staggered digital counting periods to the SPAD pixel units in the image sensor, the problem that the global shutter working mode cannot be effectively operated is solved, and independent counting and effective operation of the SPAD pixel units in the image sensor is realized, and resolution and optics are improved performance.
Patent Information
- Application Number
- CN202411196889.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-08-29
AI Technical Summary
The existing global shutter working mode cannot operate effectively in CMOS single-photon image sensors, especially in multiple SPAD pixel units scenarios, making it difficult for SPAD pixel units in the image sensor to synchronize and use the target digital counter for independent counting.
By adopting an image sensor control method, each SPAD pixel unit is assigned to each other in a SPAD pixel array, so that the SPAD pixel units in the same column and different rows are staggered with each other using a digital counter, thereby realizing that Independent counting processing of each SPAD pixel unit.
It ensures that each SPAD pixel unit in the image sensor can be counted separately using the time of the target digital counter, realizes the effective operation of each SPAD pixel unit in the image sensor, and improves the effective resolution and optical performance of the image sensor.
Smart Images

Figure CN119071655A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of electronic technology, and specifically relates to a control method of an image sensor, an image sensor and an electronic device. Background Art
[0002] In CMOS image sensors (Complementary Metal-Oxide Semiconductor Image Sensors, CIS), SPAD (Single-Photon Avalanche Diode) devices can be used as pixel photosensors of image sensors. SPAD pixel units have ultra-high sensitivity at the single-photon sensing level and have significant imaging capabilities.
[0003] In the related art, CMOS image sensors can use a global shutter working mode to synchronously control and operate each SPAD pixel unit in the SPAD pixel array of the CMOS image sensor. However, in the scenario where there are multiple SPAD pixel units in the CMOS single-photon image sensor, the existing global shutter working mode may not work effectively. Summary of the invention
[0004] The embodiments of the present application provide a control method of an image sensor, an image sensor, and an electronic device, which solve the technical problem in the related art that the existing working mode of the global shutter may not operate effectively.
[0005] In a first aspect, an embodiment of the present application provides a control method for an image sensor, wherein the image sensor includes a SPAD pixel array and a target digital counter, wherein there are M SPAD pixel units in the SPAD pixel array that share the target digital counter, and the M SPAD pixel units are connected to the target digital counter via the same output wire, and the control method includes:
[0006] One SPAD pixel unit among the M SPAD pixel units obtains a clock signal and a control signal; M is a positive integer greater than 1;
[0007] The SPAD pixel unit generates a digital signal based on the clock signal;
[0008] The SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal;
[0009] The target digital counter performs counting processing on the digital signal within the digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image;
[0010] The target digital counter allocates mutually staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units.
[0011] In a second aspect, an embodiment of the present application proposes an image sensor, which is used to implement the control method as described in the first aspect.
[0012] In a third aspect, an embodiment of the present application provides an electronic device, comprising the image sensor as described in the second aspect.
[0013] In an embodiment of the present application, an image sensor includes a single-photon avalanche transistor SPAD pixel array and a target digital counter, wherein there are M SPAD pixel units in the SPAD pixel array that share the target digital counter, and the M SPAD pixel units are connected to the target digital counter via the same output wire, and one SPAD pixel unit among the M SPAD pixel units obtains a clock signal and a control signal; M is a positive integer greater than 1; the SPAD pixel unit generates a digital signal based on the clock signal; the SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; the target digital counter counts the digital signal within a digital counting period assigned to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image; wherein the target digital counter assigns staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units. In this way, in a scenario where there are M SPAD pixel units in the image sensor that share a target digital counter for counting, compared with the related art in which the global shutter may not be able to operate effectively, since the target digital counter in the embodiment of the present application assigns staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units, it is ensured that the M SPAD pixel units that share the target digital counter for counting can use the time of the target digital counter separately for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0015] Figure 1 is a schematic diagram of an image sensor provided in an embodiment of the present application;
[0016] Figure 2 is a schematic diagram of another image sensor provided in an embodiment of the present application;
[0017] Figure 3 is a schematic flow chart of a control method of an image sensor provided in an embodiment of the present application;
[0018] Figure 4 is a schematic diagram of a timeline of an operating mode of an image sensor per unit time provided by an embodiment of the present application;
[0019] Figure 5 is a schematic diagram of another operating mode timeline of an image sensor per unit time provided by an embodiment of the present application;
[0020] Figure 6 is a schematic diagram of another operating mode timeline of an image sensor per unit time provided by an embodiment of the present application;
[0021] Figure 7 is a schematic diagram of another operating mode timeline of an image sensor per unit time provided by an embodiment of the present application;
[0022] Figure 8 It is a schematic diagram of an electronic device provided in an embodiment of the present application.
[0023] Description of reference numerals:
[0024] 10-image sensor; 100-pixel layer; 200-logic layer; 300-connection point; 400-input / output pin; 110-SPAD pixel array; 111-SPAD pixel unit; 120-digital counter; 130-clock and control circuit; 140-signal processing circuit; 150-port circuit. DETAILED DESCRIPTION
[0025] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as limitations on the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.
[0026] The term "first" or "second" in the specification and claims of this application may include one or more of the features explicitly or implicitly. In the description of this application, unless otherwise specified, "plurality" means two or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / " generally means that the objects connected before and after are in an "or" relationship.
[0027] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal connection of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0028] In the related art, each SPAD pixel unit of the image sensor includes a clock and control circuit, a digital counter, and a digital readout circuit. When the excitation signal reaches the clock and control circuit, the clock and control circuit controls the quenching and charging of the SPAD device in the SPAD pixel unit and is counted by the digital counter. The counting result is read by the digital readout circuit and output as a pixel output signal. Based on the working principle of the above-mentioned SPAD pixel unit, the SPAD-based CMOS single-photon image sensor adopts a global shutter working mode to synchronously control and operate each SPAD pixel unit in the SPAD pixel array.
[0029] See also Figure 1 The image sensor 10 provided by the present application removes the digital counter 120 from the SPAD pixel unit 111, and adopts a column-parallel digital counter on the logic layer 200. Each column of SPAD pixel units in the pixel layer 100 shares an output wire, and this output wire is electrically connected to the corresponding digital counter 120 located on the logic layer 200. The connection point 300 of this output wire in the cross-layer area of the chip adopts Cu-Cu (copper-copper) bonding or TSV (Trans-Silicon Via, through silicon via) process.
[0030] Among them, at least one digital counter 120, a clock and control circuit 130, a signal processing circuit 140, a port circuit 150 and other auxiliary circuits are arranged on the logic layer 200. Each row (Row) of SPAD pixel units 111 of the pixel layer 100 shares a clock and control signal. The clock and control signals generated by the clock and control circuit 130 reach a row of SPAD pixel units corresponding to the pixel layer through wires. Similarly, the connection points 300 of the wires in the cross-layer area of the chip adopt Cu-Cu bonding or TSV technology. The clock and control circuit 130 can control and transmit signals to any row or multiple rows of SPAD pixel units.
[0031] The output signal (multi-bit digital signal) of each digital counter is scanned or transmitted to the signal processing circuit 140 for subsequent image signal processing and buffering, and finally transmitted to the outside of the image sensor 10 through the port circuit 150 and the input / output pin 400.
[0032] In the embodiment of the present application, by removing the digital counter 120 from the SPAD pixel unit 111 and setting the digital counter 120 separately in the logic layer 200, the complexity of the circuit in the SPAD pixel unit 111 is greatly reduced, and the unit pixel area is reduced. At the same time, the digital counter 120 on the logic layer 200 adopts a column parallel design, and each digital counter 120 can be shared by multiple SPAD pixel units 111 in the same column, reducing the number of digital counters 120 required for the image sensor 10. Furthermore, more SPAD pixel units can be arranged per unit chip area, thereby improving the effective resolution of the image sensor.
[0033] In addition, refer to Figure 2 In the image sensor provided by the present application, a SPAD pixel array 110 composed of a plurality of SPAD pixel units 111 is placed in a pixel layer 100. Compared with the related art, the pixel circuit array corresponding to the SPAD array in the logic layer is eliminated. The saved space can be used to place digital counters and auxiliary circuits such as signal processing circuits and port circuits, so that the layout area of the logic layer and the pixel layer is consistent, thereby improving the optical performance of the image sensor in the CCM module.
[0034] It should be pointed out that since the multiple SPAD pixel units 111 in the same column of the image sensor 10 provided in the embodiment of the present application need to share a digital counter 120 for counting, the multiple SPAD pixel units 111 sharing a digital counter 120 are difficult to operate synchronously (that is, it is difficult to synchronously use a digital counter for independent counting), resulting in the existing global shutter working mode being unable to operate effectively.
[0035] Based on this, the present application provides a control method for an image sensor. The present application adopts an operating mode similar to a rolling shutter to control the M rows of SPAD pixel units of the image sensor row by row, and allocates staggered digital counting time periods to each row of SPAD pixel units, so that the SPAD pixel units in the same column but different rows use the digital counting time periods of the digital counter staggered, ensuring that each SPAD pixel unit in the same column but different row can use the time of the digital counter separately for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor.
[0036] Figure 3 It is a schematic diagram of a control method of an image sensor provided in an embodiment of the present application.
[0037] like Figure 3 As shown, the control method of the image sensor provided in the embodiment of the present application may include:
[0038] Step 310: One SPAD pixel unit among the M SPAD pixel units obtains a clock signal and a control signal;
[0039] Step 320: The SPAD pixel unit generates a digital signal based on the clock signal;
[0040] Step 330: The SPAD pixel unit outputs a digital signal to a target digital counter based on the control signal;
[0041] Step 340: The target digital counter performs counting processing on the digital signal within the digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit;
[0042] The target digital counter allocates mutually staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units.
[0043] Among them, the digital counting signal of the SPAD pixel unit is used to generate the target image.
[0044] In an embodiment of the present application, the image sensor includes a SPAD pixel array and a target digital counter. The SPAD pixel array includes M SPAD pixel units that share a common target digital counter. The M SPAD pixel units are connected to the target digital counter via the same output wire.
[0045] For example, refer to Figure 1 and Figure 2 , the M SPAD pixel units may be the M SPAD pixel units 111 located in the same column in the SPAD pixel array 110; M is a positive integer greater than 1. The M SPAD pixel units located in the same column are connected to a digital counter 120 and share the digital counter 120. The target digital counter may be a digital counter 120 connected to the M SPAD pixel units located in the same column.
[0046] The digital counting periods allocated by the target digital counter to the M SPAD pixel units are staggered, so that the M SPAD pixel units use the time of the target digital counter independently.
[0047] In step 310, referring to Figure 1 The clock and control circuit 130 can provide a clock signal and a control signal to each row of SPAD pixel units 111 respectively.
[0048] In practical applications, the SPAD pixel unit 111 may include: a PMOS switch transistor, a SPAD, an inverter, and a pixel selection switch transistor SEL. The working principle of the SPAD pixel unit 111 may include: CLKUnder the modulation of , the SPAD device continuously cycles between quenching and charging under exposure, forming an analog pulse signal; the inverter can be regarded as a 1-bit analog-to-digital converter, which converts the analog pulse signal into a digital signal (i.e., a 1-bit digital signal). When the pixel selection switch transistor SEL receives the control signal S SEL (which can be understood as a pixel output selection signal), the converted digital signal will be used as the pixel output signal S PIX The photons are transmitted to the corresponding digital counter 120 through the output wire, so that the digital counter 120 performs digital counting processing on the photons sensed by the SPAD device to obtain a digital counting signal (multi-bit digital signal).
[0049] In step 320, when the clock signal S CLK Under the modulation, the SPAD device of the SPAD pixel unit 111 continuously cycles between quenching and charging under exposure to form an analog pulse signal, and an inverter is used to convert the analog pulse signal into a digital signal.
[0050] In step 330, the SPAD pixel unit 111 receives a control signal S SEL After that, the pixel selection switch transistor SEL is controlled to be turned on to output the digital signal as a pixel output signal to the target digital counter.
[0051] In step 340 , the target digital counter performs counting processing on the digital signal within the digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit.
[0052] The digital counting signal may reflect the number of photons received by the SPAD pixel unit, and the digital counting signal may be subsequently used to generate a target image.
[0053] The digital counting signal of the SPAD pixel unit output by the target digital counter may be scanned or transmitted to the signal processing circuit 140 for subsequent image signal processing and buffering, and finally transmitted to the outside of the image sensor through the port circuit 150 .
[0054] In an embodiment of the present application, the target digital counter allocates staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units. Furthermore, different SPAD pixel units among the M SPAD pixel units use staggered digital counting time periods of the target digital counter, ensuring that each SPAD pixel unit can use the time of the digital counter alone for counting processing. Furthermore, in an application scenario where the M SPAD pixel units of the image sensor share one digital counter, it can be ensured that each SPAD pixel unit can use the time of the digital counter alone for counting processing, thereby achieving effective operation of each SPAD pixel unit in the image sensor.
[0055] According to the control method of the image sensor provided by the embodiment of the present application, the image sensor includes a single-photon avalanche transistor SPAD pixel array and a target digital counter, wherein there are M SPAD pixel units in the SPAD pixel array that share the target digital counter, the M SPAD pixel units are connected to the target digital counter via the same output wire, and one SPAD pixel unit among the M SPAD pixel units obtains a clock signal and a control signal; M is a positive integer greater than 1; the SPAD pixel unit generates a digital signal based on the clock signal; the SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; the target digital counter counts the digital signal within a digital counting period assigned to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image; wherein the target digital counter assigns staggered digital counting periods to different SPAD pixel units among the M SPAD pixel units. In this way, in a scenario where there are M SPAD pixel units in the image sensor that share a target digital counter for counting, compared with the related art in which the global shutter may not be able to operate effectively, since the target digital counter in the embodiment of the present application assigns staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units, it is ensured that the M SPAD pixel units that share the target digital counter for counting can use the time of the target digital counter separately for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor.
[0056] In a specific embodiment, the control method of the image sensor provided in the embodiment of the present application can be applied to an image sensor with row parallel counting processing. Figure 1 The image sensor 10 includes a SPAD pixel array, the SPAD pixel array includes M*N SPAD pixel units 111, M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; in the above step 310, the M SPAD pixel units are located in the same column of the SPAD pixel array.
[0057] In other words, the M SPAD pixel units in the same column of the SPAD pixel array and the same digital counter 120 among the N digital counters share the same digital counter.
[0058] The digital counter allocates staggered digital counting periods to the M different SPAD pixel units in the same column. Furthermore, the M different SPAD pixel units in the same column use staggered digital counting periods of the target digital counter, ensuring that each SPAD pixel unit can use the digital counter time alone for counting processing.
[0059] In this way, in the application scenario where the SPAD pixel units in the same column of the image sensor share a digital counter, it can be ensured that each SPAD pixel unit can use the time of the digital counter independently for counting processing, thereby realizing the effective operation of each SPAD pixel unit in the image sensor.
[0060] In another specific embodiment, in order to improve the control efficiency of the SPAD pixel array, the embodiment of the present application can adopt an operating mode similar to a rolling shutter to control M rows of SPAD pixel units row by row, and N SPAD pixel units located in the same row of the SPAD pixel array can operate synchronously.
[0061] For example, in a specific embodiment, before step 310, the control method of the image sensor provided by the embodiment of the present application may further include:
[0062] Clock signals and control signals are provided to M rows of SPAD pixel units in the SPAD pixel array in a row-by-row manner.
[0063] refer to Figure 1 The image sensor 10 may include M rows of SPAD pixel units, the N SPAD pixel units in the same row share a clock signal, and the N SPAD pixel units in the same row share a control signal, so that the N SPAD pixel units in the same row of the SPAD pixel array can operate synchronously.
[0064] In the embodiment of the present application, each row of SPAD pixel units operates synchronously within a unit time. Figure 4 , multiple SPAD pixel units in the first row (Row 1) can operate synchronously within a unit time, multiple SPAD pixel units in the second row (Row 2) can operate synchronously within a unit time, and so on, multiple SPAD pixel units in the Mth row (Row M) can operate synchronously within a unit time.
[0065] In this way, in the embodiment of the present application, multiple SPAD pixel units located in the same row share a clock signal and a control signal, ensuring that multiple SPAD pixel units in each row (Row) operate synchronously within a unit time, thereby improving the control efficiency of the SPAD pixel array.
[0066] In a specific embodiment, in an operating mode similar to a rolling shutter, the image sensor can output an image signal required for a frame of image within a unit time, and the unit time can refer to the operating time of each SPAD pixel unit 111 in the process of the image sensor generating the image signal required for a frame of image.
[0067] For example, there are a first SPAD pixel unit and a second SPAD pixel unit among the M SPAD pixel units; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and there is a time difference between the starting moment of the unit time of the first SPAD pixel unit and the starting moment of the unit time of the second SPAD pixel unit;
[0068] The unit time includes K cycles, where K is a positive integer; each of the K cycles includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same;
[0069] The time difference is greater than or equal to the length of the digital counting period.
[0070] Among them, Figure 4 and Figure 5 For example, for a SPAD pixel unit 111, a unit time (a frame time) may include K cycles, where K is a positive integer; each of the K cycles includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same.
[0071] In order to ensure that the digital counting time periods of the M SPAD pixel units that share the target digital counter are staggered, there is a time difference in the operation of the unit time of the M SPAD pixel units that share the target digital counter.
[0072] For example, there are a first SPAD pixel unit and a second SPAD pixel unit among the M SPAD pixel units; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and there is a time difference between the starting time of the unit time of the first SPAD pixel unit and the starting time of the unit time of the second SPAD pixel unit;
[0073] The time difference is greater than or equal to the length of the digital counting period.
[0074] Wherein, during the SPAD exposure period, the SPAD pixel unit generates a digital signal under the exposure state and modulation of the clock signal;
[0075] Wherein, during the digital counting period, the target digital counter counts the digital signal output by the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit.
[0076] In the embodiment of the present application, among the K cycles within a unit time, the time length of each cycle is the same, and each cycle is independent of each other.
[0077] The time length of each cycle and the number of cycles K can be adjusted according to different application scenarios, and this application does not impose any specific restrictions on this.
[0078] The length of the unit time of the SPAD pixel units in each row may be the same, and there is a time difference T (in seconds) in the operation of the unit time of the SPAD pixel units in adjacent rows, and the time difference is greater than or equal to the length of the digital counting period. For example, there is a time difference T in the operation of the SPAD pixel units in the first row (Row 1) and the SPAD pixel units in the second row (Row 2) within a unit time, and this time difference T is used to ensure that the SPAD pixel units in each row of the pixel layer use the time of the digital counter separately in each cycle.
[0079] For example, refer to Figure 5 , the first SPAD pixel unit may be a SPAD pixel unit of the first row (Row 1), the second SPAD pixel unit may be a SPAD pixel unit of the second row (Row 2), the length of the unit time of the SPAD pixel units of the first row (Row 1) is the same as the length of the unit time of the SPAD pixel units of the second row (Row 2), and there is a time difference T between the starting moment of the unit time of the first SPAD pixel unit and the starting moment of the unit time of the second SPAD pixel unit, and the time difference is greater than or equal to the length of the digital counting period.
[0080] And, as the time difference T between rows rolls in sequence until the SPAD pixel units in the Mth row complete digital counting, all SPAD pixel rows complete the entire operation of one cycle.
[0081] For each cycle in the unit time, the digital counting periods of the M rows of SPAD pixel units in each cycle are staggered. For example, for the first cycle in the unit time, the digital counting periods of the M rows of SPAD pixel units in the first cycle can be staggered. Figure 5, the SPAD pixel units of the first row (Row1) operate in the first cycle, first expose in the SPAD exposure period, generate digital signals when receiving photons under exposure, and then enter the digital counting period of the SPAD pixel units of the first row (Row 1) when receiving the control signal, and output the digital signals to the target counter during the digital counting period of the SPAD pixel units of the first row (Row1) to perform digital counting processing. During the digital counting period of the SPAD pixel units of the first row (Row 1), the exposure of the SPAD pixel units of the second row (Row 2) is waited at the same time. After the digital counting period of the SPAD pixel units of the first row (Row 1), the digital counting period of the SPAD pixel units of the second row (Row 2) is entered, and the digital counting process of the SPAD pixel units of the second row (Row 2) is performed. And so on, until the SPAD pixel units of the Mth row complete the digital counting in the first cycle, all SPAD pixel rows have completed the entire operation of the first cycle.
[0082] In the embodiment of the present application, since there is a time difference in the operation of the unit time of each SPAD pixel unit, it is ensured that the target digital counter digitally counts one row of SPAD pixel units alone without being occupied by other rows of SPAD pixel units.
[0083] In this way, all SPAD pixel units can effectively operate through a digital counter and output the number of recorded photons in each unit time. In addition, the number of cycles K in each unit time and the duration of the SPAD exposure period in the cycle can be flexibly adjusted according to application requirements to meet the shooting requirements of complex scenes.
[0084] In a specific example, in order to ensure that the target digital counter digitally counts a row of SPAD pixel units separately in each cycle without being occupied by other rows of SPAD pixel units, the first SPAD pixel unit and the second SPAD pixel unit can be located in different rows of the SPAD pixel array, and the digital counting period of the first SPAD pixel unit in the i-th cycle among K cycles is staggered with the digital counting period of the second SPAD pixel unit in the i-th cycle among K cycles; 1≤i≤K.
[0085] For example, the digital counting period of the SPAD pixel units in the first row (Row 1) in the first cycle is staggered with the digital counting period of the SPAD pixel units in the second row (Row 2) in the first cycle, the digital counting period of the SPAD pixel units in the second row (Row 2) in the first cycle is staggered with the digital counting period of the SPAD pixel units in the third row (Row 3) in one cycle, and so on, the digital counting period of the SPAD pixel units in the M-1th row (Row M) in the first cycle is staggered with the digital counting period of the SPAD pixel units in the Mth row (Row M) in one cycle.
[0086] For another example, the digital counting time period of the SPAD pixel units in the first row (Row 1) in the second cycle is staggered with the digital counting time period of the SPAD pixel units in the second row (Row 2) in the second cycle, and the digital counting time period of the SPAD pixel units in the second row (Row 2) in the second cycle is staggered with the digital counting time period of the SPAD pixel units in the third row (Row 3) in the two cycles, and so on, the digital counting time period of the SPAD pixel units in the M-1th row (Row M) in the second cycle is staggered with the digital counting time period of the SPAD pixel units in the Mth row (Row M) in the two cycles.
[0087] In this way, since the digital counting periods in each cycle of each SPAD pixel unit are staggered in operation, it is ensured that the target digital counter digitally counts one row of SPAD pixel units in each cycle without being occupied by other rows of SPAD pixel units.
[0088] In addition, in a specific example, the digital counting periods in two adjacent cycles are staggered in operation, ensuring that the target digital counter digitally counts one row of SPAD pixel units independently in two adjacent cycles without being occupied by other rows of SPAD pixel units.
[0089] For example, the first SPAD pixel unit is located in the first row of the SPAD pixel array, and the second SPAD pixel unit is located in the last row of the SPAD pixel array; the digital counting period of the first SPAD pixel unit in the j+1th cycle of the K cycles is temporally located after the digital counting period of the second SPAD pixel unit in the jth cycle of the K cycles; 1≤j <K。
[0090] For example, the first SPAD pixel unit is located in the first pixel row, and the second SPAD pixel unit is located in the last pixel row. The digital counting period of the SPAD pixel units in the first row (Row 1) in the second cycle is after the digital counting period of the SPAD pixel units in the last row (Row M) in the first cycle.
[0091] Specifically, refer to Figure 5 , after the digital counter completes the digital counting period of the last row (Row M) of SPAD pixel units in the first cycle, the SPAD pixel units in the first row (Row 1) may still run in the SPAD exposure period of the second cycle, and exist in a counting blanking period (Blanking Time). After the counting blanking period, the SPAD pixel units in the first row (Row 1) run in the digital counting period of the second cycle. Among them, the counting blanking period is located after the digital counting period of the first cycle of the SPAD pixel units in the last row (Row M) is completed, and before the digital counting period of the SPAD pixel units in the first row (Row 1) in the second cycle begins.
[0092] In other embodiments, if the SPAD exposure period of each row of SPAD pixel units is adjusted to be short enough, this counting blank period may not exist, ensuring that the digital counting period of the SPAD pixel units in the first row (Row 1) in the second cycle is just after the digital counting period of the SPAD pixel units in the last row (Row M) in the first cycle.
[0093] In this way, the digital counting periods of SPAD pixel units in different pixel rows in two adjacent cycles are staggered in operation, ensuring that the target digital counter digitally counts one row of SPAD pixel units in two adjacent cycles without being occupied by other rows of SPAD pixel units.
[0094] In practical applications, for some high-light-ratio shooting scenes, image sensors can implement high dynamic range (HDR) related technologies, such as a multi-frame fusion technology. In the operating method of the image sensor used in this application, the multi-frame fusion technology is implemented by coordinating different cycles within a unit time.
[0095] In the embodiment of the present application, at least two cycles can be grouped together within a unit time, and the position of the digital counting period in each cycle can be adjusted within the entire cycle to achieve the exposure period length modulation of the SPAD pixel unit.
[0096] For example, within a unit time, there are a first cycle and a second cycle in K cycles, the first cycle includes a first digital counting period, the second cycle includes a second digital counting period, and the position of the first digital counting period in the first cycle is different from the position of the second digital counting period in the second cycle.
[0097] The first cycle also includes a first SPAD exposure period, and the second cycle also includes a second SPAD exposure period. The duration of the first SPAD exposure period is different from the duration of the second SPAD exposure period.
[0098] In a specific embodiment, referring to Figure 5 , in K cycles, the positions of the digital counting periods in each cycle are the same, both at the end of the cycle.
[0099] In the embodiment of the present application, the present application can adjust the position of the first digital counting period in the first cycle so that the position of the first digital counting period in the first cycle is different from the position of the second digital counting period in the second cycle. Alternatively, by adjusting the position of the second digital counting period in the second cycle, the position of the first digital counting period in the first cycle is different from the position of the second digital counting period in the second cycle. Furthermore, the length of the SPAD period in the first cycle and the second cycle is modulated to implement the HDR technology.
[0100] In a specific embodiment, the first cycle includes odd cycles among K cycles, the second cycle includes even cycles among K cycles, the first digital counting period is located at a first position of the first cycle, the second digital counting period is located at a second position of the second cycle, the first position is one of a tail position and a middle position, and the second position is the other of the tail position and the middle position.
[0101] In the embodiment of the present application, by controlling the position of the digital counting period within the first cycle / the second cycle, the SPAD pixel unit can implement different long and short exposure combinations, and the length of each cycle remains consistent.
[0102] In this way, two adjacent cycles can be grouped together within a unit time, and the position of the digital counting period within the cycle can be adjusted within the entire cycle to achieve the exposure period length modulation of the SPAD pixel unit.
[0103] For example, Figure 7 As shown, in the odd-numbered cycles (1st, 3rd, 5th ...), the digital counting period is located at the end of the cycle. In the even-numbered cycles (2nd, 4th, 6th ...), the digital counting period is located in the middle of the cycle. Figure 7As shown, one of the methods is to immediately enter the digital counting period of the 1st row (Row 1) of the even cycle after the digital counting period of the Mth row (Row M) of the odd cycle is completed. If necessary, there can also be a certain counting blank period.
[0104] Among them, the exposure period of the SPAD pixel unit is a long and short configuration, including a long exposure period and a short exposure period. The SPAD long exposure period is composed of an odd cycle and a part of an even cycle period. The SPAD short exposure period is composed of a part of an even cycle period. This method can ensure that the length of all cycles is consistent, and the SPAD pixel long and short exposure modulation is achieved within a unit time to achieve the implementation of HDR technology.
[0105] Of course, in addition to Figure 7 As shown, two cycles are used as a group, and the position of the digital counting period in each cycle is adjusted in the entire cycle to achieve exposure period length modulation of the SPAD pixel unit. The embodiment of the present application can also use more than two cycles as a group to form multi-exposure duration modulation to achieve more complex and refined HDR technology.
[0106] In this way, by modulating the position of the digital counting period in each cycle, the long and short exposure modulation of the SPAD pixel unit per unit time can be achieved, so as to implement HDR technology to cope with complex shooting scenes.
[0107] In the embodiments of the present application, Figure 6 As shown, the unit time of the SPAD pixel unit may also include a signal processing period, which is located after K cycles; the signal processing period is used to process the digital counting signal output from the digital counting period within the K cycles;
[0108] After step 340, the control method provided in the embodiment of the present application may further include:
[0109] In the signal processing period, the digital counting signal outputted in the digital counting period of the K cycles is processed to obtain an image signal, and the image signal is outputted.
[0110] In an embodiment of the present application, the unit time may also include a signal processing period, the beginning of the unit time is the first cycle, and the signal processing period is entered after the Kth cycle is completed. When the signal processing period is completed, the unit time ends and enters the next unit time. In one embodiment, for a SPAD pixel array having M rows of SPAD pixel units, each unit time of the image sensor includes K cycles. Each cycle consists of a SPAD exposure period and a digital counting period. During the SPAD exposure period, the SPAD device in the SPAD pixel unit is exposed and detects the incident photons to generate a digital signal. During the digital counting period, the SPAD pixel unit located at the pixel layer outputs a digital signal to a digital counter located at the logic layer for counting to obtain a digital counting signal. The digital counting signal can be cached in the subsequent signal processing circuit 140 for image signal processing, or it can be directly scanned and output by the port circuit 150.
[0111] For example, the SPAD pixel unit may enter a signal processing period after completing K cycles. The signal processing period may be used by the signal processing circuit 140, the port circuit 150 or other auxiliary circuits to perform image signal processing (such as digital amplification processing, bad pixel screening processing, signal output processing, etc.), circuit reset, and preparation for the next frame.
[0112] Specifically, within a unit of time, when the Kth cycle is completed, the signal processing period begins. Figure 6 As shown, the signal processing period does not overlap with the digital counting period of the Kth cycle. During the signal processing period, the signal processing circuit at the logic layer will complete signal processing, signal transmission, and necessary reset steps and prepare for the start of the first cycle of the next unit time. Similarly, there is a time difference T in the signal processing period of each SPAD pixel unit, and the size of T is at least one digital counting period. The signal processing circuit may process multiple lines of image signals in the signal processing period of a certain line.
[0113] In this way, the processing of image signals is completed by the signal processing circuit of the logic layer, and its signal processing period is placed after all cycles and is independent of the operation within the cycle. Therefore, the interference and signal crosstalk caused by the operation of the signal processing circuit on the operation of the SPAD pixel unit of the pixel layer can be effectively reduced. In addition, the signal processing circuit can be ignored, which improves flexibility.
[0114] Based on the same concept as the image sensor control method provided in any of the above embodiments, an embodiment of the present application further provides an image sensor, which can be used to implement the image sensor control method provided in any of the above embodiments.
[0115] For example, refer to Figure 1 and Figure 2 , the image sensor 10 provided in the embodiment of the present application may include a pixel layer 100 and a logic layer 200 which are stacked;
[0116] The pixel layer 100 is provided with a SPAD pixel array 110, and the SPAD pixel array 110 includes M*N SPAD pixel units 111, where M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer;
[0117] The logic layer 200 is provided with N digital counters 120, a clock and control circuit 130, a signal processing circuit 140 and a port circuit 150;
[0118] The M SPAD pixel units 111 located in the same column in the SPAD pixel array are connected to the same digital counter 120 among the N digital counters.
[0119] The SPAD pixel unit 111 may include a PMOS switch transistor, a SPAD, an inverter, and a pixel selection switch transistor SEL. The working principle of the SPAD pixel unit 111 includes: CLK Under the modulation of the , the SPAD device continuously cycles between quenching and charging under exposure, forming an analog pulse signal; the inverter converts the analog pulse signal into a digital signal. SEL (which can be understood as a pixel output selection signal), and the converted digital signal is used as the pixel output signal S PIX The photons are output to the corresponding digital counter 120 through the output wire, so that the digital counter 120 performs digital counting processing on the photons sensed by the SPAD device within the digital counting period allocated to the SPAD pixel unit.
[0120] Among them, the digital counter will count once after each photon is received by the SPAD. If X photons reach the SPAD, the digital counter will count X times.
[0121] Among them, the PMOS switch transistor can be equivalent to a variable resistor, which prevents the peak value of the analog pulse signal from being too large or too small by controlling the size of the analog pulse signal flowing through the SPAD.
[0122] In an embodiment of the present application, the digital counter allocates staggered digital counting time periods to different SPAD pixel units in the same column of SPAD pixel units. Furthermore, different SPAD pixel units in a plurality of SPAD pixel units use staggered digital counting time periods of the target digital counter, ensuring that each SPAD pixel unit can use the time of the digital counter alone for counting processing. Furthermore, in an application scenario where a plurality of SPAD pixel units of an image sensor share a digital counter, it can be ensured that each SPAD pixel unit can use the time of the digital counter alone for counting processing, thereby achieving effective operation of each SPAD pixel unit in the image sensor.
[0123] It should be noted that, in practical applications, the embodiments of the present application do not impose specific restrictions on the number and type of target digital counters. The type of the target digital counter can be a single-input digital counter or a multi-input digital counter, and the number of the target digital counters can be one or more. The embodiments of the present application can set the type and number of the target digital counters according to actual needs.
[0124] For example, in other embodiments (not shown), the target digital counter may be a multi-input multi-output digital counter, and the number of target digital counters may be one. For example, the target digital counter may include N input terminals, and the target digital counter may be used to count pixel output signals input from the N input terminals at the same time. The SPAD pixel array may include M*N SPAD pixel units, where M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; wherein the M SPAD pixel units in the first column are connected to the first input terminal of the target digital counter through the same output conductor, and the M SPAD pixel units in the second column are connected to the second input terminal of the target digital counter through the same output conductor, and so on, the M SPAD pixel units in the Nth column are connected to the Nth input terminal of the target digital counter through the same output conductor.
[0125] Among them, the target digital counter can be used to count the photons sensed by the M SPAD pixel units in the first column, count the photons sensed by the M SPAD pixel units in the second column, and so on, to count the photons sensed by each SPAD pixel unit in the Nth column.
[0126] Furthermore, the target digital counter can be used to synchronously count pixel output signals from different input terminals. For example, the target digital counter can be used to synchronously count the photons sensed by the N SPAD pixel units in the first row, synchronously count the photons sensed by the N SPAD pixel units in the second row, and so on, synchronously count the photons sensed by the N SPAD pixel units in the Mth row.
[0127] In this way, by connecting M SPAD pixel units in the same column to the same input terminal of the target digital counter through the same output wire, and connecting SPAD pixel units in different columns to different input terminals of the target digital counter, M*N SPAD pixel units can share a target digital counter with N input terminals for counting, thereby realizing the effective operation of M*N SPAD pixel units.
[0128] It should be pointed out that the image sensor can be used to implement the control method of the image sensor provided in any of the above embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described here.
[0129] Based on the same concept as the image sensor provided in any of the above embodiments, an embodiment of the present application further provides an electronic device.
[0130] refer to Figure 8 An embodiment of the present application provides an electronic device 20, which may include the image sensor 10 provided in any of the above embodiments.
[0131] In practical applications, the electronic device 20 may include at least one CCM (Compact Camera Module) module, each CCM module may include an image sensor 10, and the image sensor communicates bidirectionally with the ISP (Image Signal Processor) module in the AP (Application Processor) or SoC (System on Chip) chip through a port link. The AP / SoC sends a control signal to the image sensor in each CCM module, and the image sensor generates an image signal and transmits it back to the ISP module in the AP / SoC for back-end processing.
[0132] In the embodiment of the present application, the electronic device 20 may be a device with a camera function. The electronic device may be a terminal or other device other than a terminal. Exemplarily, the electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. It may also be a server, a network attached storage (NAS), a personal computer (PC), etc., which is not specifically limited in the embodiment of the present application.
[0133] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0134] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A method for controlling an image sensor, characterized in that: The image sensor comprises a single photon avalanche transistor SPAD pixel array and a target digital counter, wherein the SPAD pixel array comprises M SPAD pixel units that share the target digital counter, and the M SPAD pixel units are connected to the target digital counter via a same output wire, and the method comprises: One SPAD pixel unit among the M SPAD pixel units obtains a clock signal and a control signal; M is a positive integer greater than 1; The SPAD pixel unit generates a digital signal based on the clock signal; The SPAD pixel unit outputs the digital signal to the target digital counter based on the control signal; The target digital counter performs counting processing on the digital signal within the digital counting period allocated to the SPAD pixel unit to obtain a digital counting signal of the SPAD pixel unit; the digital counting signal of the SPAD pixel unit is used to generate a target image; The target digital counter allocates mutually staggered digital counting time periods to different SPAD pixel units among the M SPAD pixel units.
2. The method according to claim 1, characterized in that The SPAD pixel array includes M*N SPAD pixel units, where M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; the M SPAD pixel units are located in the same column of the SPAD pixel array.
3. The method according to claim 2, characterized in that Before one of the M SPAD pixel units acquires the clock signal and the control signal, the method further includes: A clock signal and a control signal are provided to M rows of SPAD pixel units in the SPAD pixel array in a row-by-row manner.
4. The method according to claim 3, characterized in that The number of the target digital counters is N. All SPAD pixel units in the same column of the SPAD pixel array share the same target digital counter, and SPAD pixel units in different columns share different target digital counters. SPAD pixel units in the same row operate synchronously within a unit time.
5. The method for controlling an image sensor according to any one of claims 1 to 4, characterized in that: There are a first SPAD pixel unit and a second SPAD pixel unit among the M SPAD pixel units; the length of the unit time of the first SPAD pixel unit is the same as the length of the unit time of the second SPAD pixel unit, and there is a time difference between the starting moment of the unit time of the first SPAD pixel unit and the starting moment of the unit time of the second SPAD pixel unit; The unit time includes K cycles, K is a positive integer; each of the K cycles includes a SPAD exposure period and a digital counting period, and the length of the digital counting period in each cycle is the same; The time difference is greater than or equal to the length of the digital counting period.
6. The method for controlling an image sensor according to claim 5, characterized in that: The first SPAD pixel unit and the second SPAD pixel unit are located in different rows of the SPAD pixel array, and a digital counting period of the first SPAD pixel unit in the i-th cycle among the K cycles is staggered with a digital counting period of the second SPAD pixel unit in the i-th cycle among the K cycles; 1≤i≤K.
7. The method for controlling an image sensor according to claim 5, characterized in that: The first SPAD pixel unit is located in the first row of the SPAD pixel array, and the second SPAD pixel unit is located in the last row of the SPAD pixel array; the digital counting period of the first SPAD pixel unit in the j+1th cycle of the K cycles is temporally located after the digital counting period of the second SPAD pixel unit in the jth cycle of the K cycles; 1≤j <K。 8. The method for controlling an image sensor according to claim 5, characterized in that: The K cycles include a first cycle and a second cycle, the first cycle includes a first digital counting period, the second cycle includes a second digital counting period, and a position of the first digital counting period in the first cycle is different from a position of the second digital counting period in the second cycle.
9. The method for controlling an image sensor according to claim 8, characterized in that: The first cycle includes odd cycles among K cycles, the second cycle includes even cycles among K cycles, the first digital counting period is located at a first position of the first cycle, the second digital counting period is located at a second position of the second cycle, the first position is one of a tail position and a middle position, and the second position is the other of the tail position and the middle position.
10. The method for controlling an image sensor according to claim 5, characterized in that: The unit time also includes a signal processing period, and the signal processing period is located after the K cycles; The signal processing period is used to process the digital counting signal outputted by the digital counting period within the K cycles; The method further comprises: In the signal processing period, the digital counting signal outputted in the digital counting period of the K cycles is processed to obtain an image signal, and the image signal is outputted.
11. An image sensor, characterized in that: The image sensor is used to implement the control method according to any one of claims 1 to 10.
12. The image sensor according to claim 11, characterized in that The image sensor includes a pixel layer and a logic layer which are stacked; The pixel layer is provided with a SPAD pixel array, and the SPAD pixel array includes M*N SPAD pixel units, where M is the number of rows, N is the number of columns, M is a positive integer greater than 1, and N is a positive integer; The SPAD pixel unit includes a P-channel metal oxide semiconductor (PMOS) switch transistor, a SPAD, an inverter, and a pixel selection switch transistor; The logic layer is provided with N digital counters, a clock and control circuit, a signal processing circuit and a port circuit; The M SPAD pixel units located in the same column in the SPAD pixel array are connected to the same digital counter among the N digital counters via the same output wire.
13. An electronic device, characterized in that: Comprising the image sensor as claimed in claim 11.
Citation Information
Patent Citations
TDC unit, TDC array and ranging system
CN114114211A
SPAD pixel array, distance measuring device and distance measuring method
CN118169655A
DDoF ranging circuit and DDoF ranging chip
CN118393522A
Control method for electronic device, electronic device, and computer-readable storage medium
WO2022110947A1
