Image sensor and electronic device

Abstract

The utility model provides an image sensor and electronic equipment, the image sensor comprises a pixel unit, a signal conversion circuit and an interface circuit, and the pixel unit comprises a first pixel circuit and a second pixel circuit; the output end of the first pixel circuit is connected with the first input end of the signal conversion circuit; the output end of the second pixel circuit is connected with the second input end of the signal conversion circuit; the first input end of the signal conversion circuit is connected with the output end of the first pixel circuit, the second input end is connected with the output end of the second pixel circuit, and the output end is connected with the input end of the interface circuit; and the input end of the interface circuit is connected with the output end of the signal conversion circuit, so that the dynamic range of the image sensor can be effectively improved under the condition that the hardware cost is not greatly increased.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical fields of photoelectric detection, image sensing and signal processing, and in particular, to an image sensor and an electronic device. BACKGROUND

[0002] The photoelectric image sensor is a new type of neuromorphic vision sensor, which records the continuous light intensity information in the scene by emitting a high-density single-bit pulse sequence, can realize high-precision capture and recording of high-speed light processes, and can also reconstruct the texture details in the scene, thus having great application value in machine vision and dynamic scene capture.

[0003] The dynamic range of the image sensor refers to the range of light intensity between the maximum light intensity and the minimum light intensity that can be distinguished by the image sensor, and the dynamic range is an important evaluation index of the photoelectric image sensor.

[0004] In the process of implementing the present disclosure, it is found through research that in the related art, the photoelectric image sensor reflects the light intensity by recording the cumulative value of the photoelectric signal in a fixed time, and the main ways to widen the dynamic range of the photoelectric image sensor include: increasing the number of photodiodes in the photoelectric image sensor to realize the separate photosensing of strong and weak light; or increasing a plurality of signal conversion readout channels to realize different gains of strong and weak light signals. These ways all need to increase the hardware cost, and the improvement effect of the dynamic range is directly related to the hardware cost, for example, if the dynamic range is to be increased by ten times, the hardware photosensitive area needs to be increased by ten times, thereby increasing the hardware cost and the complexity of process preparation, resulting in a substantial increase in cost. UTILITY MODEL CONTENT

[0005] The embodiments of the present disclosure provide an image sensor and an electronic device to effectively improve the dynamic range of the image sensor without substantially increasing the hardware cost.

[0006] In one aspect of the embodiments of the present disclosure, an image sensor is provided, which includes: a pixel array, a plurality of signal conversion circuits, and an interface circuit;

[0007] The pixel array is arranged by a plurality of rows and a plurality of columns of pixel units, and the pixel unit includes a first pixel circuit and a second pixel circuit; each signal conversion circuit is connected to the first pixel circuit and the second pixel circuit in the pixel unit in one dimension direction in the pixel array, wherein the pixel unit in one dimension direction is a row of pixel units or a column of pixel units;

[0008] The output end of the first pixel circuit is connected with the first input end of the corresponding signal conversion circuit, and outputs first and second electric signals to the corresponding signal conversion circuit; wherein the second electric signal is an electric signal output by the first pixel circuit after an exposure phase ends, and the second electric signal is an electric signal obtained by generating photo-generated charges based on a light signal received by the first pixel circuit during the exposure process after the first pixel circuit is reset; the first electric signal is an electric signal generated by the first pixel circuit during the exposure phase and after the first pixel circuit is reset;

[0009] The output end of the second pixel circuit is connected with the second input end of the corresponding signal conversion circuit, and outputs a third electric signal to the corresponding signal conversion circuit; wherein the third electric signal is an electric signal continuously output by the second pixel circuit during an exposure phase, and the third electric signal is obtained by generating photo-generated charges based on a light signal received by the second pixel circuit during the exposure process after the second pixel circuit is reset;

[0010] The first input end of the signal conversion circuit is connected with the output end of each first pixel circuit in the corresponding dimension direction, the second input end of the signal conversion circuit is connected with the output end of each second pixel circuit in the corresponding dimension direction, the output end of the signal conversion circuit is connected with the input end of the interface circuit, and the interface circuit outputs a digital signal obtained by performing analog-digital conversion on a difference between the first and second electric signals output by the same pixel unit, and / or a first time required for the third electric signal output by the same pixel unit to reach a preset threshold value.

[0011] The multiple input ends of the interface circuit are connected with the output ends of multiple signal conversion circuits, respectively, and the output end of the interface circuit sequentially outputs output data of the multiple signal conversion circuits.

[0012] In yet another aspect of the embodiments of the present disclosure, an electronic device is provided, which includes a processor, a memory connected with the processor in communication, respectively, and the image sensor according to any of the embodiments of the present disclosure.

[0013] Based on the embodiment of the present disclosure, the image sensor comprises a pixel array, a plurality of signal conversion circuits, a plurality of signal processing circuits, and an interface circuit, wherein the pixel array is arranged by a plurality of rows and a plurality of columns of pixel units, each pixel unit comprises a first pixel circuit and a second pixel circuit, each signal conversion circuit is connected to the first pixel circuit and the second pixel circuit in the pixel unit in one dimension direction of the pixel array, in each pixel unit, the first pixel circuit generates a first electrical signal by resetting in an exposure stage and outputs the first electrical signal, and outputs a second electrical signal generated by exposure after the exposure stage ends, the second pixel circuit continuously outputs a third electrical signal generated by exposure in the exposure process, the first input end of the signal conversion circuit is connected to the output end of each first pixel circuit in the corresponding dimension direction, the second input end is connected to the output end of each second pixel circuit in the corresponding dimension direction, the output end of the signal conversion circuit is connected to the input end of the interface circuit, and the interface circuit outputs a digital signal obtained by analog-to-digital conversion of the difference between the first electrical signal and the second electrical signal output by the same pixel unit and / or a first time required for the third electrical signal output by the same pixel unit to reach a preset threshold, the plurality of input ends of the interface circuit are connected to the output ends of the plurality of signal conversion circuits, and the output end of the interface circuit sequentially outputs the output data of the plurality of signal conversion circuits. In the embodiment of the present disclosure, each pixel unit simultaneously adopts the first pixel circuit and the second pixel circuit to expose the same space in different exposure modes. Since the digital signal corresponding to the first pixel circuit can accurately reflect the exposure intensity of weak light intensity, the first time corresponding to the second pixel circuit can accurately reflect the exposure intensity of strong light intensity, and the combination of the digital signal and the first time can reflect the exposure intensity of different light intensities, not only the accurate recording of weak light signals and strong light signals is realized, but also the dynamic range of the image sensor is effectively expanded without greatly increasing the hardware cost and cost.

[0014] The technical solutions of the present disclosure are described in further detail below by means of the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0015] The accompanying drawings, which form a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0016] The present disclosure can be more clearly understood by referring to the following detailed description in conjunction with the accompanying drawings, in which:

[0017] Figure 1 FIG. 1 is a structural schematic diagram of an embodiment of an image sensor of the present disclosure.

[0018] Figure 2 FIG. 2 is a structural schematic diagram of another embodiment of an image sensor of the present disclosure.

[0019] Figure 3Structure diagram of still another embodiment of the image sensor of the present disclosure.

[0020] Figure 4 Structure diagram of still another embodiment of the image sensor of the present disclosure.

[0021] Figure 5 Structure diagram of still another embodiment of the image sensor of the present disclosure.

[0022] Figure 6 Structure diagram of still another embodiment of the image sensor of the present disclosure.

[0023] Figure 7 Timing diagram of one row driving operation in the embodiment of the present disclosure.

[0024] Figure 8 Structure diagram of one application embodiment of the electronic device of the present disclosure. DETAILED DESCRIPTION

[0025] Various exemplary embodiments of the present disclosure will now be described in detail below with reference to the accompanying drawings. Note that the relative arrangement, numerical expressions, and numerical values of the components and steps set forth in these embodiments are not intended to limit the scope of the present disclosure unless otherwise specifically stated.

[0026] Those skilled in the art can understand that the terms "first", "second", and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they represent a necessary logical sequence between them.

[0027] It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two, or more.

[0028] It should also be understood that for any component, data, or structure mentioned in the embodiments of the present disclosure, unless specifically limited or given a contrary implication by the context, it can generally be understood as one or more.

[0029] In addition, the term "and / or" in the present disclosure is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the front and rear associated objects.

[0030] It should also be understood that the description of various embodiments of the present disclosure focuses on the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0031] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0032] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0033] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0034] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0035] Figure 1 FIG. 1 is a schematic diagram of the structure of an embodiment of the image sensor disclosed herein. Figure 1 As shown, the image sensor of this embodiment includes a pixel array 10 formed by a plurality of rows (hereinafter referred to as m rows for ease of description, where m is an integer greater than 1) and a plurality of columns (hereinafter referred to as n columns for ease of description, where n is an integer greater than 1) of pixel units 102, a plurality of signal conversion circuits 104, and an interface circuit 106. Each pixel unit 102 includes a first pixel circuit 1022 and a second pixel circuit 1024, that is, the first pixel circuit 1022 and the second pixel circuit 1024 together constitute a complete pixel unit. Each signal conversion circuit is connected to the first pixel circuit 1022 and the second pixel circuit 1024 of the pixel units 102 in a dimension of the pixel array, where the pixel units in a dimension can be a row of pixel units or a column of pixel units. When each signal conversion circuit 104 is connected to a row of pixel units in the pixel array 10, the number of signal conversion circuits 104 is consistent with the number m of pixel rows in the pixel array 10. Thus, each signal conversion circuit 104 can sequentially read the signals of the corresponding row of pixel units column by column. When each signal conversion circuit 104 is connected to a column of pixel units in the pixel array 10, the number of signal conversion circuits 104 is consistent with the number n of columns of pixel units in the pixel array. Therefore, each signal conversion circuit 104 can realize row-by-row signal reading of the corresponding column of pixel units in sequence. Figure 1 Only the schematic diagram of the structure in which each signal conversion circuit 104 is connected to a column of pixel units in the pixel array 10 is shown. The implementation structure in which each signal conversion circuit 104 is connected to a row of pixel units in the pixel array 10 can be referred to and will not be given here.

[0036] The output end of the first pixel circuit 1022 is connected with the first input end of the corresponding signal conversion circuit 104, and outputs the first electric signal and the second electric signal to the corresponding signal conversion circuit 104. The second electric signal is an electric signal output by the first pixel circuit 1022 after the end of the exposure stage, and the second electric signal is an electric signal obtained by generating photo-generated charges based on the light signal received by the first pixel circuit 1022 during the exposure process after the reset.

[0037] The output end of the second pixel circuit 1024 is connected with the second input end of the corresponding signal conversion circuit 104, and outputs the third electric signal to the corresponding signal conversion circuit 104; wherein the third electric signal is an electric signal output by the second pixel circuit 1024 during the exposure stage, and the third electric signal is obtained by generating photo-generated charges based on the light signal received by the second pixel circuit 1024 during the exposure process after the reset.

[0038] In the embodiment of the present disclosure, the first electric signal, the second electric signal and the third electric signal are electric signals obtained by generating photo-generated charges based on the light signal received by the pixel unit, which can be a voltage signal or a current signal, and the embodiment of the present disclosure does not limit this. In the embodiment of the present disclosure, the electric signal is taken as a voltage signal as an example, and the case of the electric signal being a current signal is also applicable.

[0039] The first input end of the signal conversion circuit 104 is connected with the output end of each first pixel circuit 1022 in the corresponding dimension direction respectively, the second input end of the signal conversion circuit 104 is connected with the output end of each second pixel circuit 1024 in the corresponding dimension direction respectively, the output end of the signal conversion circuit 104 is connected with the input end of the interface circuit 106, and the interface circuit 106 outputs the digital signal obtained by analog-digital conversion of the difference between the first electric signal and the second electric signal output by the same pixel unit, and / or the first time required for the third electric signal output by the same pixel unit to reach the preset threshold.

[0040] Through the digital signal, the weak light signal can be imaged. Through the first time, the strong light signal can be imaged.

[0041] The plurality of input ends of the interface circuit 106 are connected with the output ends of the plurality of signal conversion circuits 104 respectively, and the output end of the interface circuit 106 outputs the output data of the plurality of signal conversion circuits 104 in sequence.

[0042] The interface circuit 106 is provided with a plurality of input ports in parallel with the number of the signal conversion circuits 104, as a plurality of inputs which are connected one by one with the outputs of the plurality of signal conversion circuits 104, and sequentially outputs the output data of the plurality of signal conversion circuits 104 in the order of the plurality of signal conversion circuits 104, to obtain the output data sequence of the plurality of signal conversion circuits 104 in the same data output period, so as to realize the parallel-to-serial conversion of the output data of the plurality of signal conversion circuits 104 and the data identification information thereof, for external transmission.

[0043] Based on the embodiments of the present disclosure, the image sensor includes a pixel array, a plurality of signal conversion circuits, a plurality of signal processing circuits, and an interface circuit, wherein the pixel array is arranged by a plurality of rows and a plurality of columns of pixel units, each pixel unit includes a first pixel circuit and a second pixel circuit, each signal conversion circuit is connected to the first pixel circuit and the second pixel circuit in a pixel unit in a corresponding dimension direction of the pixel array, in each pixel unit, the first pixel circuit generates a first electrical signal by resetting in an exposure stage and outputs, and outputs a second electrical signal generated by exposure after the exposure stage ends, the second pixel circuit continuously outputs a third electrical signal generated by exposure during exposure, the first input end of the signal conversion circuit is connected to the output end of each first pixel circuit in the corresponding dimension direction, the second input end is connected to the output end of each second pixel circuit in the corresponding dimension direction, the output end of the signal conversion circuit is connected to the input end of the interface circuit, and the interface circuit outputs a digital signal obtained by analog-to-digital conversion of the difference between the first electrical signal and the second electrical signal output by the same pixel unit, and / or a first time required for the third electrical signal output by the same pixel unit to reach a preset threshold, the plurality of input ends of the interface circuit are connected to the output ends of the plurality of signal conversion circuits, and the output end of the interface circuit sequentially outputs the output data of the plurality of signal conversion circuits, in the embodiments of the present disclosure, each pixel unit simultaneously uses the first pixel circuit and the second pixel circuit to expose the same space in different exposure modes, since the digital signal corresponding to the first pixel circuit can accurately reflect the exposure intensity of weak light intensity, the first time corresponding to the second pixel circuit can accurately reflect the exposure intensity of strong light intensity, and the combination of the digital signal and the first time can reflect the exposure intensity of different light intensities, not only realizing accurate recording of weak light signals and strong light signals, but also effectively expanding the dynamic range of the image sensor without significantly increasing the hardware cost and cost.

[0044] Figure 2 The structure schematic diagram of another embodiment of the image sensor of the present disclosure is shown in FIG. 4. Figure 2 As shown in FIG. 4, an implementation example of each signal conversion circuit 104 connected to a column of pixel units in the pixel array 10 is shown.

[0045] Optionally, referring back to Figure 2In some implementations, the first pixel circuit 1022 can be implemented in a 4T pixel structure. For example, the first pixel circuit 1022 can include a first photodiode 202, a transfer transistor 204, a capacitor 206, a first reset transistor 208, a first source follower transistor 310, and a first readout transistor 312, as shown in the solid line box in FIG. 2. Figure 2

[0046] wherein:

[0047] One end of the first photodiode 202 is grounded, and the other end is connected to one end of the capacitor 206 and a gate end of the first source follower transistor 310 through the transfer transistor 204.

[0048] The source end of the transfer transistor 204 is connected to the other end of the first photodiode 202, and the drain end is connected to one end of the capacitor 206 and the gate end of the first source follower transistor 310, respectively. The gate end receives an external control signal (for ease of distinction, referred to below as a first external control signal, denoted as Tx) that controls the conduction or disconnection of the transfer transistor 204. The first external control signal Tx of the same row of pixel units 102 in the pixel array 10 is uniformly controlled.

[0049] One end of the capacitor 206 is connected to the source end of the first reset transistor 208 and the gate end of the first source follower transistor 310, respectively, and to the other end of the first photodiode 202 through the transfer transistor 204, and the other end is grounded.

[0050] The source end of the first reset transistor 208 is connected to one end of the capacitor 206, and the drain end is connected to a power supply signal VDD. The gate end receives an external control signal (for ease of distinction, referred to below as a second external control signal, denoted as Rst) that controls the conduction or disconnection of the first reset transistor 208. The second external control signal Rst of the same row of pixel units 102 in the pixel array 10 is uniformly controlled.

[0051] The source end of the first source follower transistor 310 is connected to the drain end of the first readout transistor 312, and the drain end is connected to a power supply signal VDD. The gate end is connected to one end of the capacitor 206 and the drain end of the transfer transistor 204, respectively.

[0052] The source end of the first readout transistor 312 serves as the output end of the first pixel circuit 1022, and the drain end is connected to the source end of the first source follower transistor 310. The gate end receives an external control signal (for ease of distinction, referred to below as a third external control signal, denoted as Read) that controls the conduction or disconnection of the first readout transistor 312. The third external control signal Read of the same row of pixel units 102 in the pixel array 10 is uniformly controlled.

[0053] ​In this embodiment, the first external control signal Tx, the second external control signal Rst and the third external control signal Read of all row pixel units 102 in the pixel array 10 are generated by the same drive circuit (for example, the first drive circuit in Figure 5 , Figure 6 . The first pixel circuit 1022 in each pixel unit 102.

[0054] Based on this embodiment, a pixel circuit implemented by using a 4T pixel structure is provided, which can be used for detecting light signals in a weak light intensity environment and generating an electrical signal representing the exposure intensity. In addition, the first pixel circuit in the embodiment of the present disclosure can also be implemented by using other circuit structures, as long as the second pixel unit can be reset, exposed, generate an analog electrical signal representing the exposure intensity and output externally under the control of the drive circuit. The specific structure of the first pixel circuit is not limited in the embodiment of the present disclosure.

[0055] Optionally, referring back to Figure 2 , in some implementations, the second pixel circuit 1024 can be implemented by using a 3T pixel structure, for example, the second pixel circuit 1024 includes a second photodiode 302, a second reset transistor 304, a second source follower transistor 306 and a second read transistor 308, as shown in the dashed box in Figure 2 . Wherein:

[0056] One end of the second photodiode 302 is grounded, and the other end is connected with the source end of the second reset transistor 304 and the gate end of the second source follower transistor 306, respectively.

[0057] The source end of the second reset transistor 304 is connected with the other end of the second photodiode 302, the drain end is connected with a power supply signal VDD, and the gate end receives an external control signal (for the sake of distinction, referred to as a fourth external control signal, represented as RSEL) for controlling the conduction or disconnection of the second reset transistor 304. The fourth external control signal RSEL of the same row pixel unit 102 in the pixel array 10 is uniformly controlled.

[0058] The source end of the second source follower transistor 306 is connected with the drain end of the second read transistor 308, the drain end is connected with the power supply signal VDD, and the gate end is connected with the other end of the second photodiode 302 and the source end of the second reset transistor 304, respectively.

[0059] The source terminal of the second readout transistor 308 serves as the output terminal of the second pixel circuit 1024. The drain terminal is connected to the source terminal of the second source-follower transistor 306. The gate terminal is used to receive an external control signal (hereinafter referred to as the fifth external control signal, denoted as VSEL for ease of distinction) that controls the conduction or disconnection of the second readout transistor 308. The fifth external control signal VSEL controls all pixel units 102 in the same row in the pixel array 10.

[0060] In this embodiment, the fourth external control signal RSEL and the fifth external control signal VSEL of all rows of pixel units 102 in the pixel array 10 are driven by the same driving circuit (for example, Figure 5 The first drive circuit or Figure 6 The second driving circuit in the pixel unit 102 generates and sends it to the second pixel circuit 1024 in each pixel unit 102.

[0061] Based on this embodiment, a pixel circuit implemented using a 3T pixel structure is provided. This circuit can be used to detect light signals in relatively strong light environments, generate an electrical signal indicating exposure intensity, and perform pulse sequence imaging of relatively strong light. Furthermore, the second pixel circuit in the disclosed embodiment can also be implemented using other circuit structures. As long as the second pixel circuit can reset, expose, and generate and output an electrical signal indicating exposure intensity under the control of a driver circuit, the disclosed embodiment does not limit the specific structure of the second pixel circuit.

[0062] Figure 3 FIG. 1 is a structural diagram of another embodiment of the image sensor disclosed herein. Figure 3 As shown, based on any of the above embodiments, in some implementations, the signal conversion circuit 104 may include: an analog to digital converter (ADC) 1042, a comparator 1044, and a timer (COMP) 1046.

[0063] The input end of the analog-to-digital conversion circuit 1042 serves as the first input end of the signal conversion circuit 104, and is respectively connected to the output end of each first pixel circuit 1022 in the corresponding dimensional direction, receives the first electrical signal (V1) and the second electrical signal (V2) output by each first pixel circuit 1022 in the corresponding dimensional direction, and performs analog-to-digital conversion on the difference (V1-V2) between the first electrical signal and the second electrical signal output by each first pixel circuit to obtain a digital signal; the output end of the analog-to-digital conversion circuit 1042 is connected to the input end of the interface circuit 106, and outputs the digital signal to the interface circuit 106.

[0064] Optionally, in some implementations, the analog-digital conversion circuit 1042 can include a correlated double sampling circuit and an analog-digital converter, the difference (V1-V2) between the first electric signal and the second electric signal output by the same first pixel circuit is obtained through the correlated double sampling circuit, and the difference between the first electric signal and the second electric signal is converted from an analog signal to a digital signal through the analog-digital converter. The correlated double sampling circuit and the analog-digital converter can be implemented by any correlated double sampling circuit and analog-digital converter in the related art, and the present disclosure does not limit the specific structure and implementation of the analog-digital conversion circuit 1042.

[0065] One input end of the comparator 1044 is connected with the output end of each second pixel circuit 1024 in the corresponding dimension direction as the second input end of the signal conversion circuit 104, and receives the third electric signal (V3) output by each second pixel circuit 1024 in the corresponding dimension direction; the other input end of the comparator 1044 receives the preset threshold signal (Vref), which can be generated by the threshold signal generation circuit; the output end of the comparator 1044 is connected with the input end of the timer 1046, and is used to output the comparison result signal (1 or 0) to the timer 1046, wherein one input end is one of the positive input end and the negative input end, and the other input end is the other of the positive input end and the negative input end. Taking one input end as the positive input end and the other input end as the negative input end as an example, the comparator 1044 can output high level (1) as the comparison result signal when the input signal of the positive input end is greater than the input signal of the negative input end, i.e., the third electric signal output by the second pixel circuit is greater than the preset threshold signal Vref, otherwise, output low level (0) as the comparison result signal. Thus, according to the comparison result signal output by the comparator, it can be determined whether the third electric signal output by the second pixel circuit is greater than the preset threshold signal Vref. When the comparison result signal of the comparator 1044 changes from low level (0) to high level (1), it indicates that the third electric signal output by the second pixel circuit reaches the preset threshold signal Vref.

[0066] The input end of the timer 1046 is connected with the output end of the comparator 1044, receives the comparison result signal inputted by the comparator 1044, and counts the holding time of the comparison result signal according to a preset time frequency; the output end of the timer 1046 is connected with the input end of the interface circuit 106, and outputs the counting result as the first time when the comparison result signal changes (i.e. flips). For example, taking the case that one input end of the comparator is the positive input end and the other input end is the negative input end, the timer 1046 counts the time when the comparison result signal is 0, and obtains the time when the comparison result signal is 0 when the comparison result signal changes (i.e. changes from 0 to 1), which is the time used by the third electric signal outputted by the second pixel circuit to accumulate from 0 to the preset threshold signal Vref, i.e. can represent the strength of the light signal received by the second pixel circuit, and the shorter the counting result is, the stronger the light signal received by the second pixel circuit is. Thus, based on the counting result of the timer for the third electric signal outputted by the same second pixel circuit, the pulse sequence imaging based on the light signal detection can be realized.

[0067] In the embodiment, the case that one input end of the comparator is the positive input end and the other input end is the negative input end is taken as an example for description, and if one input end of the comparator is the negative input end and the other input end is the positive input end, it is just the opposite, and when the comparison result signal of the comparator 1044 changes from the high level (1) to the low level (0), it represents that the third electric signal outputted by the second pixel circuit reaches the preset threshold signal Vref, and then the timer counts the time when the comparison result signal is 1.

[0068] In the embodiment, in the same pixel unit 102, the first electric signal and the second electric signal outputted by the first pixel circuit 1022 are inputted into the column-level analog-digital conversion circuit 1042 for processing and outputting the digital signal, the third electric signal outputted by the second pixel circuit 1024 is inputted into one input end of the column-level comparator 1044, and compared with the preset threshold signal Vref inputted into the other input end of the comparator 1044, and the obtained comparison result controls the timer 1046 to stop counting, and the counting result (i.e. the first time) is obtained.

[0069] Figure 4 The structure schematic diagram of still another embodiment of the image sensor of the present disclosure is shown in FIG. 6. As shown in FIG. 6, in the embodiment, the first pixel circuit 1022 and the second pixel circuit 1024 are arranged in the same pixel unit 102, and the first pixel circuit 1022 and the second pixel circuit 1024 are connected with the column-level analog-digital conversion circuit 1042 and the column-level comparator 1044 respectively. Figure 4 Figure 3 ​On the basis of the shown embodiment, the signal conversion circuit 104 can further include a multiplexer (MUX) 1048. In some implementations, a first input of the multiplexer 1048 is connected to an output of the analog-digital conversion circuit 1042, for receiving a digital signal output by the analog-digital conversion circuit 1042; a second input of the multiplexer 1048 is connected to an output of the timer 1046, for receiving a timing result output by the timer 1046; and an output of the multiplexer 1048 serves as an output of the signal conversion circuit 104, and is connected to an input of the interface circuit 106, for outputting, to the interface circuit 106, the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time, or the first time as the output data of this time, and data identification information of the output data, the data identification information being used to identify whether the output data is a digital signal or the first time, so that the meaning of the data of this time is determined based on the data identification information, and accordingly a corresponding manner is used to determine the light signal intensity (exposure intensity) of the pixel unit detection space. The preset upper limit corresponding to the digital signal is a preset code value upper limit, and the preset code value upper limit is a code value corresponding to the maximum light intensity that can be distinguished by the first pixel circuit 1022; and the preset upper limit corresponding to the first time is a first preset time upper limit, and the first preset time upper limit is a time length corresponding to the maximum light intensity.

[0070] Further participate Figure 4 In some other implementations, a first input of the multiplexer 1048 is connected to an output of the analog-digital conversion circuit 1042, for receiving a digital signal output by the analog-digital conversion circuit 1042; a second input of the multiplexer 1048 is connected to an output of the timer 1046, for receiving a timing result output by the timer 1046; and an output of the multiplexer 1048 serves as an output of the signal conversion circuit 104, and is connected to an input of the interface circuit 106, for outputting, to the interface circuit 106, the digital signal and the first data identification information when the digital signal input by the analog-digital conversion circuit 1042 is received, the first data identification information being used to indicate that the output data of this time is a digital signal; and for outputting, to the interface circuit 106, the first time and the second data identification information when the first time input by the timer 1046 is received, the second data identification information being used to indicate that the output data of this time is the first time.

[0071] Correspondingly, the interface circuit 206 is provided with parallel input ports having the same number as the signal conversion circuits 204, as multiple input terminals, which are connected one-to-one with the output terminals of the multiple signal conversion circuits 204. The output data and data identification information of the multiple signal conversion circuits are output in sequence according to the order of the multiple signal conversion circuits 204, and the output data sequence of the multiple signal conversion circuits 204 in the same data output cycle (corresponding to the above-mentioned signal readout stage or the signal output stage of the second pixel circuit) is obtained, thereby realizing the parallel-serial conversion of the output data and data identification information of the multiple signal conversion circuits 204 for external transmission.

[0072] In this embodiment, the digital signal output by the first pixel circuit 1022 in the same pixel unit 102 and the timing result output by the timer 1046 are both input to the multiplexer 1048, and then uniformly output by the multiplexer 1048 to the interface circuit 106. The interface circuit 106 transmits the output data of the image sensor to the outside, for example, to a local terminal connected via wired communication, or to a server remotely connected via wireless communication, etc., so that the terminal or server can reconstruct the image based on the output data of the image sensor, calculate the light intensity of the detection space of the image sensor, etc. The embodiment of the present disclosure does not limit the transmission method, receiving object and subsequent application of the output data of the image sensor.

[0073] Figure 5 FIG. 1 is a structural diagram of another embodiment of the image sensor disclosed herein. Figure 5 As shown in the above Figures 1-4 Based on any of the embodiments shown, the image sensor of this embodiment may further include a first driving circuit 108, which is respectively connected to the first pixel circuit 1022 and the second pixel circuit 1024 in all the pixel units 102 in the pixel array 10, and outputs corresponding external control signals to the first pixel circuit 1022 and the second pixel circuit 1024.

[0074] Specifically, in some of these implementations, Figure 2In the implementation example in which each signal conversion circuit 104 is connected to a column of pixel units in the pixel array 10, the first driving circuit 108 can respectively control, for a row of pixel units 102 in the pixel array 10: reset the first pixel circuit 1022 and the second pixel circuit 1024 in the row of pixel units 102 according to a preset period, control the first pixel circuit 1022 in the row of pixel units 102 to enter an exposure phase for exposure according to a preset period with a first preset delay, control the second pixel circuit 1024 in the row of pixel units 102 to enter an exposure phase and a signal output phase for simultaneous exposure and third signal reading, and control the first pixel circuit 1022 in the row of pixel units 102 to enter a signal readout phase for sequentially reading the first signal and the second signal according to a preset period with a second preset delay; and

[0075] For the pixel array 1024: the first pixel circuit 102 in each row of pixel units 102 is controlled row by row for exposure in a rolling exposure mode with a third preset delay, and the second pixel circuit 1024 in each row of pixel units 102 is controlled row by row for exposure and third signal reading in a row by row exposure mode with a third preset delay.

[0076] Alternatively, in some other implementations, corresponding to an implementation example in which each signal conversion circuit 104 is connected to a row of pixel units in the pixel array 10 (not shown in the present disclosure), the first driving circuit 108 may, for each column of pixel units 102 in the pixel array 1010: control the first pixel circuits 1022 and the second pixel circuits 1024 in the column of pixel units 102 to reset according to a preset period; control the first pixel circuits 1022 in the column of pixel units 102 to enter an exposure phase for exposure according to a preset period with a first preset delay; control the second pixel circuits 1024 in the column of pixel units 102 to enter an exposure phase and a signal output phase for simultaneous exposure and third signal reading; control the first pixel circuits 1022 in the column of pixel units 102 to enter a signal readout phase for sequentially reading the first and second signals according to a preset period with a second preset delay; and

[0077] For the pixel array 10: a rolling exposure mode with a third preset delay is adopted to control the first pixel circuit 1022 in each column of pixel units 102 for exposure, and a column-by-column exposure mode with a third preset delay is adopted to control the second pixel circuit 1024 in each column of pixel units 102 for exposure and third signal reading.

[0078] Figure 6 FIG. 1 is a structural diagram of another embodiment of the image sensor disclosed herein. Figure 6 As shown in the above Figures 1-4Based on any of the illustrated embodiments, the image sensor of the embodiment can further include a first driving circuit 108 and a second driving circuit 110. In the embodiment, the first driving circuit 108 is connected to the first pixel circuit 1022 in each of the pixel units 102 in the pixel array 10 respectively, and outputs an external control signal to the first pixel circuit 1022; the second driving circuit 110 is connected to the second pixel circuit 1024 in each of the pixel units 102 in the pixel array 10 respectively, and outputs an external control signal to the second pixel circuit 1024.

[0079] Specifically, in some implementations, the first pixel circuit 1022 and the second pixel circuit 1024 are controlled by the first driving circuit 108 and the second driving circuit 110 respectively, and the first pixel circuit 1022 and the second pixel circuit 1024 are controlled by the first driving circuit 108 and the second driving circuit 110 respectively. Figure 2 In the implementation example corresponding to each signal conversion circuit 104 connected to a column of pixel units in the pixel array 10, the first driving circuit 110 can control the first pixel circuit 1022 in each row of pixel units 102 in the pixel array 10 respectively: reset the first pixel circuit 1022 in each row of pixel units 102 according to a preset period, control the first pixel circuit 1022 in each row of pixel units 102 to enter an exposure stage for exposure according to a first preset time delay according to a preset period, and control the first pixel circuit 1022 in each row of pixel units 102 to enter a signal readout stage for first signal and second signal readout in turn according to a second preset time delay according to a preset period; and control the first pixel circuit 1022 in each row of pixel units 102 in the pixel array 10 to enter an exposure stage for exposure in a mode of rolling exposure with a third preset time delay.

[0080] The second driving circuit 110 can control the second pixel circuit 1024 in each row of pixel units 102 in the pixel array 10 respectively: reset the second pixel circuit 1024 according to a preset period, control the second pixel circuit 1024 in each row of pixel units 102 to enter an exposure stage and a signal output stage for exposure and third signal readout at the same time according to a first preset time delay according to a preset period; and control the second pixel circuit 1024 in each row of pixel units 102 in the pixel array 10 to enter an exposure stage for exposure and third signal readout in a mode of rolling exposure with a third preset time delay.

[0081] Alternatively, in some other implementations, corresponding to an implementation example (not shown in the disclosure) in which each signal conversion circuit 104 corresponds to connecting a row of pixel units in the pixel array 10, the first driving circuit 108 can control the first pixel circuit 1022 in a column of pixel units 102 in the pixel array 10 to reset, enter the exposure stage for exposure, and sequentially perform the first signal and the second signal readout in the signal readout stage according to the preset period, the first preset time delay, and the second preset time delay, respectively; and control the first pixel circuit 1022 in each column of pixel units 102 in the pixel array 10 to expose in the mode of rolling exposure with the third preset time delay.

[0082] The second driving circuit 110 can control the second pixel circuit 1024 in a column of pixel units 102 in the pixel array 10 to reset, enter the exposure stage and the signal output stage for exposure and the third signal readout according to the preset period, the first preset time delay, and the second preset time delay, respectively; and control the second pixel circuit 1024 in each column of pixel units 102 in the pixel array 10 to expose and perform the third signal readout in the mode of column-by-column exposure with the third preset time delay.

[0083] Optionally, in some of the implementations, the photoelectric conversion gains of the first pixel circuit 1022 and the second pixel circuit 1024 are consistent, i.e., the photoelectric conversion gains of the first photodiode 202 and the second photodiode 302 are consistent, and the preset threshold value can be the signal size corresponding to the maximum light intensity that the first pixel circuit 1022 can distinguish.

[0084] Alternatively, in some of the implementations, the photoelectric conversion gains of the first pixel circuit 1022 and the second pixel circuit 1024 can also be inconsistent, i.e., the photoelectric conversion gains of the first photodiode 202 and the second photodiode 302 are inconsistent, and the preset threshold value is a signal size determined based on the signal size corresponding to the maximum light intensity that the first pixel circuit 1022 can distinguish and the ratio between the photoelectric conversion gain of the second pixel circuit 1024 and the photoelectric conversion gain of the first pixel circuit 1022. For example, assuming that the signal size corresponding to the maximum light intensity that the first pixel circuit 1022 can distinguish is Vm, the photoelectric conversion gain of the first pixel circuit 1022 is G1, and the photoelectric conversion gain of the second pixel circuit 1024 is G2, the preset threshold value can be determined based on Vm*(G2 / G1), which is not limited in the embodiments of the disclosure.

[0085] Based on this example, the photoelectric conversion gains of the first pixel circuit 1022 and the second pixel circuit 1024 may also be inconsistent. In this case, the coverage of different light intensity ranges can be achieved by adjusting the flip threshold of the comparator.

[0086] In some implementations, for each first pixel circuit 1022: the first pixel circuit can be controlled to enter the exposure stage after being reset for exposure. In the exposure stage (the exposure duration is the first duration), the first pixel circuit receives the light signal during the exposure process to generate photogenerated charges and accumulate them; before the end of the exposure stage of the first pixel circuit, the first pixel circuit is controlled to enter the signal readout stage (the duration is the second duration). In the first stage of the signal readout stage, the first pixel circuit outputs the reset signal as the first electrical signal. In the embodiment of the present disclosure, the signal readout stage can be divided into two stages: a first stage and a second stage located after the first stage; after the exposure stage of the first pixel circuit ends, the first pixel circuit is controlled to enter the second stage of the signal readout stage. In the second stage, the first pixel circuit outputs the exposure signal generated by the accumulated photogenerated charges as the second electrical signal.

[0087] In some implementations, for each second pixel circuit 1022: the second pixel circuit can be controlled to enter the exposure stage and the signal output stage after being reset. In the exposure stage, the second pixel circuit receives the light signal during the exposure process to generate photogenerated charges, and outputs the exposure signal generated by the photogenerated charges as the third electrical signal. The exposure stage and the signal output stage of the second pixel circuit are consistent in time period, and the duration of the exposure stage and the signal output stage of the second pixel circuit are respectively the third duration; the time required for the third electrical signal output by the second pixel circuit in the signal output stage to reach a preset threshold is obtained.

[0088] Figure 7 This is a row drive operation timing diagram in an embodiment of the present disclosure. Figure 7 ,by Figure 5 or Figure 6 Taking the structure of the image sensor shown in FIG. 1 as an example, for the first pixel circuit 1022, a rolling exposure mode is adopted:

[0089] First, the reset signal (i.e., the second external control signal) RST[1] controlling the gate terminal of the first reset transistor 208 in the first row of pixel units 102 is set to a high level, the first reset transistor 208 is turned on, and the capacitor 206 in the first pixel circuit 1022 is reset; then, RST[1] is set to a low level, and the first external control signal TX[1] controlling the gate terminal of the transfer transistor 204 is set to a high level, the transfer transistor 204 is turned on, and the charge on the first photodiode 202 is transferred to the capacitor 206, clearing the charge in the first photodiode 202. Wherein, [1] represents the first row;

[0090] After that, TX[1] is set to low level, transfer transistor 204 is turned off, and the first pixel circuit 1022 in the first row of pixel units 102 enters the exposure stage for exposure. In the exposure stage (i.e. Figure 7 In the first row of 4T pixel exposure time, the incident light signal received by the first pixel circuit 1022 is converted into photo-generated charges by the first photodiode 202 and accumulated in the first photodiode 202;

[0091] When the exposure of the first pixel circuit 1022 in the first row of pixel units 102 is about to end, RST[1] is set to high level, and the first reset transistor 208 is turned on to reset the capacitor 206 in the first pixel circuit 1022; after the capacitor 206 is reset, the read signal (i.e. the third external control signal) Read[1] controlling the gate end of the first read transistor 312 is set to high level, the reset signal of the first pixel circuit 1022 at this time is read out as the first electric signal to the column-level connected analog-digital conversion circuit 1042, Read[1] is continuously maintained at high level, TX[1] is set to high level, the transfer transistor 204 is turned on, the photo-generated charges accumulated in the first photodiode 202 during this exposure process are transferred to the capacitor 206, and the exposure signal generated during this exposure stage is read out as the second electric signal to the column-level connected analog-digital conversion circuit 1042 through the first read transistor 312;

[0092] The analog-digital conversion circuit 1042 performs analog-digital conversion on the difference (V1-V2) between the first electric signal and the second electric signal to obtain a digital signal, thereby converting the exposure information into a digital signal that is easy to transmit and process.

[0093] Since the signal conversion circuit 104 can only read out the signal output by one row of pixel units in one column of pixel units at a time, the RST, TX and Read signals of the second row of pixel units are delayed by one readout time (i.e. the length of the third preset time delay) compared to the first row of pixel units, and so on. When the signal output by the last row of pixel units is read out, the signal reading returns to the first row of pixel units, and the cycle continues.

[0094] For the second pixel circuit 1024, the mode of exposure by row is adopted:

[0095] For 3T pixels, the mode of exposure by row is adopted:

[0096] First, the reset signal (i.e., the fourth external control signal) RSel[1] controlling the gate terminal of the second reset transistor 304 in the first row of pixel units 102 is set to a high level, the second reset transistor 304 is turned on, and the second photodiode 302 in the second pixel circuit 1024 is reset. Wherein, [1] represents the first row;

[0097] After the second photodiode 302 is reset, RSel[1] is set to a low level, and the output signal (i.e., the fifth external control signal) Vsel[1] that controls the gate terminal of the second readout transistor 308 is set to a high level. The second pixel circuit 1024 in the first row of pixel units 102 enters the exposure phase for exposure, and the incident light signal received by the second pixel circuit 1024 is converted into photogenerated charge by the second photodiode 302. In the exposure phase (i.e., Figure 7 During the exposure time of 3T pixels in the first row), Vsel[1] is always kept at a high level, and Vsel of the second pixel circuit 1024 in the pixel units 102 in the other rows is kept at a low level. The second readout transistor 308 in the pixel unit 102 in the first row is kept turned on. Therefore, the exposure signal (i.e., the third electrical signal) of the second pixel circuit 1024 in the pixel unit 102 in the first row is always kept connected to the column-level connected comparator 1044. The column-level connected timer 1046 starts from zero at the rising edge of Vsel[1] (i.e., the start moment of setting it to a high level) and counts according to the preset timing frequency. During the exposure phase of the second pixel circuit 1024 in the pixel unit 102 in the first row, the photogenerated charge gradually accumulates. When the photogenerated charge When the charge accumulates to a preset threshold, the comparison result signal output by the comparator 1044 is triggered to flip, for example, from 0 to 1 (the third electrical signal is connected to the positive input terminal of the comparator, and the electrical signal of the preset threshold is connected to the negative input terminal of the comparator) or from 1 to 0 (the third electrical signal is connected to the negative input terminal of the comparator, and the electrical signal of the preset threshold is connected to the positive input terminal of the comparator). After the comparison result signal output by the comparator 1044 flips, the corresponding timer 1046 is controlled to stop timing. The timing result at this time (i.e., the first time) can represent the strength of the light signal received by the second pixel circuit 1024 in the first row of pixel units 102. The shorter the timing result, the stronger the light signal received by the second pixel circuit 1024.

[0098] During the exposure process of the second pixel circuits 1024 in the current row, the timer 1046 corresponding to each second pixel circuit 1024 stops timing at different times, ie, the timing results (ie, the first time) are different, while the duration for which Vsel maintains a high level is fixed. For example, the duration for each Vsel to maintain a high level can be set to 8 microseconds (us), and the timing frequency of the timer 1046 can be set to 125 MHz, that is, 1 is added every 8ns timing unit. Then, during the exposure time of 8us, the second pixel circuit 1024 in the current row may reach the preset threshold received by the comparator 1044 after 80ns of exposure, causing the comparison result signal to be flipped, while the remaining second pixel circuits 1024 in the current row have not yet reached the preset threshold received by the comparator 1044. Then, the timer 1046 corresponding to the second pixel circuit 1024 that reaches the preset threshold stops timing, and the remaining second pixel circuits 1024 continue to be exposed, and the corresponding timer 1046 also continues timing until the exposure time of 8us ends, and all timing results of the second pixel circuits 1024 in the entire row are uniformly output, and then the exposure of the second pixel circuits 1024 in the next row is carried out, and the timer 1046 also starts timing again.

[0099] When the second pixel circuit 1024 is exposed and outputs signals, the signal conversion circuit 104 can only read one row of the second pixel circuit 1024 at a time. Therefore, the RSel and VSel signals of the second pixel circuit 1024 in the second row are delayed by the length of the exposure time (i.e., the third preset delay) compared to the first row, and so on. When the signal output of the second pixel circuit 1024 in the last row is completed, the signal is read back to the first row of the second pixel circuit 1024, and the cycle is repeated.

[0100] In the disclosed embodiment, the frame period of an image sensor is defined as the length of time from the start of reading the signals from the first row of pixel units to the completion of reading the signals from the last row of pixel units. It can be seen that the time from the start of reading the signals from the first row to the completion of reading the signals from the last row of pixel units 1022 is aligned with the time from the start of reading the signals from the first row to the completion of reading the signals from the second pixel circuit 1024. In this case, the frame period of the first pixel unit 1022 or the second pixel circuit 1024 is the same as the overall frame period of the entire image sensor.

[0101] For traditional image sensors based on a 4T pixel structure, their dynamic range depends largely on the ADC's resolution. For example, a common 12-bit ADC can output 4096 code values, from 0 to 4095. If the maximum voltage is 2 volts (V), the minimum resolvable voltage is approximately 0.5 millivolts (mV). The ratio between the maximum and minimum signals is approximately 4000 times, which translates to approximately 72 decibels (dB) in dynamic range, a typical value for traditional image sensors.

[0102] In the disclosed embodiment, the difference between the first electrical signal and the second electrical signal output by the first pixel circuit of the 4T pixel structure is converted into digital signal by a 12-bit ADC, which covers a dynamic range of 72 dB. The third electrical signal output by the second pixel circuit of the 3T pixel structure can be used to represent a dynamic range higher than the aforementioned 72 dB. The following description takes the same photoelectric conversion gain of the first pixel circuit and the second pixel circuit as an example:

[0103] Suppose the pixel array in the image sensor includes 1000 rows of pixel units, the signal reading time (i.e. the length of the signal readout stage) of each row of pixel units is set to 8us, the counting frequency of the timer is 125MHz, the minimum counting unit is 8ns, and the maximum counting result is 1000, which corresponds to 8us. The exposure time (i.e. the length of the exposure stage) of the 1000 rows of first pixel circuits is about 8ms, and the exposure time of each row of second pixel circuits is 8us. The frame period is about 8ms, and the frame rate is about 120 frames per second. If the voltage signal generated by the first pixel circuit exposed for 8ms under the light intensity of 0.1 lux is 0.5mV, which is the minimum light intensity that the image sensor can distinguish, then the voltage signal generated by the first pixel circuit exposed for 8ms under the light intensity of 400lux is about 2V, reaching the maximum voltage. Under the light intensity of 400lux, the voltage signal generated by the second pixel circuit exposed for 8us is about 2mV. If the signal (referred to as the flip voltage) of the preset threshold compared by the comparator is 2mV, then under the light intensity of 400lux, the counting result of the second pixel circuit is exactly the maximum value 1000, and the minimum value of the counting result of the second pixel circuit corresponds to 8ns exposure to reach the signal voltage of 2mV, i.e. 400,000 lux. Therefore, in the image sensor, the first pixel circuit can cover the light intensity of 0.1 lux to 400 lux, the second pixel circuit can cover the light intensity of 400 lux to 400,000 lux, and the overall dynamic range can cover the light intensity of 0.1 lux to 400,000 lux, which is 1000 times higher than the dynamic range of the traditional image sensor based on the 4T pixel structure, i.e. 132dB.

[0104] In the above example, the frame period is 8ms, the exposure time is 8us, the counting frequency is 125Mhz, the resolution is 12bit, and the number of rows of pixel units is 1000. These parameters are only used to illustrate the effect of the disclosed embodiment, and their specific values can be adjusted in actual application. They do not constitute a limitation on the implementation scheme or the achievable effect of the present disclosure.

[0105] Based on Figure 7As shown in the row driving operation timing diagram, the third external control signal Read of the first pixel circuit 1022 and the fifth external control signal Vsel of the second pixel circuit 1024 have the same waveform, and the second external control signal Rst of the first pixel circuit 1022 and the fourth external control signal RSel of the second pixel circuit 1024 have the same waveform. Thus, the third external control signal Read and the fifth external control signal Vsel of each row of pixel units can be controlled by the same control signal, and the second external control signal Rst and the fourth external control signal RSel of each row of pixel units can be controlled by the same control signal. Thus, the first pixel circuit and the second pixel circuit can be controlled by the same driving circuit, and only one driving circuit can be provided in the image sensor. Figure 5 As shown, the third external control signal Read of the first pixel circuit 1022 and the fifth external control signal Vsel of the second pixel circuit 1024 have the same waveform, and the second external control signal Rst of the first pixel circuit 1022 and the fourth external control signal RSel of the second pixel circuit 1024 have the same waveform. Figure 6 The right second driving circuit can be omitted, thereby simplifying the structure of the image sensor.

[0106] Alternatively, in some implementations, the exposure stage and the signal readout stage of the first pixel circuit 1022 correspond to the exposure stage and the signal output stage of the second pixel circuit 1024, respectively. That is, the exposure stage of the first pixel circuit 1022 and the exposure stage of the second pixel circuit 1024 are consistent in time and duration, and the signal readout stage of the first pixel circuit 1022 and the signal output stage of the second pixel circuit 1024 are also consistent in time and duration. In this case, in the operation 108 of the above embodiment, the multi-way selector 1048 can output the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time or the first time as the output data of this time, and data identification information of the output data. The data identification information is used to identify that the output data of this time is a digital signal or a first time. The preset upper limit corresponding to the digital signal is a preset code value upper limit, which is a code value corresponding to the maximum light intensity that can be distinguished by the first pixel circuit 1022. In actual applications, it can be set as the maximum code value supported by the ADC. The preset upper limit corresponding to the first time is a first preset time upper limit, which is the duration corresponding to the maximum light intensity.

[0107] From the above analysis, for a pixel unit, the exposure stage of the first pixel circuit 1022, the signal readout stage, and the exposure stage of the second pixel circuit 1024, the signal output stage are respectively consistent in time and duration. In the digital signal output by the ADC and the timing result output by the timer, only one result is actually accurate. For each pixel unit, the digital signal output by the ADC and the timing result output by the timer are simultaneously input into a multiplexer. The multiplexer determines which result is accurate. The multiplexer can be configured to output only one accurate result of the two as the output data of this time, and a data flag bit as data identification information. The data flag bit is used to identify whether the data of this time is the digital signal obtained by the ADC or the timing result. In an implementation example, the selection criterion of the multiplexer can be set to output the result that does not reach the corresponding preset upper limit of the two results. Because for weak light intensity, the digital signal of the ADC is accurate and will not reach the maximum code value of the ADC. Because weak light intensity cannot generate a 2mV signal within 8us, the comparison result signal of the comparator will not flip, and the timer will continue to count until the maximum timing result. Therefore, the timing result must reach the maximum value. By the same reasoning, for strong light intensity, the timing result is accurate, because the digital signal of the ADC will reach the maximum code value, and the timing result will not reach the maximum value. Through the selection of the multiplexer, each pixel unit can output at most 13bit data to express 132dB light intensity. While realizing high dynamic range, the data quantity is also guaranteed to be compact.

[0108] In the embodiment of the present disclosure, for the same pixel unit, after outputting the data and data identification information based on the embodiment of the present disclosure, at the receiving end, based on the data identification information, it can be determined whether the output data is a digital signal or a timing result. For a digital signal, the larger the digital signal, the greater the light intensity in the detection space of the pixel unit within the corresponding time. The analog voltage value represented by the digital signal can be calculated by the code value represented by the digital signal and the reference voltage of the ADC (i.e., the maximum voltage that can be distinguished as described above). Then, based on the pre-set corresponding relationship coefficient k between the light intensity value and the voltage value, k is a constant greater than zero, the light intensity value within the corresponding time can be estimated. For example, assuming that the reference voltage of the 12-bit ADC is 2V and its output code value is 2048, then the corresponding analog voltage value can be obtained as approximately 1V based on (2048 / 4095)×2V≈1V. Then, based on i=k*analog voltage value, the light intensity value i within the time can be obtained. For the timing result (expressed as T), the shorter the timing result, the greater the light intensity of the pixel unit detection space within the corresponding time. The light intensity value within the corresponding time can be estimated based on the timing result T in a preset manner. For example, the light intensity value i within the time can be obtained in the following manner: i=a*1 / T. Wherein, a is a constant greater than zero, and the specific value of a can be preset according to actual needs. Thus, based on the output data and data identification information of the pixel unit each time, the light intensity value of its detection space within the corresponding time can be determined. Based on the output data and data identification information output by the pixel unit multiple times and arranged based on a time series relationship, the light intensity value and its change in the detection space in time series can be correspondingly determined.

[0109] For an image sensor including a pixel array formed by m rows and n columns of pixel units, each pixel unit in the pixel array works independently of each other, and respectively realizes time-domain acquisition of the light signal of a space in the observation scene to obtain output data (digital signal or timing result) and data identification information for output. The spaces corresponding to different pixel units do not overlap, and the pixel units in the pixel array cooperate with each other to cover the entire observation scene, thereby realizing time-domain acquisition of the light signal in the entire observation scene, outputting data (digital signal or timing result) and data identification information, and obtaining light intensity values. As a result, the light intensity value and its change of the light signal corresponding to the entire observation scene can be obtained.

[0110] Optionally, in other implementations, at least one of the exposure phase of the first pixel circuit 1022, the signal readout phase, and the exposure phase of the second pixel circuit 1024, the signal output phase do not correspond, i.e., at least one of the exposure phase of the first pixel circuit 1022 and the exposure phase of the second pixel circuit 1024 is inconsistent in time and duration, and / or the signal readout phase of the first pixel circuit 1022 and the signal output phase of the second pixel circuit 1024 are consistent in at least one of time and duration. At this time, in the operation 108 of the above embodiment, in response to obtaining the digital signal, the digital signal and the first data identification information can be directly output, the first data identification information being used to indicate that the output data this time is a digital signal; in response to obtaining the first time, the first time and the second data identification information can be directly output, the second data identification information being used to indicate that the output data this time is the first time. In this implementation, the operation 108 can be performed by the multiplexer 1048, or the multiplexer 1048 can not be provided, and the analog-to-digital conversion circuit 1042 and the timer 1046 directly output to the interface circuit 106, which is output by the interface circuit 106, and the present embodiment does not limit this.

[0111] In a specific implementation, the exposure time and frame period of the first pixel circuit 1022 and the second pixel circuit 1024 can also be inconsistent, for example, the exposure time and frame period of the first pixel circuit can be extended to improve the photosensitive effect of the image sensor on weaker light intensity. At this time, since the output period of the digital signal and the timing result is different, in the present embodiment, both the digital signal and the timing result are output, and the data identification information thereof is output at the same time, and based on the data identification information, it can be determined outside the image sensor that the corresponding data is a digital signal or a timing result, and based on the digital signal and the timing result, the light intensity represented thereby or a composite image signal is calculated, etc. In this case, the working mode of the multiplexer can be adjusted, and the two signals are not selected and output to the interface circuit; or, the multiplexer can not be provided, and the data obtained by the ADC and the timer is directly output to the interface circuit, and the interface circuit performs parallel-serial conversion on the data output by each pixel unit in the pixel array to form a data sequence and then outputs the data sequence to the outside, thereby further simplifying the structure of the image sensor; etc., and the present embodiment does not limit this.

[0112] In the implementation, for the same pixel unit, after the output of the current data and the data identification information by the embodiment of the present disclosure, at the receiving end, the light intensity value in the corresponding time can be estimated in a corresponding manner based on the first data identification information after determining that the current data is output as a digital signal, and details can be referred to the above implementation, which will not be repeated here. At the same time, the light intensity value in the corresponding time can be estimated in a corresponding manner based on the second data identification information after determining that the current data is a timing result, and details can be referred to the above implementation, which will not be repeated here. Based on the digital signal and the first data identification information output by the first pixel circuit in the pixel unit multiple times and arranged based on the timing relationship, the light intensity value and its change in the timing of the detection space of the pixel unit can be determined correspondingly. Similarly, based on the timing result and the first data identification information output by the second pixel circuit in the pixel unit multiple times and arranged based on the timing relationship, the light intensity value and its change in the timing of the detection space of the pixel unit can be determined correspondingly. Then, the light intensity values obtained by the two methods can be aligned in time, so as to obtain the light intensity value and its change in the timing of the detection space of the pixel unit. In addition, the embodiment of the present disclosure can also determine the light intensity value and its change in the timing of the detection space of the pixel unit based on the two signals and the data identification information in other manners, and the embodiment of the present disclosure does not limit this.

[0113] For the image sensor including the pixel array arranged by m rows and n columns of pixel units, each pixel unit in the pixel array works independently, respectively realizes the time domain acquisition of the light signal of one space in the observation scene to obtain the output data (digital signal or timing result) and the data identification information, the spaces corresponding to different pixel units do not overlap, and each pixel unit in the pixel array cooperates with each other to cover the entire observation scene, so as to realize the time domain acquisition of the light signal in the entire observation scene, the output of the data (digital signal or timing result) and the data identification information, and the acquisition of the light intensity value. Thus, the light intensity value and its change of the light signal corresponding to the entire observation scene can be obtained.

[0114] Optionally, in one application of the embodiment of the present disclosure, after the light intensity value and its change of the light signal corresponding to the entire observation scene are obtained, target detection can be performed based on the light intensity value of one or more spaces in the observation scene at each time. Since the visual information of a specific target is basically unchanged in a specific observation scene, the light intensity value of one or more space regions in the observation scene in the time domain can be used to realize the tracking of the target.

[0115] Optionally, in another application of the embodiments of the present disclosure, after obtaining the light intensity values of the corresponding spaces at corresponding times, a preset pulse reconstruction algorithm can be used to generate a reconstructed image of the observation scene at each sampling time based on the light intensity values of all spaces in the observation scene at each time. The gray value of each pixel point in the reconstructed image represents the light intensity value of the space region corresponding to the pixel unit corresponding to the pixel point, i.e., the pixel value of each pixel point (i, j) in the reconstructed image at each time is the light intensity value of the space region corresponding to the pixel unit corresponding to the pixel point.

[0116] The preset pulse reconstruction algorithm can be, for example, a pulse reconstruction algorithm (TFI) based on an interspike interval (ISI), a pulse reconstruction algorithm (TFP) based on a fixed window sliding, a pulse reconstruction algorithm based on a convolutional neural network (CNN), etc. The embodiments of the present disclosure do not limit the specific pulse reconstruction algorithm used.

[0117] The image sensor of the embodiments of the present disclosure can use various exposure incident modes, for example, a normal incidence exposure mode or a back-illumination exposure mode. In addition, different shutter modes can also be used, for example, a global exposure shutter and a rolling shutter. The embodiments of the present disclosure do not limit this.

[0118] Any of the signal processing methods provided by the embodiments of the present disclosure can be executed by any appropriate device with data processing capability, including but not limited to: terminal devices and servers, etc. Alternatively, any of the signal processing methods provided by the embodiments of the present disclosure can be executed by a processor, for example, the processor executes any of the signal processing methods mentioned by the embodiments of the present disclosure by calling corresponding instructions stored in a memory. The following will not be described in detail.

[0119] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction-related hardware. The aforementioned program can be stored in a computer-readable storage medium, and the program is executed to perform the steps of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0120] The image sensor and the signal processing method of the embodiments of the present disclosure correspond to each other in terms of specific implementation and beneficial technical effects. The related contents can be mutually referred, cited, combined, and will not be described in detail.

[0121] The embodiments of the present disclosure also provide an electronic device, which includes: a processor, a memory connected to the processor in communication, and an image sensor provided by any of the embodiments of the present disclosure. Wherein:

[0122] The memory is used to store computer execution instructions.

[0123] The processor is configured to execute computer-executable instructions stored in the memory to control the image sensor to perform corresponding operations.

[0124] The electronic device in the embodiments of the present disclosure can be any of the following devices or applied to any of the following devices: image data acquisition device, audio / video player, navigation device, entertainment device, communication device, roadside traffic facility, device in motor vehicle, industrial detection device, flight device, medical device, security device, etc.

[0125] In the embodiments of the present disclosure, the image data acquisition device can include but is not limited to various cameras (such as pulse camera, high-speed camera, visual camera, motion or wearable camera, etc.), various cameras (such as mobile phone camera, vehicle-mounted camera, traffic camera, security camera, camera installed on flyable object or other various devices, etc.), etc.; the entertainment device can include but is not limited to wearable device (such as AR / VR glasses, sports watch, etc.), AR / VR projection device, etc.; the communication device can include but is not limited to router, server with various functions (such as data storage function, authentication function, etc.), etc.; the roadside traffic facility can include but is not limited to traffic signal, street lamp, roadblock, road sign, traffic sign, limiting rod, warning triangular stand, etc.; the device in motor vehicle can include but is not limited to various devices arranged on vehicles such as car, train, subway, airplane, etc.; the flight device can include but is not limited to airplane, aircraft, aerial vehicle, etc.

[0126] Figure 8 The structure schematic diagram of an application embodiment of the electronic device of the present disclosure is shown in FIG. 1. Hereinafter, the electronic device according to the embodiments of the present disclosure is described with reference to FIG. 1. Figure 8 The electronic device can be any one or both of the first device and the second device, or a single device independent of them, which can communicate with the first device and the second device to receive the acquired input signals therefrom. Figure 8 The structure schematic diagram of an application embodiment of the electronic device of the present disclosure is shown in FIG. 1. Hereinafter, the electronic device according to the embodiments of the present disclosure is described with reference to FIG. 1. Figure 8 The electronic device can be any one or both of the first device and the second device, or a single device independent of them, which can communicate with the first device and the second device to receive the acquired input signals therefrom.

[0127] As shown in FIG. 1, the electronic device includes one or more processors, memory and image sensor. The processor can be a central processing unit (CPU) or other forms of processing unit with data processing capability and / or instruction execution capability, and can control other components in the electronic device, such as the image sensor, to perform desired functions. Figure 8

[0128] ​The memory can store one or more computer program products, which can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk, flash memory, and / or the like. One or more computer program products can be stored on the computer-readable storage media, and the processor can execute the computer instructions to implement the signal processing method of various embodiments of the present disclosure described above and / or other desired functions.

[0129] In one example, the electronic device can further include an input device and an output device, which are interconnected through a bus system and / or other forms of connection mechanism (not shown). In addition, the input device can further include, for example, a keyboard, a mouse, and / or the like. The output device can output various information, including determined distance information, direction information, and / or the like, to the outside. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0130] Of course, in order to simplify, Figure 8 In the electronic device, only some of the components related to the present disclosure are shown, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device can further include any other appropriate components according to specific application cases.

[0131] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, benefits, effects, and / or the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, benefits, effects, and / or the like cannot be considered as mandatory for each embodiment of the present disclosure. In addition, the above-described specific details are only for the purpose of illustration and understanding, and do not limit the present disclosure to the above-described specific details.

[0132] Each embodiment in the present specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between each embodiment can be mutually referred to.

[0133] The block diagrams of devices, apparatuses, equipment, systems referred to in this disclosure are merely illustrative examples and are not intended to require or imply that the connection, arrangement, configuration must be as shown in the block diagrams. These devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner as will be appreciated by those skilled in the art. Words such as "include," "contain," "have," and the like are open-ended words that are intended to mean "including but not limited to," and are to be used interchangeably. The words "or" and "and" as used herein are intended to mean "and / or," and are to be used interchangeably. The word "such as" as used herein is intended to mean "such as but not limited to," and is to be used interchangeably.

[0134] The devices of the present disclosure can be implemented in a number of ways. For example, the methods and apparatus of this disclosure can be implemented by software, hardware, firmware or any combination of software, hardware, firmware.

[0135] It is also important to note that the devices of the present disclosure can be embodied in a variety of ways. These variations are contemplated as being within the scope of the present disclosure.

[0136] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0137] The above description has been given for the purpose of illustration and description. Furthermore, this description does not purport to be exhaustive or to limit the embodiments of the disclosure to the precise forms disclosed. Although various example aspects and embodiments have been discussed above, those of ordinary skill in the art will appreciate a variety of modifications, alternatives, permutations, additions and sub-combinations.

Claims

1. An image sensor, comprising: A pixel array, a plurality of signal conversion circuits, and an interface circuit; The pixel array is formed by arranging multiple rows and columns of pixel units, each of which includes a first pixel circuit and a second pixel circuit; each of the signal conversion circuits is correspondingly connected to the first pixel circuit and the second pixel circuit in the pixel units in one dimension direction of the pixel array, where the pixel units in one dimension direction are a row of pixel units or a column of pixel units; The output end of the first pixel circuit is connected to the first input end of the corresponding signal conversion circuit, and outputs a first electrical signal and a second electrical signal to the corresponding signal conversion circuit; wherein the second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on a light signal received during the exposure process after the first pixel circuit is reset; and the first electrical signal is an electrical signal generated by resetting the first pixel circuit during the exposure phase; The output terminal of the second pixel circuit is connected to the second input terminal of the corresponding signal conversion circuit, and outputs a third electrical signal to the corresponding signal conversion circuit; wherein the third electrical signal is an electrical signal continuously output by the second pixel circuit during an exposure phase, and the third electrical signal is obtained by generating photogenerated charges based on a light signal received during the exposure process after the second pixel circuit is reset; The first input end of the signal conversion circuit is respectively connected to the output end of each first pixel circuit in the corresponding dimensional direction, the second input end of the signal conversion circuit is respectively connected to the output end of each second pixel circuit in the corresponding dimensional direction, the output end of the signal conversion circuit is connected to the input end of the interface circuit, and outputs to the interface circuit a digital signal obtained by analog-to-digital conversion of the difference between the first electrical signal and the second electrical signal output by the same pixel unit, and / or a first time required for the third electrical signal output by the same pixel unit to reach a preset threshold; The multiple input terminals of the interface circuit are respectively connected to the output terminals of the multiple signal conversion circuits, and the output terminals of the interface circuit sequentially output the output data of the multiple signal conversion circuits.

2. The image sensor according to claim 1, wherein The first pixel circuit includes: a first photodiode, a transfer transistor, a capacitor, a first reset transistor, a first source follower transistor and a first readout transistor; One end of the first photodiode is grounded, and the other end is connected to one end of the capacitor and the gate end of the first source follower transistor respectively through the transfer transistor; The source terminal of the transfer transistor is connected to the other end of the first photodiode, the drain terminal is connected to one end of the capacitor and the gate terminal of the first source follower transistor respectively, and the gate terminal receives an external control signal; One end of the capacitor is respectively connected to the source terminal of the first reset transistor and the gate terminal of the first source follower transistor, and is connected to the other end of the first photodiode through the transfer transistor, and the other end is grounded; The source terminal of the first reset transistor is connected to one end of the capacitor, the drain terminal is connected to a power signal, and the gate terminal receives an external control signal; The source terminal of the first source follower transistor is connected to the drain terminal of the first readout transistor, the drain terminal is connected to a power signal, and the gate terminal is respectively connected to one end of the capacitor and the drain terminal of the transfer transistor; The source terminal of the first readout transistor serves as the output terminal of the first pixel circuit, the drain terminal is connected to the source terminal of the first source follower transistor, and the gate terminal receives an external control signal.

3. The image sensor according to claim 1, wherein The second pixel circuit includes: a second photodiode, a second reset transistor, a second source follower transistor and a second readout transistor; One end of the second photodiode is grounded, and the other end is connected to the source end of the second reset transistor and the gate end of the second source follower transistor respectively; The source terminal of the second reset transistor is connected to the other end of the second photodiode, the drain terminal is connected to the power signal, and the gate terminal receives an external control signal; The source terminal of the second source follower transistor is connected to the drain terminal of the second readout transistor, the drain terminal is connected to a power signal, and the gate terminal is respectively connected to the other end of the second photodiode and the source terminal of the second reset transistor; The source terminal of the second readout transistor serves as the output terminal of the second pixel circuit, the drain terminal is connected to the source terminal of the second source follower transistor, and the gate terminal receives an external control signal.

4. The image sensor according to any one of claims 1 to 3, wherein: The signal conversion circuit includes: an analog-to-digital conversion circuit, a comparator and a timer; The input end of the analog-to-digital conversion circuit serves as the first input end of the signal conversion circuit, and is respectively connected to the output end of each first pixel circuit in the corresponding dimensional direction, receives the first electrical signal and the second electrical signal output by each first pixel circuit, and performs analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal output by each first pixel circuit to obtain a digital signal; the output end of the analog-to-digital conversion circuit is connected to the input end of the interface circuit, and outputs the digital signal to the interface circuit; One input end of the comparator serves as the second input end of the signal conversion circuit and is respectively connected to the output end of each second pixel circuit in the corresponding dimensional direction to receive the third electrical signal output by each second pixel circuit; another input end of the comparator receives the signal of the preset threshold; the output end of the comparator is connected to the input end of the timer to output a comparison result signal to the timer; The input end of the timer is connected to the output end of the comparator; the output end of the timer is connected to the input end of the interface circuit, and when the comparison result signal changes, the timing result is output as the first time.

5. The image sensor according to claim 4, wherein: The signal conversion circuit further includes: a multiplexer; The first input terminal of the multiplexer is connected to the output terminal of the analog-to-digital conversion circuit to receive the digital signal output by the analog-to-digital conversion circuit; The second input terminal of the multiplexer is connected to the output terminal of the timer to receive the timing result output by the timer; The output end of the multiplexer serves as the output end of the signal conversion circuit, is connected to the input end of the interface circuit, and outputs the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time or the first time as the output data of this time, as well as data identification information of the output data to the interface circuit.

6. The image sensor according to claim 4, wherein: The signal conversion circuit further includes: a multiplexer; The first input terminal of the multiplexer is connected to the output terminal of the analog-to-digital conversion circuit to receive the digital signal output by the analog-to-digital conversion circuit; The second input terminal of the multiplexer is connected to the output terminal of the timer to receive the timing result output by the timer; The output end of the multiplexer serves as the output end of the signal conversion circuit, is connected to the input end of the interface circuit, outputs the received digital signal and the first data identification information of the digital signal, and outputs the received first time and the second data identification information of the first time.

7. The image sensor according to any one of claims 1 to 3, wherein: Also comprising a first drive circuit; The first driving circuit is respectively connected to the first pixel circuit and the second pixel circuit in all pixel units in the pixel array, and outputs external control signals to the first pixel circuit and the second pixel circuit.

8. The image sensor according to any one of claims 1 to 3, wherein: It also includes a first driving circuit and a second driving circuit; the first driving circuit is respectively connected to the first pixel circuits in all pixel units in the pixel array, and outputs an external control signal to the first pixel circuits; The second driving circuit is respectively connected to the second pixel circuits in all pixel units in the pixel array, and outputs an external control signal to the second pixel circuits.

9. An electronic device comprising: A processor, a memory respectively connected to the processor for communication, and an image sensor according to any one of claims 1 to 8.

10. The electronic device according to claim 9, wherein the electronic device is any one of the following devices: image data acquisition equipment, audio / video player, navigation equipment, communication equipment, roadside traffic facilities, industrial detection equipment, flight equipment, medical equipment, and security equipment.

Images