Image sensor, signal processing method and electronic equipment

Abstract

The embodiment of the invention discloses an image sensor, a signal processing method and electronic equipment, and the method comprises the steps: obtaining a first electric signal and a second electric signal outputted by a first pixel circuit, and enabling the second electric signal to be an electric signal outputted by the first pixel circuit after the exposure stage is finished; the first electric signal is an electric signal generated by resetting of the first pixel circuit in an exposure stage; performing analog-to-digital conversion on a difference value between the first electric signal and the second electric signal to obtain a digital signal; acquiring time required for a third electric signal output by the second pixel circuit to reach a preset threshold value as first time, wherein the third electric signal is an electric signal continuously output by the second pixel circuit in the exposure stage; and outputting at least one of the digital signal and the first time. The dynamic range of the image sensor can be effectively improved under the condition that the hardware cost is not greatly increased.

Description

Technical Field

[0001] The present disclosure relates to photoelectric detection technology, image sensing technology and signal processing technology, and in particular to an image sensor, a signal processing method and an electronic device. Background Art

[0002] Photoelectric image sensor is a new type of neuromorphic vision sensor. It imitates the imaging method of the retina in primates and records the continuous light intensity information in the scene by emitting high-density single-bit pulse sequences. It can achieve high-precision capture and recording of high-speed light processes and reconstruct the texture details in the scene. Therefore, it has great application value in machine vision and dynamic scene capture.

[0003] The dynamic range of an image sensor refers to the range of light intensities between the maximum and minimum light intensities that the image sensor can distinguish. Dynamic range is an important evaluation indicator of photoelectric image sensors.

[0004] In the process of realizing the present disclosure, it is found through research that in the related art, the photoelectric image sensor reflects the light intensity by recording the cumulative value of the photoelectric signal at a fixed time. The main ways to broaden the dynamic range of the photoelectric image sensor include: increasing the number of photodiodes in the photoelectric image sensor to achieve separate photosensitivity to strong and weak light; or, adding multiple signal conversion readout channels to achieve different gains for strong and weak light signals, etc. These methods require an increase in hardware cost, and the improvement effect of the dynamic range is directly related to the hardware cost. For example, if the dynamic range is to be increased tenfold, the hardware photosensitivity area needs to be increased tenfold, thereby increasing the hardware cost and the complexity of process preparation, resulting in a significant increase in cost. Summary of the invention

[0005] Embodiments of the present disclosure provide an image sensor and a signal processing method, an electronic device, a program product, and a medium, so as to effectively improve the dynamic range of an image sensor without significantly increasing the hardware cost.

[0006] In one aspect of an embodiment of the present disclosure, a signal processing method is provided, which is applied to an image sensor, wherein the image sensor includes a pixel unit, and the pixel unit includes a first pixel circuit and a second pixel circuit. The method includes:

[0007] Acquire a first electrical signal and a second electrical signal output by the first pixel circuit; wherein the second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on a light signal received during the exposure process after the first pixel circuit is reset; and the first electrical signal is an electrical signal generated by the first pixel circuit being reset during the exposure phase;

[0008] Performing analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal;

[0009] Acquire the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold value as the first time; wherein the third electrical signal is an electrical signal continuously output by the second pixel circuit in an exposure phase, and the third electrical signal is obtained based on photogenerated charges generated by the light signal received during the exposure process after the second pixel circuit is reset;

[0010] At least one of the digital signal and the first time is output.

[0011] Another aspect of the embodiments of the present disclosure provides an image sensor, comprising a pixel unit, a signal conversion unit and an interface circuit; wherein the pixel unit comprises a first pixel circuit and a second pixel circuit, and the signal conversion unit is connected to the first pixel circuit, the second pixel circuit and the interface circuit respectively;

[0012] The first pixel circuit is used to output a first electrical signal and a second electrical signal; wherein the second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on the light signal received during the exposure process after the first pixel circuit is reset; and the first electrical signal is an electrical signal generated by the first pixel circuit being reset during the exposure phase;

[0013] The signal conversion unit is used to perform analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal; and obtain the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold as the first time; wherein the third electrical signal is an electrical signal continuously output by the second pixel circuit during an exposure phase, and the third electrical signal is obtained by generating photogenerated charges based on the light signal received by the second pixel circuit during the exposure process after the second pixel circuit is reset;

[0014] The interface circuit is used to output at least one of the digital signal and the first time.

[0015] Another aspect of the embodiments of the present disclosure provides an electronic device, including: a processor, memories respectively connected to the processor for communication, and the image sensor according to any one of the embodiments of the present disclosure;

[0016] The memory is used to store computer-executable instructions;

[0017] The processor is used to execute the computer-executable instructions stored in the memory to control the image sensor to implement the signal processing method described in any embodiment of the present disclosure.

[0018] According to another aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the instructions are executed, the signal processing method described in any embodiment of the present disclosure is implemented.

[0019] According to another aspect of the embodiments of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the signal processing method described in any embodiment of the present disclosure is implemented.

[0020] Based on the embodiment of the present disclosure, the pixel unit in the image sensor includes a first pixel circuit and a second pixel circuit. The first pixel circuit is reset to generate and output a first electrical signal in an exposure phase, and outputs a second electrical signal generated by exposure after the exposure phase ends. The difference between the first electrical signal and the second electrical signal is converted into a digital signal. The second pixel circuit continuously outputs a third electrical signal generated by exposure during the exposure process, and obtains a first time required for the third electrical signal to reach a preset threshold value, and outputs at least one of the digital signal and the first time to reflect the exposure intensity of the pixel unit. At the same time, the first pixel circuit and the second pixel circuit are used to expose the same space in different exposure modes. Since the digital signal corresponding to the first pixel circuit can accurately reflect the exposure intensity of weaker light intensity, and the first time corresponding to the second pixel circuit can accurately reflect the exposure intensity of stronger light intensity, the digital signal and the first time are combined to reflect the exposure intensity of different light intensities, which not only realizes the accurate recording of weak light signals and strong light signals, but also can effectively expand the dynamic range of the image sensor without significantly increasing the hardware cost.

[0021] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0023] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0024] Figure 1 The figure is a flow chart of an embodiment of a signal processing method disclosed in the present invention.

[0025] Figure 2 FIG. 1 is a schematic diagram of the structure of an image sensor according to an embodiment of the present disclosure.

[0026] Figure 3 FIG. 4 is a schematic structural diagram of another embodiment of the image sensor disclosed in the present invention.

[0027] Figure 4FIG. 4 is a schematic structural diagram of another embodiment of the image sensor disclosed in the present invention.

[0028] Figure 5 FIG. 4 is a schematic structural diagram of another embodiment of the image sensor disclosed in the present invention.

[0029] Figure 6 FIG. 4 is a schematic structural diagram of another embodiment of the image sensor disclosed in the present invention.

[0030] Figure 7 FIG. 4 is a schematic structural diagram of another embodiment of the image sensor disclosed in the present invention.

[0031] Figure 8 The flowchart of another embodiment of the signal processing method disclosed in the present invention.

[0032] Fig. 9 FIG. 4 is a timing diagram of a row driving operation in an embodiment of the present disclosure.

[0033] Fig.10 The figure is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. DETAILED DESCRIPTION

[0034] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0035] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0036] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0037] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0038] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0039] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0040] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0042] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0043] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0044] Figure 1 1 is a flowchart of an embodiment of the signal processing method of the present disclosure. The signal processing method of the embodiment of the present disclosure is applied to an image sensor, the image sensor includes a pixel unit, the pixel unit includes a first pixel circuit and a second pixel circuit. The pixel unit in the embodiment of the present disclosure can be any pixel unit in the image sensor, that is, the operation of the embodiment of the present disclosure can be performed for any pixel unit in the image sensor. Figure 1 As shown, the signal processing method of this embodiment includes:

[0045] 102. Acquire a first electrical signal and a second electrical signal output by a first pixel circuit.

[0046] The second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on the light signal received during the exposure process after the first pixel circuit is reset. The first electrical signal is an electrical signal generated by the first pixel circuit being reset during the exposure phase.

[0047] 104 , performing analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal, converting the difference between the first electrical signal and the second electrical signal from an analog signal to a digital signal to obtain a digital signal.

[0048] Through digital signals, photosensitivity imaging can be performed on weaker light signals.

[0049] Thereafter, operation 108 is performed.

[0050] 106 , obtaining the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold as the first time.

[0051] The third electrical signal is an electrical signal continuously output by the second pixel circuit during the exposure phase, and the third electrical signal is obtained by generating photogenerated charges based on the light signal received during the exposure process after the second pixel circuit is reset.

[0052] By doing so for the first time, photosensitivity imaging can be performed on stronger light signals.

[0053] In the embodiment of the present disclosure, the first electrical signal, the second electrical signal, and the third electrical signal are electrical signals obtained by photogenerated charges generated by the light signal received by the pixel unit, and the electrical signal may be a voltage signal or a current signal, which is not limited in the embodiment of the present disclosure. In the embodiment of the present disclosure, the electrical signal is described as a voltage signal as an example, and the same is applicable to the case where the electrical signal is a current signal.

[0054] There is no restriction on the execution time and sequence between operations 106 and 102 - 104 , and the two operations can be executed simultaneously or in sequence at any time, and the embodiment of the present disclosure does not impose any restriction on this.

[0055] 108, output at least one of a digital signal and a first time.

[0056] Based on the embodiment of the present disclosure, the pixel unit in the image sensor includes a first pixel circuit and a second pixel circuit. The first pixel circuit is reset to generate a first electrical signal and output it in an exposure phase, and outputs a second electrical signal generated by exposure after the exposure phase ends. The difference between the first electrical signal and the second electrical signal is converted into a digital signal to obtain a digital signal. Based on the digital signal, imaging can be performed for a relatively weak light signal. The second pixel circuit continuously outputs a third electrical signal generated by exposure during the exposure process, and obtains a first time required for the third electrical signal to reach a preset threshold. Based on the first time, pulse sequence imaging can be performed, and at least one of the digital signal and the first time is output to reflect the exposure intensity of the pixel unit. At the same time, the first pixel circuit and the second pixel circuit are used to expose the same space in different exposure modes. Since the digital signal corresponding to the first pixel circuit can accurately reflect the exposure intensity of the relatively weak light intensity, and the first time corresponding to the second pixel circuit can accurately reflect the exposure intensity of the relatively strong light intensity, the digital signal and the first time are combined to reflect the exposure intensity of different light intensities, which not only realizes the accurate recording of weak light signals and strong light signals, but also can effectively expand the dynamic range of the image sensor without significantly increasing the hardware cost.

[0057] Figure 2 FIG. 1 is a schematic diagram of the structure of an embodiment of the image sensor disclosed in the present invention. The image sensor of the embodiment of the present invention can be used to implement the signal processing method of the embodiment of the present invention. Figure 2 As shown, the image sensor of this embodiment includes a pixel unit 202, a signal conversion unit 204 and an interface circuit 206. Each pixel unit 202 includes a first pixel circuit 2022 and a second pixel circuit 2024, that is, the first pixel circuit 2022 and the second pixel circuit 2024 together constitute a complete pixel unit.

[0058] The signal conversion unit 204 is connected to the first pixel circuit 2022, the second pixel circuit 2024 and the interface circuit 206 respectively.

[0059] The first pixel circuit 2022 is used to output a first electrical signal and a second electrical signal. The second electrical signal is an electrical signal output by the first pixel circuit 2022 after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on the light signal received during the exposure process after the first pixel circuit 2022 is reset. The first electrical signal is an electrical signal generated by the first pixel circuit 2022 when it is reset during the exposure phase.

[0060] The signal conversion unit 204 is used to perform analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal; and obtain the time required for the third electrical signal output by the second pixel circuit 2024 to reach a preset threshold as the first time. The third electrical signal is an electrical signal continuously output by the second pixel circuit 2024 during the exposure phase, and the third electrical signal is obtained based on the light signal received by the second pixel circuit 2024 during the exposure process after the second pixel circuit 2024 is reset to generate photogenerated charges.

[0061] The interface circuit 206 is configured to output at least one of a digital signal and a first time.

[0062] Based on the embodiment of the present disclosure, the pixel unit in the image sensor includes a first pixel circuit and a second pixel circuit. The first pixel circuit is reset to generate and output a first electrical signal in an exposure phase, and outputs a second electrical signal generated by exposure after the exposure phase ends. The difference between the first electrical signal and the second electrical signal is converted into a digital signal. The second pixel circuit continuously outputs a third electrical signal generated by exposure during the exposure process, and obtains a first time required for the third electrical signal to reach a preset threshold value, and outputs at least one of the digital signal and the first time to reflect the exposure intensity of the pixel unit. At the same time, the first pixel circuit and the second pixel circuit are used to expose the same space in different exposure modes. Since the digital signal corresponding to the first pixel circuit can accurately reflect the exposure intensity of weaker light intensity, and the first time corresponding to the second pixel circuit can accurately reflect the exposure intensity of stronger light intensity, the digital signal and the first time are combined to reflect the exposure intensity of different light intensities, which not only realizes the accurate recording of weak light signals and strong light signals, but also can effectively expand the dynamic range of the image sensor without significantly increasing the hardware cost.

[0063] In some of the implementations, the image sensor specifically includes a pixel array 20 formed by a plurality of rows (for convenience of expression, hereinafter represented as m rows, m is an integer greater than 1) and a plurality of columns (for convenience of expression, hereinafter represented as n columns, n is an integer greater than 1) of pixel units 202, and a plurality of signal conversion units 204, each signal conversion unit correspondingly connects to a first pixel circuit 2022 and a second pixel circuit 2024 in a pixel unit 202 in a dimensional direction in the pixel array, wherein the pixel unit in one dimensional direction may be a row of pixel units or a column of pixel units. When each signal conversion unit 204 corresponds to a row of pixel units in the pixel array 20, the number of signal conversion units 204 is consistent with the number of rows m of pixel units in the pixel array 20, thereby, each signal conversion unit 204 can realize column-by-column signal reading of the corresponding row of pixel units in sequence. When each signal conversion unit 204 is connected to a column of pixel units in the pixel array 20, the number of signal conversion units 204 is consistent with the number of columns n of pixel units in the pixel array. Therefore, each signal conversion unit 204 can realize row-by-row signal reading of the corresponding column of pixel units in sequence.

[0064] Figure 3 FIG. 1 is a schematic diagram of the structure of another embodiment of the image sensor disclosed in the present invention. Figure 3 As shown, an implementation example in which each signal conversion unit 204 is connected to a column of pixel units in the pixel array 20 is shown.

[0065] Optionally, see Figure 3In some implementations, the first pixel circuit 2022 may be implemented using a 4T pixel structure. For example, the first pixel circuit 2022 may include: a first photodiode 302, a transfer transistor 304, a capacitor 306, a first reset transistor 308, a first source follower transistor 310, and a first readout transistor 312. Figure 3 Shown in the solid line frame.

[0066] in:

[0067] One end of the first photodiode 302 is grounded, and the other end is connected to one end of the capacitor 306 and the gate end of the first source follower transistor 310 through the transfer transistor 304 .

[0068] The source terminal of the transfer transistor 304 is connected to the other end of the first photodiode 302, and the drain terminal is respectively connected to one end of the capacitor 306 and the gate terminal of the first source follower transistor 310, and the gate terminal is used to receive a first external control signal (expressed as Tx) for controlling the transfer transistor 304 to be turned on or off. The first external control signal Tx of the pixel units 202 in the same row in the pixel array 20 is uniformly controlled.

[0069] One end of the capacitor 306 is respectively connected to the source terminal of the first reset transistor 308 and the gate terminal of the first source follower transistor 310 , and is connected to the other end of the first photodiode 302 through the transfer transistor 304 , and the other end is grounded.

[0070] The source terminal of the first reset transistor 308 is connected to one end of the capacitor 306, the drain terminal is connected to the power signal VDD, and the gate terminal is used to receive a second external control signal (expressed as Rst) that controls the first reset transistor 308 to be turned on or off. The second external control signal Rst controls the pixel units 202 in the same row in the pixel array 20 uniformly.

[0071] The source terminal of the first source follower transistor 310 is connected to the drain terminal of the first readout transistor 312 , the drain terminal is connected to the power signal VDD, and the gate terminal is respectively connected to one end of the capacitor 306 and the drain terminal of the transfer transistor 304 .

[0072] The source terminal of the first readout transistor 312 serves as the output terminal of the first pixel circuit 2022, the drain terminal is connected to the source terminal of the first source follower transistor 310, and the gate terminal is used to receive a third external control signal (expressed as Read) for controlling the first readout transistor 312 to be turned on or off. The third external control signal Read of the pixel units 202 in the same row in the pixel array 20 is uniformly controlled.

[0073] In this embodiment, the first external control signal Tx, the second external control signal Rst and the third external control signal Read of all rows of pixel units 202 in the pixel array 20 are driven by the same driving circuit (eg Figure 6 , Figure 7 The first driving circuit in the pixel unit 202 generates and sends the first pixel circuit 2022 in each pixel unit 202.

[0074] Based on this embodiment, a pixel circuit implemented with a 4T pixel structure is provided, which can be used to detect light signals in a relatively weak light intensity environment and generate an electrical signal indicating the exposure intensity. In addition, the first pixel circuit in the embodiment of the present disclosure can also be implemented with other circuit structures, as long as the second pixel unit can be reset, exposed, generate an analog electrical signal indicating the exposure intensity and output it to the outside according to the control of the driving circuit, the embodiment of the present disclosure does not limit the specific structure of the first pixel circuit.

[0075] Optionally, see Figure 3 In some implementations, the second pixel circuit 2024 may be implemented using a 3T pixel structure. For example, the second pixel circuit 2024 includes: a second photodiode 402, a second reset transistor 404, a second source follower transistor 406, and a second readout transistor 408. Figure 3 As shown in the dashed box.

[0076] One end of the second photodiode 402 is grounded, and the other end is connected to the source end of the second reset transistor 404 and the gate end of the second source follower transistor 406 .

[0077] The source terminal of the second reset transistor 404 is connected to the other end of the second photodiode 402, the drain terminal is connected to the power signal VDD, and the gate terminal is used to receive a fourth external control signal (expressed as RSEL) for controlling the conduction or disconnection of the second reset transistor 404. The fourth external control signal RSEL of the pixel units 202 in the same row of the pixel array 20 is uniformly controlled.

[0078] The source terminal of the second source follower transistor 406 is connected to the drain terminal of the second readout transistor 408 , the drain terminal is connected to the power signal VDD, and the gate terminal is respectively connected to the other end of the second photodiode 402 and the source terminal of the second reset transistor 404 .

[0079] The source terminal of the second readout transistor 408 is used as the output terminal of the second pixel circuit 2024, the drain terminal is connected to the source terminal of the second source follower transistor 406, and the gate terminal is used to receive a fifth external control signal (denoted as VSEL) for controlling the conduction or disconnection of the second readout transistor 408. The fifth external control signal VSEL of the pixel units 202 in the same row in the pixel array 20 is uniformly controlled.

[0080] In this embodiment, the fourth external control signal RSEL and the fifth external control signal VSEL of all rows of pixel units 202 in the pixel array 20 are driven by the same driving circuit (for example, Figure 6 The first drive circuit or Figure 7 The second driving circuit in the pixel unit 202 generates and sends the second pixel circuit 2024 in each pixel unit 202.

[0081] Based on this embodiment, a pixel circuit implemented with a 3T pixel structure is provided, which can be used to detect light signals in a strong light environment, generate an electrical signal indicating exposure intensity, and perform pulse sequence imaging of strong light. In addition, the second pixel circuit in the embodiment of the present disclosure can also be implemented with other circuit structures, as long as the second pixel circuit can be reset, exposed, generate an electrical signal indicating exposure intensity and output it to the outside according to the control of the driving circuit, the embodiment of the present disclosure does not limit the specific structure of the second pixel circuit.

[0082] Figure 4 FIG. 1 is a schematic diagram of the structure of another embodiment of the image sensor disclosed in the present invention. Figure 4 As shown, based on any of the above embodiments, in some implementations, the signal conversion unit 204 may include: an analog to digital converter (ADC) 2042, a comparator 2044 and a timer (COMP) 2046. Among them:

[0083] The input end of the analog-to-digital conversion circuit 2042 is connected to the output end of the corresponding first pixel circuit 2022, and is used to receive the first electrical signal (V1) and the second electrical signal (V2) output by the corresponding first pixel circuit 2022, and perform analog-to-digital conversion on the difference (V1-V2) between the first electrical signal and the second electrical signal to obtain a digital signal; the output end of the analog-to-digital conversion circuit 2042 is connected to the input end of the interface circuit 206, and is used to output a digital signal to the interface circuit 206.

[0084] Optionally, in some of the implementations, the analog-to-digital conversion circuit 2042 may include a correlated double sampling circuit and an analog-to-digital converter, and obtain the difference (V1-V2) between the first electrical signal and the second electrical signal through the correlated double sampling circuit, and convert the difference between the first electrical signal and the second electrical signal from an analog signal to a digital signal through the analog-to-digital converter, wherein the correlated double sampling circuit and the analog-to-digital converter may be implemented using any correlated double sampling circuit and analog-to-digital converter in the relevant technology, and the embodiments of the present disclosure are not limited to this.

[0085] One input terminal of the comparator 2044 is connected to the output terminal of the corresponding second pixel circuit 2024, and is used to receive the third electrical signal (V3) output by the corresponding second pixel circuit 2024; the other input terminal of the comparator 2044 is used to receive a signal (Vref) of a preset threshold, and the signal Vref of the preset threshold can be generated by a threshold signal generating circuit; the output terminal of the comparator 2044 is connected to the input terminal of the timer 2046, and is used to output a comparison result signal (1 or 0) to the timer 2046, wherein one input terminal is one of the positive input terminal and the negative input terminal, and the other input terminal is the other of the positive input terminal and the negative input terminal. Taking one input terminal as a positive input terminal and the other input terminal as a negative input terminal as an example, the comparator 2044 can output a high level (1) as a comparison result signal when its positive input signal is greater than the negative input signal, that is, when the third electrical signal output by the second pixel circuit is greater than the preset threshold signal Vref, otherwise it outputs a low level (0) as the comparison result signal. Therefore, according to the comparison result signal output by the comparator, it can be determined whether the third electrical signal output by the second pixel circuit is greater than the preset threshold signal Vref. When the comparison result signal of the comparator 2044 changes from a low level (0) to a high level (1), it indicates that the third electrical signal output by the second pixel circuit has reached the preset threshold signal Vref.

[0086] The input end of the timer 2046 is connected to the output end of the comparator 2044, and is used to receive the comparison result signal, so as to time the holding time of the comparison result signal according to the preset timing frequency; the output end of the timer 2046 is connected to the input end of the interface circuit 206, and is used to output the timing result when the comparison result signal changes (i.e., flips), as the first time. For example, taking one input end of the comparator as the positive input end and the other input end as the negative input end as an example, the timer 2046 times the time when the comparison result signal is 0, and when the comparison result signal changes (i.e., changes from 0 to 1), the time when the comparison result signal is 0 is obtained, which is the time taken by the third electrical signal output by the second pixel circuit to accumulate from 0 to the signal Vref of the preset threshold value, which can represent the strength of the light signal received by the second pixel circuit. The shorter the timing result, the stronger the light signal received by the second pixel circuit. Therefore, based on the timing result of the timer for the third electrical signal output by the same second pixel circuit, pulse sequence imaging based on light signal detection can be realized.

[0087] In this embodiment, an example in which one input terminal of the comparator is a positive input terminal and the other input terminal is a negative input terminal is used for explanation. If one input terminal of the comparator is a negative input terminal and the other input terminal is a positive input terminal, then the opposite is true. When the comparison result signal of the comparator 2044 changes from a high level (1) to a low level (0), it indicates that the third electrical signal output by the second pixel circuit has reached a preset threshold signal Vref, and the timer counts the time when the comparison result signal is 1.

[0088] In the present embodiment, in the same pixel unit 202, the first electrical signal and the second electrical signal output by the first pixel circuit 2022 are input to the column-level analog-to-digital conversion circuit 2042 for processing and outputting digital signals, and the third electrical signal output by the second pixel circuit 2024 is input to an input terminal of the column-level comparator 2044 and compared with a signal Vref of a preset threshold value input to another input terminal of the comparator 2044. The comparison result obtained controls the timer 2046 to stop timing and obtains the timing result (i.e., the first time).

[0089] Figure 5 FIG. 1 is a schematic diagram of the structure of another embodiment of the image sensor disclosed in the present invention. Figure 5 As shown, in Figure 4 On the basis of the illustrated embodiment, in this embodiment, the signal conversion unit 204 may further include a multiplexer (MUX) 2048. In some implementations, the first input end of the multiplexer 2048 is connected to the output end of the analog-to-digital conversion circuit 2042, and is used to receive the digital signal output by the analog-to-digital conversion circuit 2042; the second input end of the multiplexer 2048 is connected to the output end of the timer 2046, and is used to receive the timing result output by the timer 2046; the output end of the multiplexer 2048 is connected to the input end of the interface circuit 206 as the output end of the signal conversion unit 204, and is used to output the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time, or the first time as the current output data, and the data identification information of the output data to the interface circuit 206, and the data identification information is used to identify the output data as a digital signal or the first time, so as to receive the meaning of the data of this time determined based on the data identification information, thereby adopting a corresponding method to determine the light signal intensity (exposure intensity) of the pixel unit detection space. Among them, the preset upper limit corresponding to the digital signal is the preset code value upper limit, which is the code value corresponding to the maximum light intensity that the first pixel circuit 2022 can distinguish; the preset upper limit corresponding to the first time is the first preset time upper limit, which is the time length corresponding to the aforementioned maximum light intensity.

[0090] Participate again Figure 5In some other implementations, the first input end of the multiplexer 2048 is connected to the output end of the analog-to-digital conversion circuit 2042, and is used to receive the digital signal output by the analog-to-digital conversion circuit 2042; the second input end of the multiplexer 2048 is connected to the output end of the timer 2046, and is used to receive the timing result output by the timer 2046; the output end of the multiplexer 2048, as the output end of the signal conversion unit 204, is connected to the input end of the interface circuit 206, and is used to respond to the analog-to-digital conversion circuit 2042 to obtain a digital signal, output the digital signal and first data identification information, and the first data identification information is used to indicate that the output data this time is a digital signal; respond to the timer 2046 to obtain a first time, output the first time and second data identification information, and the second data identification information is used to indicate that the output data this time is the first time.

[0091] In this embodiment, the digital signal output by the first pixel circuit 2022 in the same pixel unit 202 and the timing result output by the timer 2046 are both input to the multiplexer 2048, and then uniformly output by the multiplexer 2048 to the interface circuit 206. The interface circuit 206 transmits the output data of the image sensor to the outside, for example, it can be transmitted to a local terminal connected via wired communication, or to a server remotely connected via wireless communication, etc., so that the terminal or server can reconstruct the image based on the output data of the image sensor, calculate the light intensity of the image sensor detection space, etc. The embodiment of the present disclosure does not limit the transmission method, receiving object and subsequent application of the output data of the image sensor.

[0092] Participate again Figure 5 In some other implementations, the interface circuit 206 may be provided with parallel input ports that are consistent with the number of the signal conversion units 204, as multiple input terminals, and the multiple input terminals are connected one-to-one with the output terminals of the multiple signal conversion units 204, and are used to output the output data and data identification information of the multiple signal conversion units in sequence according to the order of the multiple signal conversion units 204, so as to obtain the output data sequence of the multiple signal conversion units 204 in the same data output cycle (corresponding to the above-mentioned signal readout stage or the signal output stage of the second pixel circuit), thereby realizing the parallel-to-serial conversion of the output data and data identification information of the multiple signal conversion units 204 for external transmission.

[0093] Figure 6 FIG. 1 is a schematic diagram of the structure of another embodiment of the image sensor disclosed in the present invention. Figure 6 As shown in the above Figure 2-Figure 5On the basis of any of the embodiments shown, the image sensor of this embodiment may further include a first driving circuit 208 , wherein the first driving circuit 208 is respectively connected to the first pixel circuits 2022 and the second pixel circuits 2024 in all the pixel units 202 in the pixel array 20 .

[0094] Then, in a signal processing method based on the image sensor and its implementation, Figure 3 Corresponding to the implementation example in which each signal conversion unit 204 is connected to a column of pixel units in the pixel array 20, the first driving circuit 208 can respectively control a row of pixel units 202 in the pixel array 20 to reset the first pixel circuit 2022 and the second pixel circuit 2024 in the row of pixel units 202 according to a preset period, control the first pixel circuit 2022 in the row of pixel units 202 to enter the exposure phase for exposure according to a preset period with a first preset delay, control the second pixel circuit 2024 in the row of pixel units 202 to enter the exposure phase and the signal output phase for simultaneous exposure and third signal reading, and control the first pixel circuit 2022 in the row of pixel units 202 to enter the signal readout phase for sequentially reading the first signal and the second signal according to a preset period with a second preset delay; and,

[0095] For the pixel array 2024: the first pixel circuit 202 in each row of pixel units 202 is controlled row by row for exposure in a rolling exposure mode with a third preset delay, and the second pixel circuit 2024 in each row of pixel units 202 is controlled row by row for exposure and third signal reading in a row by row exposure mode with a third preset delay.

[0096] Alternatively, in another signal processing method based on the image sensor and its implementation, corresponding to an implementation example (not shown in the present disclosure) in which each signal conversion unit 204 is connected to a row of pixel units in the pixel array 20, the first driving circuit 208 can respectively control a column of pixel units 202 in the pixel array 2020 to reset the first pixel circuit 2022 and the second pixel circuit 2024 in the column of pixel units 202 according to a preset period, control the first pixel circuit 2022 in the column of pixel units 202 to enter an exposure phase for exposure according to a preset period with a first preset delay, control the second pixel circuit 2024 in the column of pixel units 202 to enter an exposure phase and a signal output phase for simultaneous exposure and third signal reading, and control the first pixel circuit 2022 in the column of pixel units 202 to enter a signal readout phase for sequentially reading the first signal and the second signal according to a preset period with a second preset delay; and,

[0097] For the pixel array 20: a rolling exposure mode with a third preset delay is adopted to control the first pixel circuit 2022 in each column of the pixel unit 202 for exposure column by column, and a column-by-column exposure mode with a third preset delay is adopted to control the second pixel circuit 2024 in each column of the pixel unit 202 for exposure and third signal reading.

[0098] Figure 7 FIG. 1 is a schematic diagram of the structure of another embodiment of the image sensor disclosed in the present invention. Figure 7 As shown in the above Figure 2-Figure 5 Based on any of the embodiments shown, the image sensor of this embodiment may further include a first driving circuit 208 and a second driving circuit 210. In this embodiment, the first driving circuit 208 is respectively connected to the first pixel circuits 2022 in all the pixel units 202 in the pixel array 20, and the second driving circuit 210 is respectively connected to the second pixel circuits 2024 in all the pixel units 202 in the pixel array 20.

[0099] Then, in a signal processing method based on the image sensor and its implementation, Figure 3 Corresponding to the implementation example in which each signal conversion unit 204 is connected to a column of pixel units in the pixel array 20, the first driving circuit 210 can respectively control a row of pixel units 202 in the pixel array 20 to reset according to a preset period, control a row of pixel units 202 to enter an exposure phase for exposure according to a preset period with a first preset delay, and control a row of pixel units 202 to enter a signal readout phase for sequentially reading out the first signal and the second signal according to a preset period with a second preset delay; and, for the pixel array 20: adopt a rolling exposure mode with a third preset delay to control the first pixel circuits 2022 in each row of pixel units 202 row by row for exposure;

[0100] The second driving circuit 210 can, for each row of pixel units 202 in the pixel array 20: control the second pixel circuit 2024 to reset according to a preset period, and control the second pixel circuit 2024 in a row of pixel units 202 to enter an exposure phase and a signal output phase according to a preset period with a first preset delay to simultaneously perform exposure and third signal reading; and, for the pixel array 20: adopt a row-by-row exposure mode with a third preset delay to control the second pixel circuit 2024 in each row of pixel units 202 to perform exposure and third signal reading.

[0101] Alternatively, in another signal processing method based on the image sensor and its implementation, corresponding to an implementation example (not shown in the present disclosure) in which each signal conversion unit 204 is connected to a row of pixel units in the pixel array 20, the first driving circuit 208 can respectively control the first pixel circuit 2022 in the column of pixel units 202 to reset according to a preset period, control the first pixel circuit 202 in the column of pixel units 202 to enter the exposure stage for exposure according to a preset period with a first preset delay, and control the first pixel circuit 202 in the column of pixel units 202 to enter the signal readout stage for sequentially reading out the first signal and the second signal according to a preset period with a second preset delay; and, for the pixel array 20: adopt a rolling exposure mode with a third preset delay to control the first pixel circuit 2022 in each column of pixel units 202 for exposure column by column;

[0102] The second driving circuit 210 can control the second pixel circuit 2024 to reset according to a preset period for a column of pixel units 202 in the pixel array 20, and control the second pixel circuit 2024 in a column of pixel units 202 to enter the exposure stage and the signal output stage according to the preset period with a first preset delay to simultaneously perform exposure and third signal reading; and, for the pixel array 20: adopt a mode of exposing column by column with a third preset delay to control the second pixel circuit 2024 in each column of pixel units 202 to perform exposure and third signal reading.

[0103] The signal processing method of the embodiment of the present disclosure is further described below in combination with the image sensor of the above-mentioned embodiment. Based on the description of the embodiment of the present disclosure, those skilled in the art can know that it can also be implemented by adopting other structures that are structurally equivalent, similar or deformed to the image sensor of the embodiment of the present disclosure.

[0104] Optionally, in some of the implementations, the photoelectric conversion gain of the first pixel circuit 2022 is consistent with that of the second pixel circuit 2024, that is, the photoelectric conversion gain of the first photodiode 302 is consistent with that of the second photodiode 402, then the above-mentioned preset threshold can be the signal size corresponding to the maximum light intensity that the first pixel circuit 2022 can distinguish.

[0105] Alternatively, in some implementations, the photoelectric conversion gains of the first pixel circuit 2022 and the second pixel circuit 2024 may be inconsistent, that is, the photoelectric conversion gains of the first photodiode 302 and the second photodiode 402 are inconsistent, and the above-mentioned preset threshold is a signal size determined based on the signal size corresponding to the maximum light intensity that can be distinguished by the first pixel circuit 2022, and the ratio between the photoelectric conversion gain of the second pixel circuit 2024 and the photoelectric conversion gain of the first pixel circuit 2022. For example, assuming that the signal size corresponding to the maximum light intensity that can be distinguished by the first pixel circuit 2022 is Vm, the photoelectric conversion gain of the first pixel circuit 2022 is G1, and the photoelectric conversion gain of the second pixel circuit 2024 is G2, then the above-mentioned preset threshold can be determined based on Vm*(G2 / G1), and the embodiments of the present disclosure are not limited to this.

[0106] Based on this example, the photoelectric conversion gains of the first pixel circuit 2022 and the second pixel circuit 2024 may also be inconsistent. In this case, the coverage of different light intensity intervals can be achieved by adjusting the flip threshold of the comparator.

[0107] Figure 8 FIG. 1 is a flow chart of another embodiment of the signal processing method disclosed in the present invention. Figure 8 As shown, based on the above embodiments, in some implementations, operation 102 may include:

[0108] 1022, control the first pixel circuit to enter an exposure phase after being reset for exposure, in which the first pixel circuit receives a light signal during the exposure process to generate and accumulate photogenerated charges, wherein the exposure duration of the first pixel circuit during the exposure phase is a first duration.

[0109] 1024, before the exposure phase of the first pixel circuit ends, control the first pixel circuit to reset and enter the signal readout phase, in the first phase of the signal readout phase, the first pixel circuit outputs a reset signal as the first electrical signal. The duration of the signal readout phase is the second duration.

[0110] In the embodiment of the present disclosure, the signal readout phase can be divided into two phases: a first phase and a second phase located after the first phase.

[0111] 1026. After the exposure phase of the first pixel circuit ends, the first pixel circuit is controlled to enter the second phase of the signal readout phase. In the second phase, the first pixel circuit outputs an exposure signal generated by the accumulated photogenerated charges as a second electrical signal.

[0112] Optionally, in some of the implementations, in the above operation 104, a correlated double sampling technique may be used to obtain the difference between the first electrical signal and the second electrical signal, and then an analog-to-digital conversion technique may be used to convert the difference between the first electrical signal and the second electrical signal from an analog signal to a digital signal. The embodiment of the present disclosure does not limit the manner in which the analog-to-digital conversion of the difference between the first electrical signal and the second electrical signal in the above operation 104 is performed.

[0113] Optionally, in Figure 8 Based on the illustrated embodiment, in some implementations, the operation 106 may include:

[0114] After the second pixel circuit is reset, it enters the exposure phase and the signal output phase. In the exposure phase, the second pixel circuit receives the light signal during the exposure process to generate photogenerated charges, and outputs the exposure signal generated by the photogenerated charges as the third electrical signal. The exposure phase and the signal output phase of the second pixel circuit are consistent in time period, and the duration of the exposure phase and the signal output phase of the second pixel circuit are respectively the third duration;

[0115] The time required for the third electrical signal output by the second pixel circuit in the signal output phase to reach a preset threshold is obtained.

[0116] Fig. 9 FIG. 1 is a timing diagram of a row drive operation in an embodiment of the present disclosure. Fig. 9 ,by Figure 6 or Figure 7 Taking the structure of the image sensor shown in FIG. 2 as an example, for the first pixel circuit 2022, a rolling exposure mode is adopted:

[0117] First, the reset signal (i.e., the second external control signal) RST[1] controlling the gate terminal of the first reset transistor 308 in the first row of pixel units 202 is set to a high level, the first reset transistor 308 is turned on, and the capacitor 306 in the first pixel circuit 2022 is reset; then, RST[1] is set to a low level, the first external control signal TX[1] controlling the gate terminal of the transfer transistor 304 is set to a high level, the transfer transistor 304 is turned on, and the charge on the first photodiode 302 is transferred to the capacitor 306, clearing the charge in the first photodiode 302. [1] represents the first row;

[0118] Afterwards, TX[1] is set to a low level, the transfer transistor 304 is turned off, and the first pixel circuit 2022 in the first row of pixel units 202 enters the exposure phase for exposure. Fig. 9 The first row has a 4T pixel exposure time), and the incident light signal received by the first pixel circuit 2022 is converted into photogenerated charges by the first photodiode 302 and accumulated in the first photodiode 302;

[0119] When the exposure of the first pixel circuit 2022 in the first row of pixel units 202 is about to end, RST[1] is set to a high level, the first reset transistor 308 is turned on, and the capacitor 306 in the first pixel circuit 2022 is reset; after the capacitor 306 is reset, the read signal (i.e., the third external control signal) Read[1] that controls the gate terminal of the first readout transistor 312 is set to a high level, and the reset signal of the first pixel circuit 2022 at this time is read out as a first electrical signal to the analog-to-digital conversion circuit 2042 connected at the column level, and Read[1] is kept at a high level, TX[1] is set to a high level, the transfer transistor 304 is turned on, and the photogenerated charge accumulated by the first photodiode 302 during this exposure process is transferred to the capacitor 306, and the exposure signal generated in this exposure stage is read out as a second electrical signal to the analog-to-digital conversion circuit 2042 connected at the column level through the first readout transistor 312;

[0120] The analog-to-digital conversion circuit 2042 performs analog-to-digital conversion on the difference (V1-V2) between the first electrical signal and the second electrical signal to obtain a digital signal, thereby converting the exposure information into a digital signal that is easy to transmit and process.

[0121] Since the signal conversion unit 204 can only read out the signals output by one row of pixel units in a column of connected pixel units at a time, the RST, TX and Read signals of the second row of pixel units are delayed by a readout time (i.e., the third preset delay) than those of the first row of pixel units, and so on. When the signals output by the last row of pixel units are read out, the signals are read back to the first row of pixel units, and the cycle continues.

[0122] For the second pixel circuit 2024, a row-by-row exposure mode is adopted:

[0123] For 3T pixels, the line-by-line exposure mode is adopted:

[0124] First, the reset signal (i.e., the fourth external control signal) RSel[1] controlling the gate terminal of the second reset transistor 404 in the first row of pixel units 202 is set to a high level, the second reset transistor 404 is turned on, and the second photodiode 402 in the second pixel circuit 2024 is reset. [1] represents the first row;

[0125] After the second photodiode 402 is reset, RSel[1] is set to a low level, and the output signal (i.e., the fifth external control signal) Vsel[1] that controls the gate terminal of the second readout transistor 408 is set to a high level, and the second pixel circuit 2024 in the first row of pixel units 202 enters the exposure phase for exposure, and the incident light signal received by the second pixel circuit 2024 is converted into photogenerated charge by the second photodiode 402. Fig. 9 During the exposure period of 3T pixels in the first row), Vsel[1] is kept at a high level, and Vsel of the second pixel circuit 2024 in the other rows of pixel units 202 is kept at a low level. The second readout transistor 408 in the first row of pixel units 202 is kept turned on. Therefore, the exposure signal (i.e., the third electrical signal) of the second pixel circuit 2024 in the first row of pixel units 202 is kept connected to the column-level connected comparator 2044. The column-level connected timer 2046 starts from zero at the rising edge of Vsel[1] (i.e., the start time when it is set to a high level) according to the preset timing frequency. During the exposure period of the second pixel circuit 2024 in the first row of pixel units 202, the photogenerated charge gradually accumulates. When the photogenerated charge is When the charge accumulates to the preset threshold, it triggers the comparison result signal output by the comparator 2044 to flip, for example, flip from 0 to 1 (the third electrical signal is connected to the positive input terminal of the comparator, and the electrical signal of the preset threshold is connected to the negative input terminal of the comparator) or flip from 1 to 0 (the third electrical signal is connected to the negative input terminal of the comparator, and the electrical signal of the preset threshold is connected to the positive input terminal of the comparator). After the comparison result signal output by the comparator 2044 flips, the corresponding timer 2046 is controlled to stop timing. The timing result at this time (i.e., the first time) can indicate the strength of the light signal received by the second pixel circuit 2024 in the first row of pixel units 202. The shorter the timing result, the stronger the light signal received by the second pixel circuit 2024.

[0126] During the exposure process of the second pixel circuits 2024 in the current row, the timer 2046 corresponding to each second pixel circuit 2024 stops timing at different times, that is, the timing results (ie, the first time) are different, while the duration for which Vsel maintains a high level is fixed. For example, the duration of each Vsel maintaining a high level can be set to 8 microseconds (us), and the timing frequency of the timer 2046 is 125 MHz, that is, 1 is added every 8ns timing unit. Then, within the exposure time of 8us, the second pixel circuit 2024 in the current row may reach the preset threshold received by the comparator 2044 after 80ns of exposure, causing the comparison result signal to flip, while the remaining second pixel circuits 2024 in the current row have not yet reached the preset threshold received by the comparator 2044, then the timer 2046 corresponding to the second pixel circuit 2024 that has reached the preset threshold stops timing, and the remaining second pixel circuits 2024 continue to be exposed, and the corresponding timer 2046 also continues timing until the exposure time of 8us ends, and all timing results of the second pixel circuits 2024 in the entire row are uniformly output, and then the exposure of the second pixel circuits 2024 in the next row is performed, and the timer 2046 also starts timing again.

[0127] When the second pixel circuit 2024 is exposed and outputs signals, the signal conversion unit 204 can only read one row of the second pixel circuit 2024 at a time. Therefore, the RSel and VSel signals of the second pixel circuit 2024 in the second row are delayed by the length of an exposure time (i.e., the third preset delay) compared with the first row, and so on. When the signal output of the second pixel circuit 2024 in the last row is completed, the signal is read back to the first row of the second pixel circuit 2024, and the cycle is repeated.

[0128] In the disclosed embodiment, the frame period of the image sensor is defined as the length of time from the start of reading the first row of pixel unit signals to the completion of reading the last row of pixel unit signals. It can be seen that the time from the start of reading the first row of signals to the completion of reading the last row of signals of the first pixel unit 2022 is aligned with the time from the start of reading the first row of signals to the completion of reading the last row of signals of the second pixel circuit 2024. In this case, the frame period of the first pixel unit 2022 or the second pixel circuit 2024 is the same as the overall frame period of the entire image sensor.

[0129] For traditional image sensors based on 4T pixel structures, their dynamic range depends largely on the resolution accuracy of the ADC. Taking a common 12-bit ADC as an example, it can output 4096 code values ​​from 0 to 4095. If the maximum voltage is 2 volts (V), the minimum voltage that can be resolved is about 0.5 millivolts (mV). The ratio between the maximum signal and the minimum signal is about 4000 times, which is about 72 decibels (dB) in terms of dynamic range, which is the normal value of traditional image sensors.

[0130] In the disclosed embodiment, a 12-bit ADC is used to perform analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal output by the first pixel circuit of the 4T pixel structure, which can cover a dynamic range of 72 dB, while the third electrical signal output by the second pixel circuit of the 3T pixel structure can be used to represent a higher dynamic range than the aforementioned 72 dB. The following is an example in which the photoelectric conversion gain of the first pixel circuit and the second pixel circuit is the same:

[0131] Assume that the pixel array in the image sensor includes 1000 rows of pixel units, the signal reading time of each row of pixel units (i.e., the duration of the signal readout phase) is set to 8us, the timing frequency of the timer is 125MHz, the minimum timing unit is 8ns, and the maximum timing result is 1000, which corresponds to 8us. Then, the exposure time of the first pixel circuit of 1000 rows (i.e., the duration of the exposure phase) is about 8ms, and the exposure time of the second pixel circuit of each row is 8us. The frame period is about 8ms, and the corresponding frame rate is about 120 frames per second. If the voltage signal generated by the first pixel circuit after exposure for 8ms is 0.5mV under the light intensity of 0.1 lux, which is the minimum light intensity that the image sensor can distinguish, then under the light intensity of 400lux, the voltage signal generated by the first pixel circuit after exposure for 8ms is about 2V, reaching the maximum voltage. Under the light intensity of 400lux, the voltage signal generated by the second pixel circuit after exposure for 8us is about 2mV. If the signal used as the preset threshold for the comparator comparison (referred to as the flip voltage) is set to 2mV, then at a light intensity of 400lux, the timing result of the second pixel circuit is exactly the maximum value of 1000, and the minimum value of the timing result of the second pixel circuit corresponds to 8ns exposure to reach a 2mV signal voltage, which corresponds to 400,000 lux. Therefore, in this image sensor, the first pixel circuit can cover light intensities from 0.1lux to 400lux, and the second pixel circuit can cover light intensities from 400lux to 400,000lux. The overall coverage is from 0.1lux to 400,000lux, which is 1000 times higher than the traditional image sensor based on 4T pixel structure, that is, 132dB of dynamic range.

[0132] Among them, in the above examples, the frame period of 8ms, the exposure time of 8us, the timing frequency of 125Mhz, the resolution accuracy of 12bit, the number of rows of pixel units of 1000, etc., are only for illustrating the effects of the embodiments of the present disclosure. The specific values ​​of the above parameters can be adjusted in actual applications, and can achieve the effect of expanding the dynamic range. They do not constitute limitations on the implementation scheme of the present disclosure or the achievable effects.

[0133] based on Fig. 9It can be seen from the row driving operation timing diagram shown that the third external control signal Read of the first pixel circuit 2022 and the fifth external control signal Vsel of the second pixel circuit 2024 have the same waveform, and the second external control signal Rst of the first pixel circuit 2022 and the fourth external control signal RSel of the second pixel circuit 2024 have the same waveform. Therefore, the third external control signal Read and the fifth external control signal Vsel of each row of pixel units can be controlled by the same control signal. Similarly, the second external control signal Rst and the fourth external control signal RSel of each row of pixel units can be controlled by the same control signal. Therefore, the first pixel circuit and the second pixel circuit can be controlled by the same driving circuit. Only one driving circuit can be set in the image sensor, such as Figure 6 As shown, relative to Figure 7 The second driving circuit on the right side can be omitted, thereby simplifying the image sensor structure.

[0134] Optionally, in some of the implementations, the exposure phase and signal readout phase of the first pixel circuit 2022 correspond to the exposure phase and signal output phase of the second pixel circuit 2024, that is, the exposure phase of the first pixel circuit 2022 is consistent with the exposure phase of the second pixel circuit 2024 in time and duration, and the signal readout phase of the first pixel circuit 2022 is consistent with the signal output phase of the second pixel circuit 2024 in time and duration. At this time, in operation 108 of the above embodiment, the multiplexer 2048 can output the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time or the first time as the output data of this time, and the data identification information of the output data, and the data identification information is used to identify the output data of this time as a digital signal or a first time. Among them, the preset upper limit corresponding to the digital signal is the preset code value upper limit, and the preset code value upper limit is the maximum light intensity corresponding code value that the first pixel circuit 2022 can distinguish, and in actual applications, it can be set to the maximum code value supported by the ADC. The preset upper limit corresponding to the first time is the first preset time upper limit, and the first preset time upper limit is the duration corresponding to the aforementioned maximum light intensity.

[0135] Through the above analysis, it can be known that for a pixel unit, the exposure phase and signal readout phase of the first pixel circuit 2022 are consistent with the exposure phase and signal output phase of the second pixel circuit 2024 in terms of time and duration, and only one of the digital signal output by the ADC and the timing result output by the timer is actually accurate. For each pixel unit, the digital signal output by the ADC and the timing result output by the timer are simultaneously input into a multiplexer, and the multiplexer determines which result is accurate. The multiplexer can be set to output only the accurate one of the two results as the output data of this time, and a data flag as data identification information, and the data flag is used to identify whether the data of this time is the digital signal obtained by the ADC or the timing result. In an implementation example, the selection criterion of the multiplexer can be set to output the result that does not reach the corresponding preset upper limit of the two results, because for weaker light intensity, the digital signal of the ADC is accurate and will not reach the maximum code value of the ADC; and because the weaker light intensity cannot generate a 2mV signal within 8us, the comparison result signal of the comparator will not flip, and the timer will continue to count until the maximum timing result, so the timing result must reach the maximum value. Similarly, for stronger light intensities, the timing result is accurate because the ADC digital signal will reach the maximum code value, while the timing result will not reach the maximum value. Through the selection of the multiplexer, each pixel unit can output up to 13 bits of data to express a light intensity of 132dB, while achieving a high dynamic range and ensuring the reduction of data volume.

[0136] In the embodiment of the present disclosure, for the same pixel unit, after the data and data identification information are output based on the embodiment of the present disclosure, at the receiving end, based on the data identification information, it can be determined that the output data is a digital signal or a timing result. For a digital signal, the larger the digital signal is, the greater the light intensity of the pixel unit detection space in the corresponding time is. The analog voltage value represented by the digital signal can be calculated by the code value represented by the digital signal and the reference voltage of the ADC (i.e., the maximum voltage that can be distinguished as described above), and then based on the preset corresponding relationship coefficient k between the light intensity value and the voltage value, k is a constant greater than zero, the light intensity value in the corresponding time can be estimated. For example, assuming that the reference voltage of the 12-bit ADC is 2V, and its output code value is 2048, then according to (2048 / 4095)×2V≈1V, the corresponding analog voltage value is about 1V, and then based on i=k*analog voltage value, the light intensity value i in the time can be known. For the timing result (expressed as T), the shorter the timing result, the greater the light intensity of the pixel unit detection space in the corresponding time. The light intensity value in the corresponding time can be estimated based on the timing result T in a preset manner. For example, the light intensity value i in the time can be obtained in the following manner: i=a*1 / T. Among them, a is a constant greater than zero, and the specific value of a can be preset according to actual needs. Therefore, based on the output data and data identification information of the pixel unit each time, the light intensity value of its detection space in the corresponding time can be determined. Based on the output data and data identification information output by the pixel unit multiple times and arranged based on the time series relationship, the light intensity value and its change in the detection space in time series can be correspondingly determined.

[0137] For an image sensor including a pixel array formed by m rows and n columns of pixel units, each pixel unit in the pixel array works independently of each other, and respectively realizes time domain acquisition of a light signal in a space in the observation scene to obtain output data (digital signal or timing result) and data identification information for output. The spaces corresponding to different pixel units do not overlap, and the pixel units in the pixel array cooperate with each other to cover the entire observation scene, thereby realizing time domain acquisition of the light signal in the entire observation scene, outputting data (digital signal or timing result) and data identification information, and obtaining light intensity values. As a result, the light intensity value and its change of the light signal corresponding to the entire observation scene can be obtained.

[0138] Optionally, in some other implementations, the exposure phase and signal readout phase of the first pixel circuit 2022 are inconsistent with at least one of the exposure phase and signal output phase of the second pixel circuit 2024, that is, the exposure phase of the first pixel circuit 2022 is inconsistent with the exposure phase of the second pixel circuit 2024 in at least one of time and duration, and / or the signal readout phase of the first pixel circuit 2022 is consistent with the signal output phase of the second pixel circuit 2024 in at least one of time and duration. At this time, in operation 108 of the above embodiment, in response to obtaining the digital signal, the digital signal and the first data identification information can be directly output, and the first data identification information is used to indicate that the output data this time is the digital signal; in response to obtaining the first time, the first time and the second data identification information can be directly output, and the second data identification information is used to indicate that the output data this time is the first time. In this implementation, the operation 108 may be performed by the multiplexer 2048, or the multiplexer 2048 may not be set, and the analog-to-digital conversion circuit 2042 and the timer 2046 may directly output to the interface circuit 206, and the interface circuit 206 may output it, and the embodiment of the present disclosure does not limit this.

[0139] In a specific implementation, the exposure time, frame period, etc. of the first pixel circuit 2022 and the second pixel circuit 2024 may also be inconsistent. For example, the exposure time and frame period of the first pixel circuit may be extended to improve the photosensitivity of the image sensor to weaker light intensity. At this time, since the output periods of the digital signal and the timing result are different, in the embodiment of the present disclosure, both signals are output and their data identification information is output simultaneously. Outside the image sensor, it can be determined based on the data identification information whether the corresponding data is a digital signal or a timing result, and the light intensity or synthetic image signal represented by it is calculated based on the digital signal and the timing result, etc. In this case, the working mode of the multiplexer can be adjusted, and the two signals are output to the interface circuit without selection; or, the multiplexer may not be set, and the ADC and the timer directly output the obtained data to the interface circuit, and the interface circuit performs parallel-to-serial conversion on the data output by each pixel unit in the pixel array to form a data sequence and then outputs it to the outside, thereby further simplifying the structure of the image sensor; etc., the embodiment of the present disclosure does not limit this.

[0140] In this implementation, for the same pixel unit, after outputting the data and data identification information based on the embodiment of the present disclosure, at the receiving end, it is possible to determine that the data output is a digital signal based on the first data identification information, and then use a corresponding method to estimate the light intensity value in the corresponding time. For details, please refer to the above implementation, which will not be repeated here. At the same time, after determining that this is a timing result based on the second data identification information, use a corresponding method to estimate the light intensity value in the corresponding time. For details, please refer to the above implementation, which will not be repeated here. Based on the digital signal and the first data identification information that are repeatedly output by the first pixel circuit in the pixel unit and are arranged based on the timing relationship, the light intensity value and its change in the detection space in the timing can be correspondingly determined. Similarly, the timing result and the first data identification information that are repeatedly output by the second pixel circuit in the pixel unit and are arranged based on the timing relationship can be correspondingly determined. The light intensity value and its change in the detection space in the timing can be determined. After that, the light intensity values ​​obtained by the two methods can be aligned in time, so as to obtain the light intensity value and its change in the detection space of the pixel unit in the timing. In addition, the embodiments of the present disclosure may also adopt other methods to determine the light intensity value and its change in the time sequence of its detection space based on the correspondence between the two signals and their data identification information, and the embodiments of the present disclosure are not limited to this.

[0141] For an image sensor including a pixel array formed by m rows and n columns of pixel units, each pixel unit in the pixel array works independently of each other, and respectively realizes time domain acquisition of a light signal in a space in the observation scene to obtain output data (digital signal or timing result) and data identification information for output. The spaces corresponding to different pixel units do not overlap, and the pixel units in the pixel array cooperate with each other to cover the entire observation scene, thereby realizing time domain acquisition of the light signal in the entire observation scene, outputting data (digital signal or timing result) and data identification information, and obtaining light intensity values. As a result, the light intensity value and its change of the light signal corresponding to the entire observation scene can be obtained.

[0142] Optionally, in an application of an embodiment of the present disclosure, after obtaining the light intensity value and its change of the light signal corresponding to the entire observation scene, target detection can also be performed based on the light intensity value of one or more spaces in the observation scene at each moment. Since the visual information of a specific target is basically unchanged in a specific observation scene, target tracking can be achieved based on the light intensity value of one or more spatial regions in the observation scene in the time domain.

[0143] Optionally, in another application of the embodiments of the present disclosure, after obtaining the light intensity value of the corresponding space at each corresponding time, a preset pulse reconstruction algorithm can be used to generate a reconstructed image of the observation scene at each sampling moment based on the light intensity value of all spaces in the observation scene at each moment; wherein the grayscale value of each pixel point in the reconstructed image represents the light intensity value of the spatial area corresponding to the corresponding pixel unit, that is, in the reconstructed image at each moment, the pixel value of each pixel point (i, j) is the light intensity value of the spatial area corresponding to the pixel unit corresponding to the pixel point.

[0144] The above-mentioned preset pulse reconstruction algorithm can be, for example, a pulse reconstruction algorithm (TFI) based on inter-peak potential interval (ISI), a pulse reconstruction algorithm (TFP) based on fixed window sliding, a pulse reconstruction algorithm based on convolutional neural network (CNN), etc. The embodiments of the present disclosure do not limit the specific pulse reconstruction algorithm used.

[0145] The image sensor of the embodiment of the present disclosure may adopt various exposure incident modes, such as a direct incidence exposure mode or a back-illumination exposure mode; in addition, different shutter modes may also be adopted, such as a global exposure shutter and a rolling exposure shutter, which is not limited by the embodiment of the present disclosure.

[0146] Any signal processing method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any signal processing method provided in the embodiments of the present disclosure may be executed by a processor, such as a processor executing any signal processing method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory. This will not be described in detail below.

[0147] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0148] The image sensor and the signal processing method of the embodiments of the present disclosure correspond to each other in terms of specific implementation and beneficial technical effects, and the relevant contents can be referenced, quoted, and combined with each other, and will not be repeated here.

[0149] The embodiment of the present disclosure also provides an electronic device, which includes: a processor, a memory connected to the processor for communication, and an image sensor provided by any embodiment of the present disclosure.

[0150] Memory is used to store computer executable instructions;

[0151] The processor is used to execute the computer-executable instructions stored in the memory to control the image sensor to implement the signal processing method of any embodiment of the present disclosure.

[0152] The electronic device in the embodiments of the present disclosure may be any of the following devices or be applied to any of the following devices: image data acquisition equipment, audio / video player, navigation equipment, entertainment equipment, communication equipment, roadside traffic facilities, equipment in motor vehicles, industrial inspection equipment, flight equipment, medical equipment, security equipment, etc.

[0153] In the embodiments of the present disclosure, image data acquisition equipment may include but is not limited to: various cameras (such as pulse cameras, high-speed cameras, visual cameras, sports or wearable cameras, etc.), various cameras (such as mobile phone cameras, vehicle-mounted cameras, traffic cameras, security cameras, cameras installed on flying objects or various other devices, etc.), etc.; entertainment equipment may include but is not limited to: wearable devices (such as AR / VR glasses, sports watches, etc.), AR / VR projection devices, etc.; communication equipment may include but is not limited to: routers, servers with various functions (such as data storage function, authentication function, etc.), etc.; roadside traffic facilities may include but are not limited to: traffic lights, street lights, roadblocks, road signs, traffic signs, limit rods, warning tripods, etc.; equipment in motor vehicles may include but is not limited to: various equipment installed on vehicles such as cars, trains, subways, and airplanes; flight equipment may include but is not limited to: airplanes, aircraft, drones, etc.

[0154] Fig.10 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Fig.10 To describe an electronic device according to an embodiment of the present disclosure. The electronic device may be a first device Fig.10 This is a schematic diagram of the structure of an application embodiment of the electronic device disclosed in the present invention. Fig.10 The electronic device according to the embodiment of the present disclosure is described. The electronic device may be any one or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the collected input signals from them.

[0155] like Fig.10 As shown, the electronic device includes one or more processors, a memory and an image sensor. The processor may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device, such as an image sensor, to perform desired functions.

[0156] The memory may store one or more computer program products, and the memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may execute the computer instructions to implement the signal processing methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0157] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). In addition, the input device may also include, for example, a keyboard, a mouse, etc. The output device may output various information to the outside, including determined distance information, direction information, etc. The output device may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, etc.

[0158] Of course, to simplify, Fig.10 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0159] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the signal processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0160] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0161] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the signal processing method according to various embodiments of the present disclosure described in the above part of this specification.

[0162] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] Any signal processing method provided in the embodiments of the present disclosure may be executed by any appropriate device with data processing capabilities, including but not limited to: a terminal device and a server, etc. Alternatively, any signal processing method provided in the embodiments of the present disclosure may be executed by a processor, such as the processor executing any signal processing method mentioned in the embodiments of the present disclosure by calling corresponding instructions stored in a memory.

[0164] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc., various media that can store program codes.

[0165] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0166] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0168] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0169] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0170] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0171] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A signal processing method, applied to an image sensor, wherein the image sensor comprises a pixel unit, wherein the pixel unit comprises a first pixel circuit and a second pixel circuit, and the method comprises: Acquire a first electrical signal and a second electrical signal output by the first pixel circuit; wherein the second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on a light signal received during the exposure process after the first pixel circuit is reset; and the first electrical signal is an electrical signal generated by the first pixel circuit being reset during the exposure phase; Performing analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal; Acquire the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold value as the first time; wherein the third electrical signal is an electrical signal continuously output by the second pixel circuit in an exposure phase, and the third electrical signal is obtained based on photogenerated charges generated by the light signal received during the exposure process after the second pixel circuit is reset; At least one of the digital signal and the first time is output.

2. The method according to claim 1, characterized in that Acquiring a first electrical signal and a second electrical signal output by the first pixel circuit includes: Controlling the first pixel circuit to enter an exposure phase for exposure after being reset, wherein the first pixel circuit receives a light signal during the exposure process to generate and accumulate photogenerated charges; wherein the exposure duration of the exposure phase of the first pixel circuit is a first duration; Before the exposure phase of the first pixel circuit ends, the first pixel circuit is controlled to be reset and then enter a signal readout phase. In a first phase of the signal readout phase, the first pixel circuit outputs a reset signal as the first electrical signal. The duration of the signal readout phase is a second duration. After the exposure stage of the first pixel circuit ends, the first pixel circuit is controlled to enter the second stage in the signal readout stage. In the second stage, the first pixel circuit outputs an exposure signal generated by the accumulated photogenerated charges as the second electrical signal; wherein the signal readout stage includes the first stage and the second stage located after the first stage.

3. The method according to any one of claims 1-2, characterized in that: Acquiring the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold, comprising: Controlling the second pixel circuit to enter an exposure phase and a signal output phase after being reset, wherein the second pixel circuit receives a light signal during the exposure process to generate photogenerated charges, and outputs an exposure signal generated by the photogenerated charges as the third electrical signal; wherein the durations of the exposure phase and the signal output phase of the second pixel circuit are respectively the third durations; The time required for the third electrical signal output by the second pixel circuit in the signal output stage to reach a preset threshold is obtained.

4. The method according to any one of claims 1 to 3, characterized in that: The exposure phase and the signal readout phase of the first pixel circuit correspond to the exposure phase and the signal output phase of the second pixel circuit respectively; Outputting at least one of the digital signal and the first time comprises: The digital signal and the digital signal that has not reached the corresponding preset upper limit in the first time or the first time are output as the output data of this time, and data identification information of the output data, and the data identification information is used to identify the output data as a digital signal or a first time; wherein the preset upper limit corresponding to the digital signal is a preset code value upper limit, and the preset code value upper limit is the code value corresponding to the maximum light intensity that can be distinguished by the first pixel circuit; the preset upper limit corresponding to the first time is a first preset time upper limit, and the first preset time upper limit is the duration corresponding to the maximum light intensity.

5. The method according to any one of claims 1 to 3, characterized in that: An exposure phase and a signal readout phase of the first pixel circuit are inconsistent with at least one of an exposure phase and a signal output phase of the second pixel circuit; Outputting at least one of the digital signal and the first time comprises: In response to obtaining the digital signal, outputting the digital signal and first data identification information, wherein the first data identification information is used to indicate that the output data this time is the digital signal; In response to obtaining the first time, the first time and second data identification information are output, and the second data identification information is used to indicate that the output data this time is the first time.

6. The method according to any one of claims 1 to 5, characterized in that: The image sensor specifically includes a pixel array formed by arranging the pixel units in multiple rows and columns, and the image sensor further includes a first driving circuit, which is respectively connected to the first pixel circuits and the second pixel circuits in all the pixel units in the pixel array; The method further comprises: The first driving circuit controls the first pixel circuit and the second pixel circuit in a row of pixel units in the pixel array to reset according to a preset period, controls the first pixel circuit in the row of pixel units to enter an exposure phase for exposure with a first preset delay according to the preset period, controls the second pixel circuit in the row of pixel units to enter an exposure phase and a signal output phase for simultaneous exposure and third signal reading, controls the first pixel circuit in the row of pixel units to enter a signal readout phase for sequentially reading out the first signal and the second signal according to the preset period with a second preset delay; and, for the pixel array, controls the first pixel circuit in each row of pixel units row by row for exposure in a rolling exposure mode with a third preset delay, and controls the second pixel circuit in each row of pixel units row by row for exposure and third signal reading in a row by row exposure mode with the third preset delay; or, The first driving circuit controls the first pixel circuit and the second pixel circuit in a column of pixel units in the pixel array to reset according to a preset period, controls the first pixel circuit in the column of pixel units to enter an exposure stage for exposure with a first preset delay according to the preset period, controls the second pixel circuit in the column of pixel units to enter an exposure stage and a signal output stage for simultaneous exposure and third signal reading, controls the first pixel circuit in the column of pixel units to enter a signal reading stage for sequentially reading the first signal and the second signal according to the preset period with a second preset delay; and, for the pixel array, controls the first pixel circuit in each column of pixel units to expose column by column in a rolling exposure mode with a third preset delay, and controls the second pixel circuit in each column of pixel units to expose column by column in a column by column exposure mode with the third preset delay, and third signal reading.

7. The method according to any one of claims 1 to 5, characterized in that: The image sensor specifically includes a pixel array formed by arranging the pixel units in multiple rows and columns, and the image sensor also includes a first driving circuit and a second driving circuit, the first driving circuit is respectively connected to the first pixel circuits in all the pixel units in the pixel array, and the second driving circuit is respectively connected to the second pixel circuits in all the pixel units in the pixel array; The method further comprises: The first driving circuit controls, for each row of pixel units in the pixel array, the first pixel circuits in the row of pixel units to reset according to a preset period, controls the first pixel circuits in the row of pixel units to enter an exposure phase for exposure according to the preset period with a first preset delay, controls the first pixel circuits in the row of pixel units to enter a signal readout phase for sequentially reading out first signals and second signals according to the preset period with a second preset delay; and, for the pixel array, controls the first pixel circuits in each row of pixel units row by row for exposure in a rolling exposure mode with a third preset delay; The second driving circuit controls the second pixel circuit to reset according to a preset period for each row of pixel units in the pixel array, controls the second pixel circuit in the row of pixel units to enter an exposure phase and a signal output phase according to the preset period with a first preset delay to simultaneously perform exposure and third signal reading; and controls the second pixel circuit in each row of pixel units row by row to perform exposure and third signal reading in a row-by-row exposure mode with the third preset delay for the pixel array; or, The first driving circuit controls, for each column of pixel units in the pixel array, the first pixel circuits in the column of pixel units to reset according to a preset period, controls the first pixel circuits in the column of pixel units to enter an exposure phase for exposure according to the preset period with a first preset delay, controls the first pixel circuits in the column of pixel units to enter a signal readout phase for sequentially reading out first signals and second signals according to the preset period with a second preset delay; and, for the pixel array, controls the first pixel circuits in each column of pixel units column by column for exposure in a rolling exposure mode with a third preset delay; The second driving circuit controls the second pixel circuit to reset according to a preset period for a column of pixel units in the pixel array, and controls the second pixel circuit in the column of pixel units to enter the exposure phase and the signal output phase with a first preset delay according to the preset period to simultaneously perform exposure and third signal reading; and, for the pixel array, controls the second pixel circuit in each column of pixel units to perform exposure and third signal reading column by column in a mode of exposing column by column with the third preset delay.

8. An image sensor, comprising a pixel unit, a signal conversion unit and an interface circuit; wherein: The pixel unit includes a first pixel circuit and a second pixel circuit, and the signal conversion unit is connected to the first pixel circuit, the second pixel circuit and the interface circuit respectively; The first pixel circuit is used to output a first electrical signal and a second electrical signal; wherein the second electrical signal is an electrical signal output by the first pixel circuit after the exposure phase ends, and the second electrical signal is an electrical signal obtained by generating photogenerated charges based on the light signal received during the exposure process after the first pixel circuit is reset; and the first electrical signal is an electrical signal generated by the first pixel circuit being reset during the exposure phase; The signal conversion unit is used to perform analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal; and obtain the time required for the third electrical signal output by the second pixel circuit to reach a preset threshold as the first time; wherein the third electrical signal is an electrical signal continuously output by the second pixel circuit during an exposure phase, and the third electrical signal is obtained by generating photogenerated charges based on the light signal received by the second pixel circuit during the exposure process after the second pixel circuit is reset; The interface circuit is used to output at least one of the digital signal and the first time.

9. The image sensor according to claim 8, characterized in that: The image sensor specifically includes a pixel array formed by arranging a plurality of rows and columns of the pixel units, and a plurality of the signal conversion units, each of the signal conversion units correspondingly connecting a first pixel circuit and a second pixel circuit in a pixel unit in a dimension direction in the pixel array, wherein the pixel unit in a dimension direction is a row of pixel units or a column of pixel units; and / or, The first pixel circuit includes: a first photodiode, a transfer transistor, a capacitor, a first reset transistor, a first source follower transistor and a first readout transistor; One end of the first photodiode is grounded, and the other end is connected to one end of the capacitor and the gate end of the first source follower transistor through the transfer transistor; the source end of the transfer transistor is connected to the other end of the first photodiode, and the drain end is connected to one end of the capacitor and the gate end of the first source follower transistor respectively, and the gate end is used to receive a first external control signal for controlling the transfer transistor to be turned on or off; one end of the capacitor is connected to the source end of the first reset transistor, the gate end of the first source follower transistor, and the other end of the first photodiode through the transfer transistor, and the other end is grounded; the source end of the first reset transistor is connected to one end of the capacitor, the drain end is connected to a power supply signal, and the gate end is used to receive a second external control signal for controlling the first reset transistor to be turned on or off; the source end of the first source follower transistor is connected to the drain end of the first readout transistor, the drain end is connected to a power supply signal, and the gate end is connected to one end of the capacitor and the drain end of the transfer transistor respectively; the source end of the first readout transistor serves as the output end of the first pixel circuit, the drain end is connected to the source end of the first source follower transistor, and the gate end is used to receive a third external control signal for controlling the first readout transistor to be turned on or off; and / or, The second pixel circuit includes: a second photodiode, a second reset transistor, a second source follower transistor, and a second readout transistor; One end of the second photodiode is grounded, and the other end is respectively connected to the source end of the second reset transistor and the gate end of the second source follower transistor; the source end of the second reset transistor is connected to the other end of the second photodiode, the drain end is connected to the power supply signal, and the gate end is used to receive a fourth external control signal for controlling the second reset transistor to be turned on or off; the source end of the second source follower transistor is connected to the drain end of the second readout transistor, the drain end is connected to the power supply signal, and the gate end is respectively connected to the other end of the second photodiode and the source end of the second reset transistor; the source end of the second readout transistor serves as the output end of the second pixel circuit, the drain end is connected to the source end of the second source follower transistor, and the gate end is used to receive a fifth external control signal for controlling the second readout transistor to be turned on or off.

10. The image sensor according to claim 9, characterized in that: The signal conversion unit includes: an analog-to-digital conversion circuit, a comparator and a timer; The input end of the analog-to-digital conversion circuit is connected to the output end of the corresponding first pixel circuit, and is used to receive the first electrical signal and the second electrical signal output by the corresponding first pixel circuit, and perform analog-to-digital conversion on the difference between the first electrical signal and the second electrical signal to obtain a digital signal; the output end of the analog-to-digital conversion circuit is connected to the input end of the interface circuit, and is used to output the digital signal to the interface circuit; One input end of the comparator is connected to the output end of the corresponding second pixel circuit, and is used to receive the third electrical signal output by the corresponding second pixel circuit; another input end of the comparator is used to receive the signal of the preset threshold; the output end of the comparator is connected to the timer, and is used to output a comparison result signal to the timer; The input end of the timer is connected to the output end of the comparator for receiving the comparison result signal; the output end of the timer is connected to the input end of the interface circuit for outputting the timing result as the first time when the comparison result signal changes.

11. The image sensor according to claim 10, characterized in that: The signal conversion unit further includes: a multiplexer; The first input terminal of the multiplexer is connected to the output terminal of the analog-to-digital conversion circuit, and is used to receive the digital signal output by the analog-to-digital conversion circuit; The second input end of the multiplexer is connected to the output end of the timer, and is used to receive the timing result output by the timer; The output end of the multiplexer serves as the output end of the signal conversion unit and is connected to the input end of the interface circuit, and is used to output the digital signal and the digital signal that does not reach the corresponding preset upper limit in the first time or the first time as the current output data, and data identification information of the output data to the interface circuit, wherein the data identification information is used to identify that the output data is a digital signal or a first time, wherein the preset upper limit corresponding to the digital signal is a preset code value upper limit, the preset code value upper limit is the code value corresponding to the maximum light intensity that can be distinguished by the first pixel circuit, the preset upper limit corresponding to the first time is a first preset time upper limit, and the first preset time upper limit is the duration corresponding to the maximum light intensity; or, the output end of the multiplexer serves as the output end of the signal conversion unit and is connected to the input end of the interface circuit, and is used to output the digital signal and the first data identification information in response to obtaining the digital signal, wherein the first data identification information is used to indicate that the current output data is the digital signal, and in response to obtaining the first time, the first time and the second data identification information are output, wherein the second data identification information is used to indicate that the current output data is the first time.

12. The image sensor according to any one of claims 9 to 11, characterized in that: Also includes a first drive circuit; The first driving circuit is respectively connected to the first pixel circuit and the second pixel circuit in all pixel units in the pixel array, and is used for: For each row of pixel units in the pixel array, respectively, the first pixel circuit and the second pixel circuit in the row of pixel units are controlled to reset according to a preset cycle, the first pixel circuit in the row of pixel units is controlled to enter an exposure phase for exposure with a first preset delay according to the preset cycle, and the second pixel circuit in the row of pixel units is controlled to enter an exposure phase and a signal output phase for simultaneous exposure and third signal reading, and the first pixel circuit in the row of pixel units is controlled to enter a signal readout phase with a second preset delay according to the preset cycle to sequentially read out the first signal and the second signal; and, for the pixel array, the first pixel circuit in each row of pixel units is controlled row by row for exposure in a rolling exposure mode with a third preset delay, and the second pixel circuit in each row of pixel units is controlled row by row for exposure and third signal reading in a row by row exposure mode with the third preset delay; or, For a column of pixel units in the pixel array, respectively, the first pixel circuit and the second pixel circuit in the column of pixel units are controlled to be reset according to a preset cycle, the first pixel circuit in the column of pixel units is controlled to enter the exposure stage for exposure with a first preset delay according to the preset cycle, and the second pixel circuit in the column of pixel units is controlled to enter the exposure stage and the signal output stage for simultaneous exposure and third signal reading, and the first pixel circuit in the column of pixel units is controlled to enter the signal reading stage with a second preset delay according to the preset cycle to sequentially read out the first signal and the second signal; and, for the pixel array, the first pixel circuit in each column of pixel units is controlled to be exposed column by column in a rolling exposure mode with a third preset delay, and the second pixel circuit in each column of pixel units is controlled to be exposed column by column in a column-by-column exposure mode with the third preset delay, and the third signal is read out column by column.

13. The image sensor according to any one of claims 9 to 11, characterized in that: It also includes a first driving circuit and a second driving circuit; the first driving circuit is respectively connected to the first pixel circuits in all pixel units in the pixel array, and the second driving circuit is respectively connected to the second pixel circuits in all pixel units in the pixel array; The first driving circuit is used to control the first pixel circuits in a row of pixel units in the pixel array to reset according to a preset period, control the first pixel circuits in the row of pixel units to enter an exposure phase for exposure according to the preset period with a first preset delay, and control the first pixel circuits in the row of pixel units to enter a signal readout phase for sequentially reading out first signals and second signals according to the preset period with a second preset delay; and, for the pixel array, control the first pixel circuits in each row of pixel units row by row for exposure in a rolling exposure mode with a third preset delay; The second driving circuit is used to control the second pixel circuit to reset according to a preset period for each row of pixel units in the pixel array, and to control the second pixel circuit in the row of pixel units to enter an exposure phase and a signal output phase according to the preset period with a first preset delay to simultaneously perform exposure and third signal reading; and, for the pixel array, to control the second pixel circuit in each row of pixel units row by row in a mode of row-by-row exposure with the third preset delay to perform exposure and third signal reading; or, The first driving circuit is used to control the first pixel circuits in a column of pixel units in the pixel array to reset according to a preset period, control the first pixel circuits in the column of pixel units to enter an exposure phase for exposure according to the preset period with a first preset delay, and control the first pixel circuits in the column of pixel units to enter a signal readout phase for sequentially reading out first signals and second signals according to the preset period with a second preset delay; and, for the pixel array, control the first pixel circuits in each column of pixel units column by column for exposure in a rolling exposure mode with a third preset delay; The second driving circuit is used to control the resetting of the second pixel circuits in a column of pixel units in the pixel array according to a preset period, and to control the second pixel circuits in the column of pixel units to enter an exposure phase and a signal output phase with a first preset time delay according to the preset period to simultaneously perform exposure and third signal reading; and, for the pixel array, to control the second pixel circuits in each column of pixel units column by column to perform exposure and third signal reading in a column-by-column exposure mode with the third preset time delay.

14. An electronic device comprising: A processor, a memory respectively connected to the processor in communication, and an image sensor according to any one of claims 8 to 13; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions stored in the memory to control the image sensor to implement the signal processing method according to any one of claims 1 to 7.

15. The electronic device according to claim 14, wherein the electronic device is any one of the following devices: image data acquisition equipment, audio / video player, navigation equipment, communication equipment, roadside traffic facilities, industrial detection equipment, flight equipment, medical equipment, and security equipment.

Images