A mode-switchable image sensor and an image sensor control method

Abstract

The present invention belongs to the technical field of image sensors, and particularly relates to a mode-switchable image sensor and an image sensor control method, including a plurality of pixels, each pixel including: a photoelectric conversion circuit, an APS pixel signal readout circuit, and used to generate an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit when the pixel is an APS pixel; a mode switching circuit, used to set the pixel as an APS pixel or an event-type sensor pixel according to a mode switching control signal; an event signal readout circuit, used to generate an event-type sensor signal corresponding to the charge generated by the photoelectric conversion circuit when the pixel is an event-type sensor pixel; a signal control processor, used to send a mode switching control signal to the mode switching circuit, and generate a first image according to the APS pixel signal, or generate a second image according to the event-type sensor signal. The present invention realizes the integration of the functions of APS pixels and event-type sensor pixels while greatly reducing the volume of the sensor.

Description

Technical Field

[0001] The present invention belongs to the technical field of image sensors, and particularly relates to a mode-switchable image sensor and an image sensor control method. Background Art

[0002] Traditional types of image sensors include complementary metal oxide semiconductor image sensors and event-based vision sensors. In recent years, image sensors based on CMOS technology have developed rapidly and are increasingly widely used in fields such as automotive electronics, intelligent manufacturing, industrial monitoring, and military reconnaissance. However, under the conditions of high resolution and high frame rate, due to the generation of a huge amount of data, the chip power consumption is large, resulting in high power consumption requirements. The event-based vision sensor only detects events of changes in light intensity and outputs event images based on the detected events. Therefore, the amount of data of the dynamic vision sensor is low, but the resolution is generally low.

[0003] To meet the usage requirements, sensors that combine CMOS image sensors and event-based vision sensors have emerged on the market. However, simply combining them easily leads to the problem of excessive sensor volume. Therefore, sensors that share optoelectronic conversion devices have gradually emerged to avoid the problem of excessive sensor size. Specifically, a patent for invention with the application number CN201910856811.4 discloses an image sensor including CIS (CMOS image sensor) pixels, DVS (dynamic vision sensor) pixels, and an image signal processor. The CIS pixel includes: an optoelectronic conversion device that generates charges corresponding to incident light; and a readout circuit that generates an output voltage corresponding to the generated charges. The DVS pixel detects changes in the intensity of incident light based on the generated charges and outputs an event signal, and does not include a separate optoelectronic conversion device. The image signal processor enables the optoelectronic conversion device to be connected to the readout circuit or the DVS pixel.

[0004] Although the image sensor in the above patent document includes CIS pixels and DVS pixels and can reduce the overall volume of the sensor to some extent, it still has the problem of excessive sensor volume. Summary of the Invention

[0005] The purpose of the present invention is to provide a mode-switchable image sensor and an image sensor control method, aiming to solve the technical problem of excessive volume of sensors including CIS pixels and DVS pixels in the prior art.

[0006] To achieve the above purpose, an embodiment of the present invention provides a mode-switchable image sensor, including a plurality of pixels, and each pixel includes:

[0007] An optoelectronic conversion circuit for generating charges corresponding to incident light incident on the pixel;

[0008] An APS pixel signal readout circuit, connected to the photoelectric conversion circuit, and configured to generate an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit when the pixel is an APS pixel; further comprising:

[0009] A mode switching circuit, connected to the photoelectric conversion unit, and configured to set the pixel as an APS pixel or an event-type sensor pixel according to a mode switching control signal;

[0010] An event signal readout circuit, connected to the photoelectric conversion unit, and configured to generate an event-type sensor signal corresponding to the charge generated by the photoelectric conversion circuit when the pixel is an event-type sensor pixel;

[0011] A signal control processor, connected to the photoelectric conversion circuit, the APS pixel signal readout circuit, the mode switching circuit, and the event signal readout circuit. The signal control processor is configured to send a mode switching control signal to the mode switching circuit, and generate a first image according to the APS pixel signal, or generate a second image according to the event-type sensor signal.

[0012] Optionally, the mode switching circuit includes a reset transistor; the source of the reset transistor is connected to the photoelectric conversion circuit, the drain of the reset transistor is connected to a power supply; the reset transistor is configured to reset the voltage of the floating diffusion node when the pixel is an APS pixel; the reset transistor is further configured to output a logarithmic voltage corresponding to the photocurrent corresponding to the charge generated by the photoelectric conversion device when the pixel is an event-type sensor pixel.

[0013] Optionally, the mode switching circuit further includes a first switch and a second switch; the signal control processor controls the first switch and the second switch to be turned on or off to set the pixel as an APS pixel or an event-type sensor pixel; when the first switch is turned on and the second switch is turned off, the drain of the reset transistor is connected to the first switch, and the pixel is set as an APS pixel; when the mode switching control signal controls the first switch to be turned off and the second switch to be turned on, the gate and the drain of the reset transistor are respectively connected to both ends of the second switch, and the pixel is set as an event-type sensor pixel.

[0014] Optionally, the photoelectric conversion circuit includes a photoelectric conversion device, a transfer transistor, and a floating diffusion node. The first end of the photoelectric conversion device is grounded, the transfer transistor is connected to the second end of the photoelectric conversion device, and the transfer transistor is also connected to the APS pixel signal readout circuit, the reset transistor, and the event signal readout circuit. The transfer transistor TX is configured to transfer the charge generated by the photoelectric conversion device to the floating diffusion node.

[0015] Optionally, the event signal readout circuit includes a first driving transistor and a first readout unit. The first driving transistor is connected to the floating diffusion node, the first readout unit is connected to the first driving transistor, and the first readout unit is further connected to the signal control processor.

[0016] Optionally, the APS pixel signal readout circuit includes a second driving transistor, a selection transistor, and a second readout unit. The second driving transistor is connected to the floating diffusion node, the selection transistor is connected to the second driving transistor, the second readout unit is connected to the selection transistor, and the second readout unit is further connected to the signal control processor.

[0017] One or more of the above technical solutions in the mode-switchable image sensor provided by the embodiments of the present invention have at least one of the following technical effects:

[0018] In the present invention, by setting the signal control processor to send a mode switching control signal to the mode switching circuit, the mode switching circuit sets the pixel as an APS pixel or an event-type sensor pixel according to the mode switching control signal. When the pixel is set as an APS pixel, the APS pixel signal readout circuit generates an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit. When the pixel is set as an event-type sensor pixel, the event signal readout circuit generates an event-type sensor signal corresponding to the charge generated by the photoelectric conversion circuit. Furthermore, through the setting of the mode switching circuit, the switching between the APS pixel and the event-type sensor pixel is realized, achieving the integration of the functions of the APS pixel and the event-type sensor pixel while greatly reducing the volume of the sensor.

[0019] To achieve the above object, an embodiment of the present invention provides an image sensor control method, which is based on the above mode-switchable image sensor. The method specifically includes:

[0020] Step S100: The signal control processor sends a mode switching control signal to the mode switching control circuit;

[0021] Step S200: When the mode switching circuit sets the pixel as an APS pixel according to the mode switching control signal, the APS pixel signal readout circuit generates an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit;

[0022] Step S300: When the mode switching circuit sets the pixel as an event-type sensor pixel according to the mode switching control signal, the event signal readout circuit generates an event-type sensor signal corresponding to the charge generated by the photoelectric conversion circuit;

[0023] Step S400: The signal control processor generates a first image when acquiring the APS pixel signal, or generates a second image when acquiring the event sensor signal.

[0024] Optionally, the method further includes:

[0025] Step S510: When the mode switching circuit sets the pixel as an APS pixel according to the mode switching control signal, the signal control processor controls the event signal readout circuit to generate an event sensor signal;

[0026] Step S520: When the mode switching circuit sets the pixel as an event sensor pixel according to the mode switching control signal, the signal control processor controls the APS pixel signal readout circuit to generate an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit.

[0027] Optionally, the method further includes:

[0028] Step S610: Based on the signal control processor 500, control a first preset number of pixels to form a pixel unit, where the pixel unit includes a plurality of pixel matrices;

[0029] Step S620: Based on the signal control processor 500, set the pixel matrices on one pair of opposite sides in the pixel unit as pixels with the same mode, and the pixel matrices on the other pair of opposite sides as the same or different.

[0030] Optionally, the method further includes:

[0031] Step S710: Based on the signal control processor, control a second preset number of pixels to form a first pixel region;

[0032] Step S720: Based on the signal control processor, control a third preset number of pixels to form a second pixel region;

[0033] Step S730: Based on the signal control processor, set all the pixels in the first pixel region as APS pixels and set all the pixels in the second pixel region as event sensor pixels, or set all the pixels in the first pixel region as event sensor pixels and set all the pixels in the second pixel region as APS pixels;

[0034] Step S740: According to the signal control processor, control to acquire the images output from the first pixel region and the second pixel region within a preset time period.

[0035] One or more of the above technical solutions in the image sensor control method provided by the embodiments of the present invention have at least one of the following technical effects:

[0036] The image sensor control method of the present invention is carried out based on the above-mentioned mode-switchable image sensor. Therefore, the image sensor control method can send a mode-switching control signal to the mode-switching control circuit through the signal control processor, so as to set the pixel as an APS pixel or an event-type sensor pixel. Furthermore, the signal control processor can generate a first image when acquiring the APS pixel signal, or generate a second image when acquiring the event-type sensor signal. The switching between the APS pixel and the event-type sensor pixel is realized through the setting of the mode-switching circuit, achieving the integration of the functions of the APS pixel and the event-type sensor pixel while greatly reducing the volume of the sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0038] Figure 1 It is a circuit structure block diagram of a pixel in the mode-switchable image sensor provided by an embodiment of the present invention;

[0039] Figure 2 It is a circuit schematic diagram of a pixel in the mode-switchable image sensor provided by an embodiment of the present invention;

[0040] Figure 3 It is a schematic flow chart of the image sensor control method provided by an embodiment of the present invention;

[0041] Figure 4 It is a schematic flow chart of the image sensor control method provided by another embodiment of the present invention;

[0042] Figure 5 It is a schematic structural diagram of a pixel unit provided by an embodiment of the present invention;

[0043] Figure 6 It is a schematic flow chart of setting a pixel unit provided by an embodiment of the present invention;

[0044] Figure 7 It is a schematic structural diagram of a pixel unit provided by another embodiment of the present invention;

[0045] Figure 8 It is a schematic structural diagram of a pixel unit provided by other embodiments of the present invention;

[0046] Figure 9A flowchart for setting a first pixel region and a second pixel region and obtaining images output by the first pixel region and the second pixel region provided by an embodiment of the present invention.

[0047] Among them, the reference numerals in the figure are as follows:

[0048] 100, a photoelectric conversion circuit; 200, an APS pixel signal readout circuit; 210, a second readout unit; 300, a mode switching circuit; 400, an event signal readout circuit; 410, a first readout unit; 500, a signal control processor; 600, a pixel unit. Detailed implementation manners

[0049] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the embodiments of the present invention, and should not be construed as a limitation of the present invention.

[0050] In the description of the embodiments of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0051] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "a plurality" means two or more unless otherwise specifically defined.

[0052] In the embodiments of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0053] In an embodiment of the present invention, as Figure 1As shown, a mode-switchable image sensor is provided, including a plurality of pixels, each pixel including a photoelectric conversion circuit 100, an APS pixel signal readout circuit 200, a mode switching circuit 300, an event signal readout circuit 400, and a signal control processor 500.

[0054] Among them, the photoelectric conversion circuit 100 is used to generate charges corresponding to the incident light incident on the pixel;

[0055] The APS pixel signal readout circuit 200 is connected to the photoelectric conversion circuit 100 and is used to generate an APS pixel signal corresponding to the charges generated by the photoelectric conversion circuit 100 when the pixel is an APS pixel.

[0056] The mode switching circuit 300 is connected to the photoelectric conversion unit and is used to set the pixel as an APS pixel or an event-type sensor pixel according to a mode switching control signal. The event signal readout circuit 400 is connected to the photoelectric conversion unit and is used to generate an event-type sensor signal corresponding to the charges generated by the photoelectric conversion circuit 100 when the pixel is an event-type sensor pixel.

[0057] The signal control processor 500 is connected to the photoelectric conversion circuit 100, the APS pixel signal readout circuit 200, the mode switching circuit 300, and the event signal readout circuit 400. The signal control processor 500 is used to send a mode switching control signal to the mode switching circuit 300 and generate a first image according to the APS pixel signal, or generate a second image according to the event-type sensor signal.

[0058] In the present invention, by setting the signal control processor 500 to send a mode switching control signal to the mode switching circuit 300, the mode switching circuit 300 sets the pixel as an APS pixel or an event-type sensor pixel according to the mode switching control signal. When the pixel is set as an APS pixel, the APS pixel signal readout circuit 200 generates an APS pixel signal corresponding to the charges generated by the photoelectric conversion circuit 100. When the pixel is set as an event-type sensor pixel, the event signal readout circuit 400 generates an event-type sensor signal corresponding to the charges generated by the photoelectric conversion circuit 100. Furthermore, through the setting of the mode switching circuit 300, the switching between the APS pixel and the event-type sensor pixel is realized, achieving the integration of the functions of the APS pixel and the event-type sensor pixel while greatly reducing the volume of the sensor.

[0059] In this embodiment, the first image is a grayscale image, an RGB image, or others. The signal control processor 500 can perform various processes on the grayscale image or the RGB image, such as color interpolation, color correction, automatic white balance, gamma correction, color saturation correction, formatting, bad pixel correction, and chromaticity correction.

[0060] The second image is an event image. The signal control processor 500 can also correct or calibrate the timestamp values of noise pixels, thermal pixels, or dead pixels based on the temporal correlation between the timestamp values of adjacent pixels in the event image.

[0061] In another embodiment of the present invention, as Figure 1 - Figure 2 shown, the mode switching circuit 300 includes a reset transistor RT; the source of the reset transistor RT is connected to the photoelectric conversion circuit 100, and the drain of the reset transistor RT is connected to a power supply, and the voltage provided by the power supply is output through the power supply terminal VDD.

[0062] The reset transistor RT is used to reset the voltage of the floating diffusion node when the pixel is an APS pixel. The reset transistor RT is also used to output the logarithmic voltage corresponding to the photocurrent corresponding to the charge generated by the photoelectric conversion device when the pixel is an event-type sensor pixel.

[0063] In another embodiment of the present invention, as Figure 1 - Figure 2 shown, the mode switching circuit 300 further includes a first switch S1 and a second switch S2.

[0064] The first switch S1 and the second switch S2 are controlled to be turned on or off based on the mode switching control signal issued by the signal control processor 500; when the mode switching control signal controls the first switch S1 to be turned on and the second switch S2 to be turned off, at this time, the drain of the reset transistor RT is connected to the first switch S1, and the pixel is set as an APS pixel; when the mode switching control signal controls the first switch S1 to be turned off and the second switch S2 to be turned on, the gate and drain of the reset transistor RT are respectively connected to both ends of the second switch S2, and the pixel is set as an event-type sensor pixel.

[0065] In another embodiment of the present invention, the first switch S1 and the second switch S2 are separately provided outside the pixel.

[0066] In another embodiment of the present invention, as Figure 1 - Figure 2 shown, the photoelectric conversion circuit 100 includes a photoelectric conversion device PSD, a transfer transistor TX, and a floating diffusion node FD. The source of the photoelectric conversion device PSD is grounded, the transfer transistor TX is connected to the drain of the photoelectric conversion device PSD, and the floating diffusion node FD is also connected to the APS pixel signal readout circuit 200, the reset transistor RT, and the event signal readout circuit 400. The transfer transistor TX is used to transfer the charge generated by the photoelectric conversion device PSD to the floating diffusion node FD.

[0067] When incident light irradiates the photoelectric conversion device PSD, the photoelectric conversion device PSD generates photoelectrons, that is, charges, in response to the incident light. That is to say, the photoelectric conversion device PSD can convert the optical signal into an electrical signal to generate a photocurrent IP.

[0068] In this embodiment, the optoelectronic conversion device can be a photodiode, a phototransistor, a pinned photodiode, or other similar components.

[0069] In another embodiment of the present invention, as Figure 1 - Figure 2 shown, the event signal readout circuit 400 includes a first driving transistor DT1 and a first readout unit 410. The first driving transistor DT1 is connected to the floating diffusion node FD, the first readout unit 410 is connected to the first driving transistor DT1, and the signal read out by the first readout unit 410 is transmitted to the signal control processor 500.

[0070] Specifically, when the mode switching control signal controls the first switch S1 to be disconnected and the second switch S2 to be closed, the gate and drain of the reset transistor RT are short-circuited through the second switch S2. After the gate-drain short-circuit, a drain-source current from the drain to the source is formed. The gate and source of the reset transistor RT form a gate-source voltage. The drain-source current and the gate-source voltage are in a logarithmic relationship. At this time, the pixel is set as an event-type sensor pixel.

[0071] Further, when the transfer transistor TX is always in the on state, the photocurrent generated by the optoelectronic conversion device PSD is transmitted to the reset transistor RT shorted by the gate-drain through the floating diffusion node FD. The reset transistor RT outputs a logarithmic voltage corresponding to the photocurrent, and the first driving transistor DT1 serves as a voltage buffer to transmit the logarithmic voltage to the first readout unit 410.

[0072] The first readout unit 410 processes the logarithmic voltage output by the first driving transistor DT1, such as comparing it with a reference voltage, to determine whether the light intensity incident on the optoelectronic conversion device PSD increases or decreases, and outputs an event-type sensor signal corresponding to the determination result.

[0073] After the first readout unit 410 outputs the event-type sensor signal, the signal control processor 500 sends a reset signal to the reset transistor RT to turn on the reset transistor RT, thereby resetting the first readout unit 410.

[0074] In another embodiment of the present invention, as Figure 1 - Figure 2 shown, the APS pixel signal readout circuit 200 includes a second driving transistor DT2, a selection transistor ST, and a second readout unit 210. The second driving transistor DT2 is connected to the floating diffusion node FD, the selection transistor ST is connected to the second driving transistor DT2, the second readout unit 210 is connected to the selection transistor ST, and the second readout unit 210 is also connected to the signal control processor 500.

[0075] Further, when the mode switching control signal controls the first switch S1 to close and the second switch S2 to open, the pixel is set as an APS pixel at this time.

[0076] During the conduction period of the transfer transistor TX, the charge provided by the photoelectric conversion device PSD accumulates in the floating diffusion region FD. The floating diffusion node FD can be reset to the power supply voltage provided when the reset transistor RT conducts, and the second driving transistor DT2 can be used as a source follower amplifier to amplify the change in the potential of the floating diffusion region FD.

[0077] The reset transistor RT can be reset by a reset signal and can supply the power supply voltage to the floating diffusion node FD. In this case, the charge accumulated in the floating diffusion node FD can move to the terminal of the power supply voltage, and the voltage of the floating diffusion node FD can be reset, that is, the power supply voltage can be used as the reset voltage, or a separate reset voltage can be provided to reset the floating diffusion node FD.

[0078] Further, the selection transistor ST can be driven by a selection signal SEL and can select the pixels to be read in units of rows. When the selection transistor ST conducts, the potential of the floating diffusion region FD is amplified by the second driving transistor DT2 and can be transmitted to the drain of the selection transistor ST, thereby realizing the output of the potential of the floating diffusion region FD through the second driving transistor DT2, so that the second readout unit 210 reads out the signal. The readout signal is an APS pixel signal, and the signal control processor 500 outputs a grayscale signal according to this signal.

[0079] It can be seen that the present invention realizes the switching between APS pixels and event-type sensor pixels by a photoelectric conversion device PSD and by controlling the first switch 121 or the second switch 122 to short-circuit or not short-circuit the gate and drain of the reset transistor RT, effectively reducing components, reducing the overall size and volume of the sensor, and reducing noise at the same time.

[0080] Further, in another embodiment of the present invention, when the pixel operates in the APS pixel mode, one ADC can be shared by a column of pixels, so that the selection transistor ST can select pixels in units of rows to perform row scanning to read out the grayscale signals of the pixels; or, it can also be divided into multiple sub-pixel arrays to share one ADC, and the ADC reads out the grayscale signals of the pixels in its sub-pixel arrays in a time-sharing manner, so as to output a grayscale image according to the grayscale signals of the pixels.

[0081] In another embodiment of the present invention, as Figure 3 shown, the present invention also provides an image sensor control method, which can be performed on the above-mentioned mode-switchable image sensor. The method specifically includes:

[0082] Step S100: The signal control processor 500 sends a mode switching control signal to the mode switching control circuit;

[0083] Among them, the mode switching control signal can be two signals respectively sent to the first switch S1 and the second switch S2 in the mode switching control circuit, thereby realizing the closing and opening of the first switch S1 and the second switch S2.

[0084] Step S200: When the mode switching circuit 300 sets the pixel to an APS pixel according to the mode switching control signal, the APS pixel signal readout circuit 200 generates an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit 100;

[0085] Step S300: When the mode switching circuit 300 sets the pixel to an event-type sensor pixel according to the mode switching control signal, the event signal readout circuit 400 generates an event-type sensor signal corresponding to the charge generated by the photoelectric conversion circuit 100;

[0086] Furthermore, through the setting of the mode switching circuit 300, the pixel can be set to the APS pixel mode or the event-type sensor pixel mode as needed, thereby realizing the switching setting of the pixel and meeting the usage requirements.

[0087] Step S400: The signal control processor 500 generates a first image when acquiring the APS pixel signal, or generates a second image when acquiring the event-type sensor signal.

[0088] In another embodiment of the present invention, as Figure 4 shown, the method further includes:

[0089] Step S510: When the mode switching circuit 300 sets the pixel to an APS pixel according to the mode switching control signal, the signal control processor 500 controls the event signal readout circuit 400 to generate an event-type sensor signal;

[0090] In this step, the signal control processor 500 controls the APS pixel signal readout circuit 200 not to work, and at the same time controls the event signal readout circuit 400 to generate an event-type sensor signal, realizing the output of the event-type sensor signal in the APS pixel mode. In this way, the data volume can be effectively reduced, and the change of the event can also be output when the environment is relatively dark. And an event image can be output according to the event signals output by multiple pixels.

[0091] Step S520: When the mode switching circuit 300 sets the pixel to an event-type sensor pixel according to the mode switching control signal, the signal control processor 500 controls the APS pixel signal readout circuit 200 to generate an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit 100.

[0092] In this step, in the event sensor pixel mode, the signal control processor 500 controls the APS pixel signal readout circuit 200 to generate APS pixel signals, realizing the output of APS pixel signals in the event sensor pixel mode. At this time, the amount of information is large, and richer image detail information can be presented. Moreover, grayscale images can be output according to the APS pixel signals output by multiple pixels.

[0093] Furthermore, when the pixel operates in the linear mode, that is, when it is set to the APS pixel mode, the signal control processor 500 can choose to output event sensor signals or APS pixel signals. When operating in the logarithmic mode, that is, when it is set to the event sensor pixel mode, the signal control processor 500 can also choose to output event sensor signals or APS pixel signals, enabling the present invention to have four selection modes, and realizing that in different modes, event images are output according to the event sensor signals output by multiple pixels or grayscale images are output according to the APS pixel signals output by multiple pixels, so as to meet the requirements of power consumption and data volume processing.

[0094] In another embodiment of the present invention, as Figure 5 - Figure 6 shown, the method further includes:

[0095] Step S610: Based on the signal control processor 500, control a first preset number of pixels to form pixel units, where the pixel units include multiple pixel matrices;

[0096] In this step, in the mode-switchable image sensor, some pixels can operate in the APS pixel mode, and some can operate in the event sensor pixel mode, and each pixel works independently.

[0097] Multiple pixel matrices can be set in the pixel units. For different pixel matrices, some can operate in the APS pixel mode, and some can operate in the event sensor pixel mode, and each pixel matrix works independently. In this way, APS images and event images can be output simultaneously according to different matrices, that is, two images can be output each time.

[0098] The pixels in the mode-switchable image sensor can also form pixel units according to different quantities. In this embodiment, the first preset number can specifically be 4.

[0099] Step S620: Based on the signal control processor 500, set the pixel matrices on one pair of opposite sides in the pixel unit as pixels with the same mode, and the pixel matrices on the other pair of opposite sides as the same or different.

[0100] In this step, as Figure 5As shown, when taking four pixels as one pixel unit, the number of pixels in one pixel matrix is 1. Then, setting the pixel matrices on opposite sides in the pixel unit to have the same pixel pattern means setting the pixels on opposite sides in the pixel unit to have the same pixel pattern. Figure 5 For one pair of opposite sides, the A pixels are pixels of the same pattern, and for the other pair of opposite sides, the D pixels are pixels of the same pattern. With this setting, when interpolating the D pixels, the surrounding A pixel values can be used for processing to improve the image resolution.

[0101] Figure 5 The pixel unit number in the example is 600.

[0102] In another embodiment of the present invention, as Figure 7 shown, the pixel unit 600 can also have other arrangements. In this embodiment, the pixel unit 600 includes four pixel matrices, and the numbers of the four pixel matrices are 611, 612, 613, and 614 respectively. Among them, the pixel matrix 611 and the pixel matrix 614 are the pixel matrices on one pair of opposite sides, and the pixel matrix 612 and the pixel matrix 613 are the pixel matrices on the other pair of opposite sides.

[0103] Furthermore, the pixels in the pixel matrix 611 and the pixel matrix 614 are all A pixels. The same label means the same pattern. The pixels in the pixel matrix 612 and the pixel matrix 613 are all D pixels. That is, the pixels in the pixel matrices on opposite sides in the pixel unit are pixels of the same pattern.

[0104] Even further, in this embodiment, the pixels in the pixel matrix 611 and the pixel matrix 614 are in the APS pixel pattern, and the pixels in the pixel matrix 612 and the pixel matrix 613 are in the event-based sensor pixel pattern, and each pixel matrix works independently. In this way, APS images and event images can be output simultaneously according to different matrices, that is, two images can be output each time.

[0105] In another embodiment of the present invention, as Figure 8 shown, in this embodiment, the number of pixel matrices is four, and the four pixel matrices are the pixel matrix 621, the pixel matrix 622, the pixel matrix 623, and the pixel matrix 624 respectively. Among them, the pixel matrix 621 and the pixel matrix 624 are the pixel matrices on one pair of opposite sides, and the pixel matrix 622 and the pixel matrix 623 are the pixel matrices on the other pair of opposite sides.

[0106] In this embodiment, the pixels in pixel matrix 621 and pixel matrix 624 are the same. All the pixels in pixel matrix 622 are A pixels, and all the pixels in pixel matrix 623 are D pixels. The pixels in pixel matrix 622 and pixel matrix 623 are pixels of different modes. That is, the pixels in the pixel matrix on one pair of opposite sides of the pixel unit are pixels of the same mode, and the pixels in the pixel matrix on the other pair of opposite sides are pixels of different modes.

[0107] Specifically, the pixels in pixel matrix 621, pixel matrix 623, and pixel matrix 624 are in the event sensor pixel mode. The pixels in pixel matrix 612 and pixel matrix 613 are both in the APS pixel mode, and each pixel matrix works independently. In this way, APS images and event images can be output simultaneously according to different matrices. That is, at this time, two images can also be output each time.

[0108] Furthermore, the setting of the pixel matrix is not limited to the above examples. The pixel matrix can also be set according to the region of interest. After setting the pixel matrix based on the region of interest, two images can be output simultaneously.

[0109] Among them, the number of pixels in pixel matrix 611, pixel matrix 614, pixel matrix 612, pixel matrix 613, pixel matrix 621, pixel matrix 622, pixel matrix 623, and pixel matrix 624 is four. Of course, the number in the pixel matrix can be set to other numbers as needed. When the number in the pixel matrix is 1, the pixel matrix is a single pixel.

[0110] In another embodiment of the present invention, as Figure 9 shown, the method further includes:

[0111] Step S710: Based on the signal control processor 500, control a second preset number of pixels to form a first pixel region;

[0112] Step S720: Based on the signal control processor 500, control a third preset number of pixels to form a second pixel region;

[0113] Among them, the second preset number and the third preset number are preset. By setting the second preset number and the third preset number, first pixel regions and second pixel regions of different sizes are realized.

[0114] The second preset number and the third preset number can be the same value or different values. That is, the sizes of the first pixel region and the second pixel region can be the same or different.

[0115] Step S730: Based on the signal control processor 500, set all pixels in the first pixel region as APS pixels and set all pixels in the second pixel region as event sensor pixels, or set all pixels in the first pixel region as event sensor pixels and set all pixels in the second pixel region as APS pixels;

[0116] In this step, by setting, it is possible to output one type of signal from the first pixel region and the second pixel region respectively. For example, output APS pixel signals through the pixels in the first pixel region, and output event sensor signals through the pixels in the second pixel region.

[0117] Step S740: Control, according to the signal control processor 500, to obtain the images output from the first pixel region and the second pixel region within a preset time period.

[0118] In this step, since the time for outputting images based on different pixel modes is different, by presetting the preset time period in advance, it is possible to obtain the images output from the first pixel region and the second pixel region within the preset time period. Furthermore, compared with the case of only the APS pixel mode, lower power can be achieved, and compared with only the event sensor pixel mode, more information can be obtained, thus meeting the usage requirements.

[0119] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A mode-switchable image sensor includes a plurality of pixels, and each pixel includes: A photoelectric conversion circuit for generating charges corresponding to incident light incident on a pixel; An APS pixel signal readout circuit is connected to the photoelectric conversion circuit and is used to generate an APS pixel signal corresponding to the charges generated by the photoelectric conversion circuit when the pixel is an APS pixel. It is characterized in that it further includes: a mode switching circuit connected to the photoelectric conversion circuit and used to set the pixel as an APS pixel or an event-type sensor pixel according to a mode switching control signal; an event signal readout circuit connected to the photoelectric conversion circuit and used to generate an event-type sensor signal corresponding to the charges generated by the photoelectric conversion circuit when the pixel is an event-type sensor pixel; the photoelectric conversion circuit includes a photoelectric conversion device, a transfer transistor, and a floating diffusion node; a signal control processor is connected to the photoelectric conversion circuit, the APS pixel signal readout circuit, the mode switching circuit, and the event signal readout circuit. The signal control processor is used to send a mode switching control signal to the mode switching circuit and generate a first image according to the APS pixel signal, or generate a second image according to the event-type sensor signal; the mode switching circuit includes a reset transistor; the source of the reset transistor is connected to the photoelectric conversion circuit, and the drain of the reset transistor is connected to the power supply; when the pixel is an APS pixel, the reset transistor resets the voltage of the floating diffusion node; when the pixel is an event-type sensor pixel, the reset transistor outputs a logarithmic voltage corresponding to the photocurrent corresponding to the charges generated by the photoelectric conversion device; the mode switching circuit further includes a first switch and a second switch; the signal control processor controls the first switch and the second switch to be turned on or off to set the pixel as an APS pixel or an event-type sensor pixel; when the first switch is closed and the second switch is open, the gate of the reset transistor is connected to the first switch, and the pixel is set as an APS pixel; when the mode switching control signal controls the first switch to be open and the second switch to be closed, the gate and the drain of the reset transistor are respectively connected to both ends of the second switch, and the pixel is set as an event-type sensor pixel; When the first image is a grayscale image or an RGB image, the signal control processor performs processing on the grayscale image or the RGB image including but not limited to color interpolation, color correction, automatic white balance, gamma correction, color saturation correction, formatting, bad pixel correction, and chromaticity correction; when the second image is an event image, the signal control processor corrects or calibrates the timestamp values of noise pixels, hot pixels, or dead pixels according to the time correlation between the timestamp values of adjacent pixels in the event image.

2. The mode-switchable image sensor according to claim 1, wherein The transfer transistor TX is used to transfer the charges generated by the photoelectric conversion device to the floating diffusion node, and the floating diffusion node is also connected to the APS pixel signal readout circuit, the reset transistor, and the event signal readout circuit.

3. The mode-switchable image sensor according to claim 2, wherein The event signal readout circuit includes a first driving transistor and a first readout unit. The first driving transistor is connected to the floating diffusion node, and the first readout unit is connected to the first driving transistor.

4. The mode-switchable image sensor according to claim 3, wherein The APS pixel signal readout circuit includes a second driving transistor, a selection transistor, and a second readout unit. The second driving transistor is connected to the floating diffusion node, the selection transistor is connected to the second driving transistor, and the second readout unit is connected to the selection transistor.

5. An image sensor control method, wherein The method is performed based on the mode-switchable image sensor according to any one of claims 1-4. Specifically, the method includes: Step S100: The signal control processor sends a mode switch control signal to the mode switch control circuit; Step S200: When the mode switch circuit sets the pixel as an APS pixel according to the mode switch control signal, the APS pixel signal readout circuit generates an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit; Step S300: When the mode switch circuit sets the pixel as an event sensor pixel according to the mode switch control signal, the event signal readout circuit generates an event sensor signal corresponding to the charge generated by the photoelectric conversion circuit; Step S400: The signal control processor generates a first image when acquiring the APS pixel signal, or generates a second image when acquiring the event sensor signal.

6. The image sensor control method according to claim 5, wherein The method further includes: Step S510: When the mode switch circuit sets the pixel as an APS pixel according to the mode switch control signal, the signal control processor controls the event signal readout circuit to generate an event sensor signal; Step S520: When the mode switch circuit sets the pixel as an event sensor pixel according to the mode switch control signal, the signal control processor controls the APS pixel signal readout circuit to generate an APS pixel signal corresponding to the charge generated by the photoelectric conversion circuit.

7. The image sensor control method according to claim 6, wherein The method further includes: Step S610: Based on the control of the signal control processor (500), a first preset number of pixels form a pixel unit, where the pixel unit includes a plurality of pixel matrices; Step S620: Based on the signal control processor (500), the pixel matrices on one opposite side of the pixel unit are set as pixels with the same mode, and the pixel matrices on the other opposite side are the same or different.

8. The image sensor control method according to claim 6, wherein The method further includes: Step S710: controlling, based on the signal control processor, a second preset number of pixels to form a first pixel region; Step S720: controlling, based on the signal control processor, a third preset number of pixels to form a second pixel region; Step S730: setting, based on the signal control processor, the pixels in the first pixel region to be all APS pixels and setting the pixels in the second pixel region to be all event sensor pixels, or setting the pixels in the first pixel region to be all event sensor pixels and setting the pixels in the second pixel region to be all APS pixels; Step S740: controlling, according to the signal control processor, to acquire images output from the first pixel region and the second pixel region within a preset time period.

Images