Image sensor control method, device and equipment and vehicle

Abstract

The invention discloses an image sensor control method and device, equipment and a vehicle. The image sensor includes a plurality of pixel units, each pixel unit includes a plurality of photodiodes, the method includes outputting an image signal based on a target pixel readout mode, a plurality of pixel readout modes are provided, at least one of exposure time, gain and number of enabled photodiodes of any two pixel readout modes is different, and the number of the photodiodes is different from that of the target pixel readout mode. The target pixel readout mode is at least one of a plurality of pixel readout modes. Through configuration of exposure time, gain and the number of enabled photodiodes, a pixel readout mode capable of adapting to different shooting scenes can be generated, so that the image sensor can output more images with different imaging effects, the dynamic range of a synthesized HDR image can be improved, the method can adapt to richer scenes, and the image quality is improved. Therefore, the imaging effect of the image sensor in various scenes is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of image sensor control, and in particular, relates to an image sensor control method, a control device, an image acquisition device and a vehicle. Background Art

[0002] With the development of vehicle intelligence, the use of camera function is becoming more and more frequent. In order to improve the shooting effect, various methods are used from the front end to the back end to achieve the required effect. For example, for the effect of HDR (High-Dynamic Range) images, the industry currently generally uses long and short double exposures to achieve high dynamic range images, but the effect is not ideal. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a control method, a control device, an image acquisition device and a vehicle for an image sensor.

[0004] In a first aspect, the present application provides a method for controlling an image sensor, wherein the image sensor comprises a plurality of pixel units, and the pixel units comprise a plurality of photodiodes. The method comprises outputting an image signal based on a target pixel readout mode, wherein the image sensor comprises a plurality of pixel readout modes, and any two of the pixel readout modes have at least one different exposure time, gain, and number of enabled photodiodes, and the target pixel readout mode is at least one of the plurality of pixel readout modes.

[0005] In a second aspect, the present application provides a control device for an image sensor, wherein the image sensor includes a plurality of pixel units, the pixel units include a plurality of photodiodes, the control device includes an output module, and the output module outputs an image signal based on a target pixel readout mode, wherein the image sensor includes a plurality of pixel readout modes, any two of the pixel readout modes have at least one different exposure time, gain, and number of enabled photodiodes, and the target pixel readout mode is at least one of the plurality of pixel readout modes.

[0006] In a third aspect, the present application provides an image acquisition device, comprising a control circuit and the above-mentioned image sensor, wherein the control circuit is used to execute the control method of the above-mentioned image sensor.

[0007] In a fourth aspect, the present application provides a vehicle, comprising the above-mentioned image acquisition device.

[0008] In the control method, control device, image acquisition equipment, and vehicle of the image sensor provided in the embodiments of the present application, multiple pixel readout modes are preset, and the configurations of any two pixel readout modes are different to achieve different imaging effects. The configuration of the pixel readout mode mainly includes three dimensions, namely, the exposure time of the pixel unit, the gain of the pixel unit, and the number of enabled photodiodes of the pixel unit. It can be understood that any difference in the exposure time, gain, and the number of enabled photodiodes will affect the imaging effect of the pixel unit. By configuring the exposure time, gain, and the number of enabled photodiodes, a pixel readout mode that can adapt to different shooting scenes can be generated, so that the image sensor can output more images with different imaging effects, which is beneficial to improving the dynamic range of the synthesized HDR image, and can adapt to richer scenes, thereby improving the imaging effect of the image sensor in various scenes.

[0009] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0011] Figure 1 is a schematic structural diagram of a pixel unit of an image sensor provided in an embodiment of the present application;

[0012] Figure 2 is a partial structural schematic diagram of an image sensor provided in an embodiment of the present application;

[0013] Figure 3 is a first flow chart of the control method of the image sensor provided by an embodiment of the present application;

[0014] Figure 4 is a second flow chart of the control method of the image sensor provided by an embodiment of the present application;

[0015] Figure 5 is a first control timing diagram of the image sensor provided by an embodiment of the present application;

[0016] Figure 6 is a second control timing diagram of the image sensor provided by an embodiment of the present application;

[0017] Figure 7 is a module schematic diagram of a control device provided in an embodiment of the present application;

[0018] Figure 8 It is a structural schematic diagram of the image acquisition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0020] For ease of understanding, the basic structure of the image sensor is introduced below:

[0021] Image sensors are electronic devices used to capture and convert light into electrical signals. They are the core components of imaging devices such as digital cameras, smartphone cameras, and surveillance cameras. The main function of an image sensor is to convert an optical image into a digital image, allowing the device to capture and process image data. There are currently two main types of image sensors: CCD (Charge-Coupled Device) and CMOS (Complementary Metal-Oxide-Semiconductor) image sensors.

[0022] The working principle of CCD image sensor is as follows: CCD sensor captures light through photosensitive elements (pixels), converts light intensity into electric charge, and then transfers the charge to the circuit for processing through charge transfer. The amount of charge in each pixel unit represents the intensity of light received by the pixel.

[0023] Working principle of CMOS image sensor: CMOS sensor integrates photodiode and amplifier in each pixel, so that each pixel can independently convert light intensity into electrical signal and read it out directly. This design enables CMOS sensor to amplify signal at each pixel, thus reducing noise in signal transmission process.

[0024] In some embodiments, the image sensor of the present application is a CMOS image sensor. CMOS image sensors have the following advantages:

[0025] Low power consumption: CMOS sensors generally consume less power than CCD sensors.

[0026] Fast readout: Since each pixel has its own amplifier, CMOS sensors have fast readout speed.

[0027] Multifunctional integration: CMOS sensors can more easily integrate other functions, such as image processing circuits.

[0028] In some embodiments, the image sensor includes:

[0029] Pixel array: A pixel array composed of multiple pixel units. Each pixel unit contains multiple photodiodes, which are used to convert light signals into electrical signals.

[0030] Pixel circuit: Each pixel unit also contains a series of circuit elements, such as amplifiers, readout circuits, and switches. These circuits are responsible for converting the current signal generated by the photodiode into a voltage signal and reading the signal out.

[0031] Row and column address logic: CMOS image sensors use row and column address logic to select the pixels that need to read out signals. By controlling the opening and closing of switches, pixels can be selected row by row for readout.

[0032] Analog signal processing circuit: including amplifiers, analog-to-digital converters (ADC), etc., used to amplify and digitize pixel signals.

[0033] Digital signal processing circuit: includes image signal processing (ISP) unit, which is used to perform noise reduction, color correction and other processing on digital signals.

[0034] Control circuit: controls the operation of the entire sensor, including exposure time and readout sequence, etc.

[0035] The following is a detailed introduction to the various components of the image sensor:

[0036] In some embodiments, Figure 1 In the embodiment shown, one pixel unit includes four photosensitive regions (ie, four photodiodes). The pixel unit may also be a pixel unit including a pair of split photodiodes.

[0037] See also Figure 1 , Figure 1The schematic diagram of the structure of a pixel unit of an image sensor, the transfer transistors M1, M2, M3, and M4 correspond to the transfer of photoelectrons generated by incident light in the photodiodes PD1, PD2, PD3, and PD4 to the floating diffusion node FD1. The control signals TX1, TX2, TX3, and TX4 are coupled to the gates of the corresponding transfer transistors M1, M2, M3, and M4, and the on and off of the transfer transistors are controlled by high and low levels. The reset transistor M5 is controlled by the reset control signal RST coupled to the gate of M5, and the potentials of FD1 and FD2 are reset to the power supply voltage VDDp during the reset phase. The gain control signal DCG is coupled to the gate of the transistor M6 to control the pixel circuit to output a high-gain or low-gain analog image signal. The RS pulse signal controls the signal of a certain row of pixels, and the output pixel signal is V_pixel.

[0038] like Figure 2 As shown, switch SW1 controls the analog pixel signal and the ramp signal to be transmitted to the analog-to-digital converter Column ADC1; switch SW2 controls the analog pixel signal to be transmitted to the analog-to-digital converter Column ADC2. AZ_CMPT1 and AZ_CMPB1 are reset signals of the analog-to-digital converters.

[0039] The three transfer transistors M1, M2, and M3 in the large pixel in the logic unit receive transfer control signals TX1, TX2, and TX3, and the small pixel transfer transistor M4 receives a transfer control signal TX4. The reset transistor M5 is coupled between the floating diffusion node FD2 and the power supply line VDDp to reset the pixel (for example, discharge or charge the floating diffusion node and the photodiode) under the control of the reset signal RST. The floating diffusion node FD1 is coupled to the gate of the source follower transistor M7 and controls the conduction of M7, and the selection transistor M8 is coupled between the source follower transistor M7 and the readout column line. Based on the floating diffusion node FD1 at the source follower transistor M7, the source follower transistor M7 and the selection transistor M8 selectively connect the output of the pixel circuit to the readout column line.

[0040] like Figure 2 The specific structure of the readout conversion circuit of the image sensor imaging system mainly includes a pixel structure, a switch and an analog-to-digital converter. The low-gain reset signal and the low-gain image signal of the pixel signal are transmitted to the analog-to-digital converter through the switch and compared with the low-gain ramp for conversion; the high-gain reset signal and the high-gain image signal are transmitted to the analog-to-digital converter through the switch and compared with the high-gain ramp for conversion ( Figure 2The pixel structure of the pixel signal and the corresponding readout conversion circuit are shown. Switch SW1 controls the analog pixel signal and the ramp signal to be transmitted to the analog-to-digital converter Column ADC1; switch SW2 controls the analog pixel signal to be transmitted to the analog-to-digital converter Column ADC2. AZ_CMPT1 and AZ_CMPB1 are reset signals of the upper and lower analog-to-digital converters Column ADC1 and Column ADC2, respectively.

[0041] The pixel array is a two-dimensional array of pixel structures, having X pixel columns and Y pixel rows. In one embodiment, each pixel is a CMOS (complementary metal oxide semiconductor) imaging pixel. The pixel array may be implemented as a back-illuminated image pixel array. As illustrated, each pixel cell is arranged into a row (e.g., rows R1-Ry) and a column (e.g., columns C1-Cx) to acquire image data of a person, place, or object, which may then be used to reproduce a 2D image of the person, place, or object.

[0042] The pixel array acquires image data, which is stored as an image charge within each pixel before readout until each pixel is selected to output its image voltage on its associated readout column. After each pixel has acquired its image data or image charge, the image data is read out by the top readout circuit and the bottom readout circuit, respectively, via the upper and lower column buses. The top readout circuit and the bottom readout circuit respectively include a column switching module and an analog-to-digital converter (ADC). The column switching module routes the image pixel signal on the column bus to the analog-to-digital converter.

[0043] like Figure 2 As shown, the pixel units from the column level are input to the upper and lower independent column exchange modules respectively: a column exchange module located at the top and a column exchange module located at the bottom. In the illustrated embodiment, the image data is read out from the column exchange module to obtain the image data on each column. The read-out image data is the analog voltage level on each read-out column. The read-out image data is then sequentially output to the analog-to-digital conversion circuit ADC.

[0044] The present application provides a method for controlling an image sensor. The following is a detailed description of the method for controlling an image sensor:

[0045] See also Figure 3 A control method for an image sensor provided in an embodiment of the present application is implemented by step 011, which is described in detail below.

[0046] Step 012: Output an image signal based on a target pixel readout mode, wherein the image sensor includes multiple pixel readout modes, and at least one of the exposure time, gain, and number of enabled photodiodes of any two pixel readout modes is different.

[0047] The pixel readout mode refers to the mode used by the pixel unit when reading out the image signal. When reading out the image signal in different pixel readout modes, at least one of the exposure time, gain, and the number of enabled photodiodes is different. In other words, between any two pixel readout modes, at least one of the exposure time, gain, and the number of enabled photodiodes is different.

[0048] In this way, through different configurations of exposure time, gain, and the number of enabled photodiodes, a large number of different pixel readout modes can be generated.

[0049] Alternatively, the target pixel readout mode may include one or more different pixel readout modes.

[0050] For example, the target pixel readout mode is any pixel readout mode, and when reading out the image signal of the pixel unit, the pixel can be controlled to output the image signal in any of the target pixel readout modes.

[0051] For another example, the target pixel readout mode includes multiple pixel readout modes. When reading the image signal of the pixel unit, the image signal corresponding to each pixel readout mode in the target pixel readout mode is output respectively.

[0052] The control method of the image sensor of the present application achieves different imaging effects by presetting multiple pixel readout modes, and the configurations of any two pixel readout modes are different. The configuration of the pixel readout mode mainly includes three dimensions, namely the exposure time of the pixel unit, the gain of the pixel unit and the number of enabled photodiodes of the pixel unit. It can be understood that any difference in the exposure time, gain and the number of enabled photodiodes will affect the imaging effect of the pixel unit. By configuring the exposure time, gain and the number of enabled photodiodes, a pixel readout mode that can adapt to different shooting scenes can be generated, so that the image sensor can output more images with different imaging effects, which is beneficial to improving the dynamic range of the synthesized HDR image, and can adapt to richer scenes, thereby improving the imaging effect of the image sensor in various scenes.

[0053] See also Figure 4 In some embodiments, the control method further comprises:

[0054] Step 011: Determine a target pixel readout mode based on the current shooting scene.

[0055] Among them, the current shooting scenes include two categories:

[0056] (1) Overexposed scenes, high-brightness scenes, medium-brightness scenes, or low-brightness scenes;

[0057] (2) High dynamic scene or low dynamic scene.

[0058] Specifically, when performing scene division, the scenes can be divided into overexposed scenes, high-brightness scenes, medium-brightness scenes and low-brightness scenes based on the intensity of the ambient light, and the ambient light intensity corresponding to the overexposed scenes, high-brightness scenes, medium-brightness scenes and low-brightness scenes gradually decreases.

[0059] It is understandable that more scenes can be divided based on the intensity of ambient light, which is not limited here.

[0060] When dividing the scenes, you can also distinguish between high-dynamic scenes and low-dynamic scenes based on the speed of scene change. For example, when the scene content changes quickly (such as shooting animals in motion, or the photographer is in high-speed motion, etc.), the scene is determined to be a high-dynamic scene. When the scene content changes slowly (such as shooting scenery, stationary objects, etc.), the scene is determined to be a low-dynamic scene.

[0061] It can be understood that more scenes can be divided based on the speed of scene change. For example, if the speed of scene change is moderate, the scene can be determined as a medium dynamic scene, which is not limited here.

[0062] After the scene is divided based on the imaging requirements of the image sensor, the current shooting scene can be determined based on the environmental information.

[0063] Optionally, based on the current ambient light intensity, it is determined whether the current shooting scene is an overexposed scene, a high-brightness scene, a medium-brightness scene, or a low-brightness scene.

[0064] For example, the image sensor can evaluate the ambient light intensity based on the amount of light entering per unit time (e.g., the amount of light entering per unit time is proportional to the ambient light intensity); or, the image acquisition device in which the image sensor is installed is provided with an ambient light sensor, which can detect the ambient light intensity. Based on the detected current ambient light intensity, it can be determined whether the current shooting scene is an overexposed scene, a high-brightness scene, a medium-brightness scene, or a low-brightness scene.

[0065] For another example, the image sensor can evaluate the ambient light intensity based on the amount of light entering per unit time, thereby determining whether the current shooting scene is an overexposed scene, a high-brightness scene, a medium-brightness scene, or a low-brightness scene.

[0066] Optionally, based on the current scene change speed, it is determined whether the current shooting scene is a high-dynamic scene or a low-dynamic scene.

[0067] For example, the scene change speed can be based on the number of different images in the continuous multiple frames of images captured within a unit time (such as 3 seconds). The larger the number, the faster the scene changes, and vice versa. Therefore, according to the current scene change speed, it is determined whether the current shooting scene is a high-dynamic scene or a low-dynamic scene.

[0068] The two images are different when the similarity between the two images is less than a preset similarity.

[0069] In some embodiments, the pixel readout mode includes a first mode, a second mode, a third mode, and a fourth mode;

[0070] Among them, the number of photodiodes enabled in the first mode, the second mode and the fourth mode (large pixels) is greater than the number of photodiodes enabled in the third mode (small pixels), the exposure time in the first mode, the second mode and the third mode (long exposure) is greater than the exposure time in the fourth mode (short exposure), and the gains of the second mode, the third mode and the fourth mode (low gain) are less than the gain of the first mode (high gain).

[0071] Based on the length of exposure time, it can be divided into two categories: long exposure and very short exposure.

[0072] Optionally, the exposure time corresponding to the long exposure is N times the exposure time corresponding to the very short exposure, where N is greater than 1. For example, N is any value in the interval [9, 11].

[0073] Based on the gain size, it can be divided into two categories: high gain and low gain.

[0074] The pixel unit can be divided into a large pixel or a small pixel based on the number of enabled photodiodes. The number of enabled photodiodes in the large pixel mode is greater than the number of enabled photodiodes in the small pixel mode.

[0075] Optionally, the pixel unit includes 4 photodiodes, the number of photodiodes enabled in the large pixel mode is 4, and the number of photodiodes enabled in the small pixel mode is 1, 2 or 3;

[0076] Alternatively, the number of photodiodes enabled in the large pixel mode is 3, and the number of photodiodes enabled in the small pixel mode is 1 or 2;

[0077] Alternatively, the number of photodiodes enabled in the large pixel mode is 2, and the number of photodiodes enabled in the small pixel mode is 1.

[0078] In this way, by setting the number of photodiodes enabled in the large pixel mode to be greater than the number of photodiodes enabled in the small pixel mode, it is possible to adapt to scenes with different ambient light intensities.

[0079] It can be understood that in order to reduce circuit complexity, the sum of the number of photodiodes enabled by the large pixel mode and the number of photodiodes enabled by the small pixel mode is the total number of photodiodes, thereby realizing branch control, simplifying circuit design, and reducing the difficulty of circuit control. For example, the number of photodiodes enabled by the large pixel mode and the small pixel mode is 3 and 1 respectively.

[0080] Optionally, in order to further reduce the amount of light entering the photodiode enabled in the small pixel mode, the photodiode may be covered with an attenuation film to reduce the amount of light entering the photodiode.

[0081] That is to say, the first mode is a pixel readout mode with large pixels, high gain and long exposure; the second mode is a pixel readout mode with large pixels, low gain and long exposure; the third mode is a pixel readout mode with small pixels, low gain and long exposure; the fourth mode is a pixel readout mode with large pixels, low gain and very short exposure.

[0082] Optionally, the relationship between the target pixel readout mode and the current shooting scene is as follows:

[0083] When the target pixel readout mode is the first mode, a good imaging effect can be achieved in low-brightness and / or low-dynamic scenes.

[0084] When the target pixel readout mode is the second mode, in a medium-brightness and / or low-dynamic scene, not only a better imaging effect is achieved, but also noise caused by gain is avoided.

[0085] When the target pixel readout mode is the third mode, the overexposure problem can be avoided in high-brightness and / or low-dynamic scenes, which not only has a better imaging effect but also avoids the noise caused by gain.

[0086] When the target pixel readout mode is the fourth mode, when the image signal is read out in an overexposure and / or high dynamic scene, the artifacts of the image generated by the image signal can be reduced, and when the exposure time is very short, the exposure time can be shortened, thereby reducing power consumption.

[0087] Optionally, the target pixel readout mode includes a first mode, a second mode, a third mode and a fourth mode, and the corresponding image signals are read out in the first mode, the second mode, the third mode and the fourth mode in sequence to generate an HDR image. In this way, the HDR image is synthesized through the image signals of the above four modes, thereby improving the imaging effect while extending the dynamic range.

[0088] In some embodiments, different pixel readout modes correspond to different control timings.

[0089] The photodiode is controlled by the corresponding enable signal. When the enable signal is at a high level, the photodiode is enabled;

[0090] The gain of the image signal is controlled by a corresponding gain control signal. When the gain control signal is at a low level, the image signal corresponds to a high gain.

[0091] The exposure time of the pixel unit is controlled by the enable signal and the reset signal of the corresponding photodiode, and the exposure time of the pixel unit is determined based on the interval between the enable signal and the reset signal;

[0092] In the first mode, the enable signals of the first number of photodiodes are kept at a high level for a preset time period, the gain control signal is at a low level, and the exposure time is a first time period;

[0093] In the second mode, the enable signals of the first number of photodiodes are at a high level, the gain control signal is at a high level, and the exposure time is a first duration;

[0094] In the third mode, the enable signal of each photodiode of the second number is high level, the gain control signal is high level, the exposure time is the second duration, the second number is less than the first number, and the second duration is greater than the first duration;

[0095] In the fourth mode, the enable signals of the first number of photodiodes are at a high level, the gain control signal is at a high level, and the exposure time is a third duration, which is shorter than the first duration.

[0096] Specifically, please combine Figure 2 and Figure 5 , Figure 5 The target pixel readout mode includes a control timing when the first mode to the fourth mode are selected.

[0097] First, each signal in the control timing is introduced:

[0098] The RS signal is a row selection control signal, and the output pixel signal is V_pixel.

[0099] RST is a reset signal. The reset transistor M5 is controlled by the reset control signal RST coupled to the gate of M5 to reset the potential of each floating diffusion node to the power supply voltage VDDp during the reset phase.

[0100] The DCG signal is a gain control signal, which is coupled to the gate of the transistor M6 to control the pixel circuit to output an analog image signal of high gain (corresponding to a low level) or low gain (corresponding to a high level).

[0101] The control signals TX1, TX2, TX3, and TX4 are coupled to the gates of the corresponding transmission transistors M1, M2, M3, and M4, and the on and off of the analog switch tubes are controlled by high and low levels.

[0102] SW1 is a switch control signal of the switch SW1 , and SW1 controls the analog pixel signal and the ramp signal to be transmitted to the analog-to-digital converter Column ADC1 .

[0103] SW2 is a switch control signal of the switch SW2, and SW2 controls the analog pixel signal to be transmitted to the analog-to-digital converter ColumnADC1.

[0104] AZ_CMPT1 and AZ_CMPB1 are reset signals of the upper and lower analog-to-digital converters Column ADC1 and Column ADC2 respectively.

[0105] Please combine Figure 6 , since the exposure stage is located before the pixel readout mode, the exposure time is explained below:

[0106] Figure 6 , ROWA represents the row address switching control signal; RST and TX are Figure 5 READOUT represents a control signal for reading pixel signals, and when it is high, it is read from the pixel unit to the analog-to-digital converter. Figure 6 The two high levels represent that the pixel unit outputs the pixel reset signal and the pixel image signal in sequence.

[0107] Among them, time a to b is the exposure time of the nth pixel unit, and the analog pixel signal of the (na)th pixel unit is read out after time b. The row address switching operation is performed at time b, and the exposure readout row address selection information is only controlled by the signal ROWA.

[0108] In this way, the exposure phase EXP and the read phase READ are controlled by the same address line (corresponding to the control signal ROWA), which is more concise and can save area and power consumption.

[0109] Among them, for Figure 5 The LPDL at the bottom indicates that the camera is currently in the second mode of large pixels, low gain and long exposure. The LPDH indicates that the camera is currently in the first mode of large pixels, high gain and long exposure. The SPD indicates that the camera is currently in the third mode of small pixels, low gain and long exposure. The LPDS indicates that the camera is currently in the fourth mode of large pixels, low gain and very short exposure.

[0110] In the first mode LPDH, the enable signals of the first number of photodiodes (i.e., TX1, TX2, and TX3) are set to maintain a high level within a preset time length, the gain control signal is a low level, and based on the interval between the reset signal RST and TX1, TX2, and TX3, the exposure time is determined to be the first time length.

[0111] In the second mode LPDL, the enable signals of the first number of photodiodes (i.e., TX1, TX2, and TX3) are set to a high level, the gain control signal is set to a high level, and the exposure time is determined to be a first duration based on the interval between the reset signal RST and TX1, TX2, and TX3;

[0112] In the third mode SPD, the enable signal of each photodiode of the second number (ie, TX4) is set to a high level, the gain control signal is set to a high level, and the exposure time is determined to be a second duration based on the interval between the reset signal RST and TX4. The second duration is greater than the first duration (ie, in the small pixel mode, the exposure duration can be appropriately increased), and the second number is less than the first number;

[0113] In the fourth mode LPDS, the enable signals of the first number of photodiodes (i.e., TX1, TX2, and TX3) are set to a high level, the gain control signal is set to a high level, and based on the interval between the reset signal RST and TX1, TX2, and TX3, the exposure time is determined to be a third duration, which is less than the first duration.

[0114] In this way, by controlling each control signal of the control timing, the pixel readout mode of the pixel unit at each moment can be accurately controlled, thereby achieving accurate control of the pixel readout mode.

[0115] In some embodiments, the image sensor also includes a first analog-to-digital converter and a second analog-to-digital converter, and the first analog-to-digital converter and the second analog-to-digital converter are both connected to the pixel unit to collect the image signal output by the pixel unit. The first analog-to-digital converter and the second analog-to-digital converter cooperate to make full use of the two analog-to-digital converters to improve the pixel readout efficiency, thereby reading out four frames of images with different gains through the four readout modes of the pixel unit, and synthesizing the four frames of images into a high dynamic range image.

[0116] Specifically, see Figure 5 It can be seen from the control timing diagram of a single pixel unit that the first analog-to-digital converter ADC1 and the second analog-to-digital converter ADC2 do not work at the same time, while the first analog-to-digital converter ADC1 and the second analog-to-digital converter ADC2 can sample different pixel units at the same time. For the same pixel unit, four frames of images with different gains can be read out through the four readout modes of the pixel unit, and the four frames of images can be synthesized into an image with a high dynamic range.

[0117] In this way, through reasonable control timing design and two analog-to-digital converters working in parallel, the frame rate of the image sensor can be improved, so that the frame rate of the image sensor can be greater than or equal to 60FPS (Frames Per Second).

[0118] Please continue reading Figure 2and Figure 5 , the following takes a pixel unit including 4 photodiodes and using 2 ADCs for sampling as an example to illustrate the readout process of the image signal of the present application:

[0119] like Figure 5 As shown, the large pixel long exposure signal, the small pixel long exposure signal and the large pixel very short exposure signal are read out sequentially in time. The readout control timing of the three pixel signals can be flexibly applied according to actual needs. For example, the exposure readout of the small pixel signal may not be performed under normal or darker lighting conditions; the large pixel very short exposure signal may not be read out when shooting non-moving objects or when the motion distortion and artifacts related to flicker in the image do not significantly affect the image quality.

[0120] At time t0, the reset signal RST is high, the transmission control signals TX1, TX2, and TX3 are pulse high, the gain control signal DCG is high, and the pixel circuit is in the reset stage. At this time, the charge storage node in the pixel unit can be reset to the power supply voltage level (such as VDDp), the selection signal changes from low level to high level, and the pixel unit outputs the reset signal in the second mode. The switch SW_1 is turned on, and the pixel reset signal is input to the left plate of the sampling capacitor of the analog-to-digital converter Column ADC1. The analog-to-digital converter reset signal AZ_CMPT1 resets the comparator input terminal, that is, the right plate of the sampling capacitor, to a fixed voltage value. The ramp signal V_ramp rises a fixed voltage value at a certain moment and then slopes down. When the voltage at the comparator input terminal is equal, the comparator output voltage changes from a high potential to a low potential, and the counter stops counting. At this time, the large pixel low gain reset signal conversion quantization of the pixel ends.

[0121] At time t1, the gain control signal DCG changes from high level to low level, and the pixel unit outputs the reset signal in the first mode; the switch SW_1 is disconnected, SW_2 is turned on, and the pixel reset signal is input to the left plate of the sampling capacitor of the analog-to-digital converter Column ADC2. The analog-to-digital converter reset signal AZ_CMPB1 resets the comparator input terminal, i.e., the right plate of the sampling capacitor, to a fixed voltage value. The ramp signal V_ramp rises to a fixed voltage value at a certain moment and then slopes down. When the voltage at the comparator input terminal is equal, the comparator output voltage changes from a high potential to a low potential, and the counter stops counting. At this time, the conversion quantization of the large pixel high gain reset signal of the pixel ends.

[0122] At t2, the transmission control signals TX1, TX2, and TX3 change from low level to high level, and the three transmission transistors M1, M2, and M3 in the large pixel are turned on, and the large pixel high gain image signal is output to the analog-to-digital converter Column ADC2 (the conversion is completed in the column-level analog-to-digital converter below). At t3, the ramp signal begins to decrease. When the pixel image signal is equal to the ramp signal, that is, when the voltages at the two input terminals of the comparator are equal, the comparator output signal changes from high level to low level, and the counter stops counting and subtracts from its reset count value to obtain the difference between the image signal and the reset signal under high conversion gain, completing the signal conversion and quantization, that is, the image quantization value of the correlated double sampling under high conversion gain of the large pixel.

[0123] At time t4, the gain conversion signal changes from low level to high level, the transmission control signals TX1, TX2, and TX3 change from low level to high level, and the three transmission transistors M1, M2, and M3 in the large pixel are turned on. At this time, the pixel unit outputs the image signal in the second mode; the switch SW_2 is closed, SW_1 is turned on, and the pixel image signal is input to the left plate of the sampling capacitor of the analog-to-digital converter Column ADC1. The analog-to-digital converter image signal AZ_CMPT1 resets the comparator input terminal, i.e., the right plate of the sampling capacitor, to a fixed voltage value. The ramp signal V_ramp rises a fixed voltage value at a certain moment and then slopes down. When the voltage at the comparator input terminal is equal, the comparator output voltage changes from a high potential to a low potential, the counter stops counting, and the difference is made with its reset count value to obtain the difference between the image signal and the reset signal under low conversion gain, completing the signal conversion and quantization, i.e., the image quantization value of the correlated double sampling under low conversion gain of the large pixel.

[0124] Next, at time t5, the third mode is entered, the row selection signal RS changes from high level to low level, the reset signal RST changes from low level to high level, the small pixel transfer transistor control signal TX4 inputs a pulse high level, and the pixel circuit is in the reset stage. At this time, the charge storage node in the pixel unit is reset to the power supply voltage level (such as VDDp). Subsequently, the row selection signal changes from low level to high level, and at time t6, RST changes from high level to low level, the switch SW_2 is turned on, and the pixel reset signal is input to the left plate of the sampling capacitor of the analog-to-digital converter Column ADC2. The analog-to-digital converter reset signal AZ_CMPB1 resets the comparator input terminal, that is, the right plate of the sampling capacitor, to a fixed voltage value. After the ramp signal V_ramp rises by a fixed voltage value, AZ_CMPB1 is disconnected, and the ramp begins to decrease. When the voltage at the comparator input terminal is equal, the comparator output voltage changes from high potential to low potential, and the counter stops counting and stores its count value. At this time, the small pixel reset signal conversion quantization of the pixel is completed.

[0125] The control signal TX4 input pulse of the small pixel signal transmission transistor is high level, and the transmission transistor M4 in the small pixel is turned on, and the small pixel image signal is output to the analog-to-digital converter Column ADC2 (the conversion is completed in the column-level analog-to-digital converter below). At time t7, the ramp signal V_ramp rises to a fixed voltage value and then slopes down. When the pixel signal is equal to the ramp signal, that is, when the voltages at the two input terminals of the comparator are equal, the comparator output signal changes from high level to low level, and the counter stops counting and stores its count value, obtaining the difference between the image signal of the small pixel and the reset signal, completing the signal conversion and quantization, that is, the image quantization value of the correlated double sampling of the small pixel.

[0126] Further, at time t8, the image signal of the large pixel with very short exposure is read out (i.e., entering the fourth mode), the reset signal RST changes from low level to high level, the transmission control signals TX1, TX2, and TX3 change from low level to high level, and the three transmission transistors M1, M2, and M3 in the large pixel are turned on. At this time, the pixel unit outputs the reset signal of the large pixel with very short exposure; the switch SW_2 is closed, SW_1 is turned on, and the pixel signal is input to the left plate of the sampling capacitor of the analog-to-digital converter Column ADC1. The analog-to-digital converter image signal AZ_CMPT1 resets the comparator input terminal, i.e., the right plate of the sampling capacitor, to a fixed voltage value. At time t9, the reset signal RST changes from high level to low level, and after the ramp signal V_ramp rises by a fixed voltage value, AZ_CMPB1 is disconnected and the ramp decreases. When the voltages at the comparator input terminals are equal, the comparator output voltage changes from high potential to low potential, and the counter stops counting and stores its count value, thereby obtaining the reset signal of the large pixel with very short exposure.

[0127] Subsequently, the transmission control signals TX1, TX2, and TX3 change from low level to high level, and the three transmission transistors M1, M2, and M3 in the large pixel are turned on, and the large pixel image signal is output to the analog-to-digital converter Column ADC1 (the conversion is completed in the column-level analog-to-digital converter above). At t10, the ramp signal begins to decrease. When the pixel image signal is equal to the ramp signal, that is, when the voltages at the two input terminals of the comparator are equal, the comparator output signal changes from high level to low level, and the counter stops counting and stores its count value, obtaining the difference between the image signal of the very short exposure large pixel and the reset signal, completing the signal conversion and quantization, that is, the image quantization value of the correlated double sampling of the very short exposure large pixel.

[0128] In this way, each image signal in the first mode to the fourth mode can be quickly acquired.

[0129] According to the method described in the above embodiment, the present application embodiment also provides a control device 300, which is used to execute the steps in the above image sensor control method. Figure 7 , Figure 7 : is a schematic diagram of the structure of a control device 300 provided in an embodiment of the present application. The control device 300 includes:

[0130] A determination module 301 determines a target pixel readout mode based on a current shooting scene;

[0131] The output module 302 outputs an image signal based on a target pixel readout mode, wherein the pixel readout modes of the image sensor include multiple pixel readout modes, at least one of the exposure time, gain, and number of enabled photodiodes of any two pixel readout modes is different, and the target pixel readout mode is at least one of the multiple pixel readout modes.

[0132] It should be noted that the specific details of each module unit in the above-mentioned control device 300 have been described in detail in the embodiment of the above-mentioned image sensor control method, and will not be repeated here.

[0133] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0134] In some embodiments, the control device in the embodiments of the present application can be implemented in hardware, such as an image acquisition device, or a component in the image acquisition device, such as an integrated circuit or a chip; the control device can also be implemented in software, such as as an application installed in the image acquisition device.

[0135] In some embodiments, see Figure 8 , Figure 8 4 is a schematic diagram of the structure of the image acquisition device provided in the embodiment of the present application. The image acquisition device 400 includes a control circuit 401 and an image sensor 403. The control circuit 401 is used to implement each process of the embodiment of the control method of the above-mentioned image sensor 403, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0136] Optionally, the control circuit 401 may be implemented by a digital circuit.

[0137] In some embodiments, the vehicle includes the image acquisition device of any of the above embodiments, that is, the image acquisition device can be arranged on the vehicle, such as on the body of the vehicle, inside the vehicle, etc.

[0138] Among them, the vehicles can be electric vehicles, gasoline vehicles, hybrid vehicles, extended-range vehicles, etc.

[0139] It is understandable that in the specific implementation of this application, data related to user identity or characteristics is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0140] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A method for controlling an image sensor, characterized in that: The image sensor includes a plurality of pixel units, each pixel unit includes a plurality of photodiodes, and the method includes: An image signal is output based on a target pixel readout mode, wherein the image sensor includes multiple pixel readout modes, any two of the pixel readout modes have at least one different exposure time, gain, and number of enabled photodiodes, and the target pixel readout mode is at least one of the multiple pixel readout modes.

2. The method for controlling an image sensor according to claim 1, wherein: Also includes: Determining the target pixel readout mode based on a current shooting scene, where the current shooting scene includes an overexposed scene, a high-brightness scene, a medium-brightness scene, or a low-brightness scene; and / or high-dynamic scenes or low-dynamic scenes.

3. The control method of the image sensor according to claim 1 or 2, characterized in that: The pixel readout modes include a first mode, a second mode, a third mode, and a fourth mode.

4. The method for controlling an image sensor according to claim 3, wherein: The number of the photodiodes enabled in the first mode, the second mode, and the fourth mode is greater than the number of the photodiodes enabled in the third mode; and / or the exposure time in the first mode, the second mode, and the third mode is greater than the exposure time in the fourth mode; And / or, the gains of the second mode, the third mode and the fourth mode are all smaller than the gain of the first mode.

5. The method for controlling an image sensor according to claim 4, characterized in that: The step of outputting an image signal based on a target pixel readout mode includes: Based on the first mode, the second mode, the third mode and the fourth mode in sequence, the corresponding image signal is output.

6. The method for controlling an image sensor according to any one of claims 3 to 5, characterized in that: The pixel unit includes 4 photodiodes, the number of the photodiodes enabled in the first mode, the second mode and the fourth mode is 4, and the number of the photodiodes enabled in the third mode is 1, 2 or 3; or, The number of the photodiodes enabled in the first mode, the second mode and the fourth mode is 3, and the number of the photodiodes enabled in the third mode is 1 or 2; or, The number of the photodiodes enabled in the first mode, the second mode, and the fourth mode is 2, and the number of the photodiodes enabled in the third mode is 1.

7. The method for controlling an image sensor according to any one of claims 3 to 5, characterized in that: The exposure time in the first mode, the second mode and the third mode is N times the exposure time in the fourth mode, where N is greater than 1.

8. The method for controlling an image sensor according to claim 3 or 4, characterized in that: In an overexposure and / or high dynamic scene, determining that the target pixel readout mode includes a fourth mode; In a high-brightness and / or low-dynamic scene, determining that the target pixel readout mode includes a third mode; In a medium-brightness and / or low-dynamic scene, determining that the target pixel readout mode includes a second mode; In a low-brightness and / or low-dynamic scene, determining the target pixel readout mode includes a first mode.

9. The method for controlling an image sensor according to claim 3, characterized in that: The target pixel readout mode includes a first mode, a second mode, a third mode and a fourth mode, and outputting an image signal based on the target pixel readout mode includes: In each of the pixel readout modes in the target pixel readout mode, the image signals corresponding to each of the pixel readout modes in the target pixel readout mode are output respectively.

10. The method for controlling an image sensor according to any one of claims 3 to 9, characterized in that: The photodiode is controlled by a corresponding enable signal, and when the enable signal is at a high level, the photodiode is enabled; the gain of the image signal is controlled by a corresponding gain control signal, and when the gain control signal is at a low level, the image signal corresponds to a high gain; the exposure time of the pixel unit is controlled by the corresponding enable signal and reset signal of the photodiode, and the exposure time of the pixel unit is determined based on the interval between the enable signal and the reset signal.

11. The method for controlling an image sensor according to claim 10, characterized in that: In the first mode, the enable signals of the first number of the photodiodes maintain a high level within a preset time period, the gain control signal is a low level, and the exposure time is a first time period.

12. The method for controlling an image sensor according to claim 10, characterized in that: In the second mode, the enable signals of the first number of the photodiodes are at a high level, the gain control signal is at a high level, and the exposure time is a first duration.

13. The method for controlling an image sensor according to claim 10, characterized in that: In the third mode, the enable signals of a second number of the photodiodes are high, the gain control signal is high, and the exposure time is a second duration, the second number is smaller than the first number of the photodiodes enabled in the first mode, and the second duration is greater than the first exposure duration in the first mode.

14. The method for controlling an image sensor according to claim 10, characterized in that: In the fourth mode, the enable signals of the first number of photodiodes are high level, the gain control signal is high level, and the exposure time is a third duration, which is shorter than the first exposure duration in the first mode.

15. The method for controlling an image sensor according to any one of claims 1 to 14, characterized in that: The image sensor further includes a first analog-to-digital converter and a second analog-to-digital converter, wherein the first analog-to-digital converter and the second analog-to-digital converter are both connected to the pixel unit to collect the image signal output by the pixel unit.

16. A control device for an image sensor, characterized in that: The image sensor includes a plurality of pixel units, each pixel unit includes a plurality of photodiodes, and the device includes: An output module is used to output an image signal based on a target pixel readout mode, wherein the pixel readout modes of the image sensor include multiple pixel readout modes, at least one of the exposure time, gain and the number of enabled photodiodes of any two of the pixel readout modes is different, and the target pixel readout mode is at least one of the multiple pixel readout modes.

17. An image acquisition device, comprising an image sensor and a control circuit, wherein the control circuit is used to execute the image sensor control method according to any one of claims 1 to 15.

18. A vehicle, characterized in that: Includes the image acquisition device as described in claim 17.

Images