Image sensor, electronic equipment and image processing method

Abstract

The invention discloses an image sensor, electronic equipment and an image processing method, and relates to the technical field of image processing. The image sensor provided by the invention comprises a pixel circuit, an image processing unit and a time sequence control unit, the pixel circuit is respectively connected with the image processing unit and the time sequence control unit, and the image processing unit is connected with the time sequence control unit; the pixel circuit comprises a photoelectric conversion module and a capacitor module, the photoelectric conversion module comprises a first switch and a second switch, and the capacitor module comprises a change-over switch; the time sequence control unit is connected with the first switch, the second switch and the change-over switch. Under the condition that the first switch, the second switch and the change-over switch are all in a conducting state, first pixel data output by the pixel circuit is used for generating a first image; under the condition that the first switch and the change-over switch are in the on state and the second switch is in the off state, the second pixel data output by the pixel circuit is used for focusing a first shot object in the first image.

Description

Technical Field

[0001] This application belongs to the technical field of image processing, and particularly relates to an image sensor, an electronic device, and an image processing method. Background Art

[0002] In order to obtain a High Dynamic Range (HDR) image, in related technologies, when taking pictures using terminals such as mobile phones, a Dual Conversion Gain (DCG) pixel circuit is generally adopted in an image sensor.

[0003] The DCG pixel circuit has two conversion gains and generally can output three types of pixel data. The first type of pixel data can be used to determine a first image in a first conversion gain mode, and the first image can reflect the content of the highlight part. The second type of pixel data can be used to determine a second image in a second conversion gain mode, and the second image can reflect the content of the shadow part. After the first image and the second image are subjected to image fusion, an HDR image can be obtained, and the third type of pixel data can be used to focus on an object in the second image to implement the Phase Detection Auto Focus (PDAF) function in the second conversion gain mode.

[0004] However, after obtaining the above three types of pixel data, related technologies still have difficulty in focusing on the photographed object in the first image, resulting in a blurred image of the photographed object in the first image, thereby affecting the image quality of the first image. Summary of the Invention

[0005] This application provides an image sensor, an electronic device, and an image processing method, which can generate a first image using first pixel data and effectively focus on a first photographed object in the first image, so that the image of the first photographed object in the first image is clear, thereby improving the image quality of the first image.

[0006] In a first aspect, an embodiment of this application proposes an image sensor, including: A pixel circuit, an image processing unit, and a timing control unit; the pixel circuit is respectively connected to the image processing unit and the timing control unit, and the image processing unit is connected to the timing control unit; The pixel circuit includes a photoelectric conversion module and a capacitance module, the photoelectric conversion module includes a first switch and a second switch, and the capacitance module includes a switching switch; the timing control unit is respectively connected to the first switch, the second switch, and the switching switch; In the case where the first switch, the second switch, and the switching switch are all in a conducting state, the first pixel data output by the pixel circuit is used to generate a first image; When the first switch and the switching switch are in the conducting state and the second switch is in the off state, the second pixel data output by the pixel circuit is used to focus on the first object to be photographed in the first image.

[0007] Optionally, in the image sensor provided in the embodiment of the present application, the timing control unit includes a first timing controller and a second timing controller; the image processing unit is connected to the first timing controller, and the first timing controller is connected to the switching switch; the second timing controller is respectively connected to the first switch and the second switch.

[0008] Optionally, in the image sensor provided in the embodiment of the present application, the first time point is the same as the second time point, and the third time point is later than the second time point; wherein, the first time point is the starting time point when the first timing controller controls the switching switch to conduct, the second time point is the starting time point when the second timing controller controls the first switch to conduct, and the third time point is the starting time point when the second timing controller controls the second switch to conduct.

[0009] Optionally, in the image sensor provided in the embodiment of the present application, the second timing controller is connected to the switching switch; the fourth time point is later than the third time point; wherein, the fourth time point is the starting time point when the second timing controller controls the switching switch to conduct.

[0010] Optionally, in the image sensor provided in the embodiment of the present application, the control priority of the first timing controller over the switching switch is higher than that of the second timing controller over the switching switch; wherein, the first control instruction output by the first timing controller to the switching switch is used to overwrite the second control instruction output by the second timing controller to the switching switch.

[0011] Optionally, in the image sensor provided in the embodiment of the present application, the image sensor further includes an analog-to-digital converter, a first end of the analog-to-digital converter is connected to the pixel circuit, a second end of the analog-to-digital converter is connected to the image processing unit, and a third end of the analog-to-digital converter is connected to the first timing controller.

[0012] Optionally, in the image sensor provided in the embodiment of the present application, when the first switch and the second switch are in the conducting state and the switching switch is in the off state, the third pixel data output by the pixel circuit is used to generate a second image; When the first switch is in the conducting state and the second switch and the switching switch are in the off state, the fourth pixel data output by the pixel circuit is used to focus on the second object to be photographed in the second image.

[0013] Optionally, in the image sensor provided in the embodiments of the present application, the pixel circuit has a first reading mode and a second reading mode; the first reading mode is a mode in which the switching switch is controlled by a first timing controller; the second reading mode is a mode in which the switching switch is controlled by a second timing controller; wherein, the pixel circuit is used to output first pixel data and second pixel data in the first reading mode; the pixel circuit is used to output first pixel data, third pixel data and fourth pixel data in the second reading mode.

[0014] Optionally, in the image sensor provided in the embodiments of the present application, when the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, the image processing unit is used to control the pixel circuit to switch from the second reading mode to the first reading mode.

[0015] Optionally, in the image sensor provided in the embodiments of the present application, when the switching switch is in the on state, the image sensor has a first conversion gain, and when the switching switch is in the off state, the image sensor has a second conversion gain, and the first conversion gain is less than the second conversion gain; the first conversion gain of the image sensor is used for the pixel circuit to output first pixel data and second pixel data, and the second conversion gain of the image sensor is used for the pixel circuit to output third pixel data and fourth pixel data.

[0016] In a second aspect, an embodiment of the present application proposes an electronic device, including a processor and the image sensor described in the first aspect.

[0017] In a third aspect, an embodiment of the present application proposes an image processing method, which is executed by the electronic device described in the second aspect. The image processing method includes: Obtain first pixel data and second pixel data output by the pixel circuit; Calculate the difference between the voltage signal value indicated by the first pixel data and the voltage signal value indicated by the second pixel data; Generate a first image based on the first pixel data; Perform focusing on a first photographed object in the first image based on the difference and the second pixel data.

[0018] Optionally, in the image processing method provided in the embodiments of the present application, before obtaining the first pixel data and the second pixel data output by the pixel circuit, the image processing method further includes: Obtain third pixel data output by the pixel circuit in the second reading mode; When the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, switch the pixel circuit from the second reading mode to the first reading mode; wherein, the first pixel data and the second pixel data are obtained in the first reading mode.

[0019] In an embodiment of the present application, since the pixel circuit in the image sensor can output first pixel data for generating a first image and second pixel data for focusing on a first photographed object in the first image, the first image can be generated using the first pixel data, and the first photographed object in the first image can be effectively focused using the second pixel data, making the picture of the first photographed object in the first image clear, thereby improving the picture quality of the first image. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic diagram of an image sensor provided by some embodiments of the present application; Figure 2 is a schematic diagram of an image sensor provided by some embodiments of the present application; Figure 3A is a schematic diagram of an image sensor provided by some embodiments of the present application; Figure 3B is a schematic diagram of a color filter array in an image sensor provided by some embodiments of the present application; Figure 4 is a schematic diagram of a camera module provided by some embodiments of the present application; Figure 5 is a schematic diagram of an electronic device provided by some embodiments of the present application; Figure 6 is a schematic flowchart of an image processing method provided by some embodiments of the present application; Figure 7 is a schematic flowchart of an image processing method provided by some embodiments of the present application.

[0021] DESCRIPTION OF REFERENCE NUMERALS: 10 - Image sensor; 100 - Pixel circuit; 101 - Photoelectric conversion module; 1011 - First photoelectric conversion element; PD1 - First photodiode; PD2 - Second photodiode; 1012 - Second photoelectric conversion element; PD3 - Third photodiode; PD4 - Fourth photodiode; 1013 - First switch; TG1 - First transfer transistor; TG2 - Second transfer transistor; 1014 - Second switch; TG3 - Third transfer transistor; TG4 - Fourth transfer transistor; 102 - Capacitance module; 1021 - First capacitor; 1022 - Second capacitor; 1023 - Switching switch; 103 - Output module; 1031 - Source follower; 1032 - Column select signal switch; 1033 - Reference power supply; 104 - Reset switch; VDD1 - First power supply voltage terminal; VDD2 - Second power supply voltage terminal; Vout - Output terminal of the pixel circuit; 200 - Image processing unit; 300 - Timing control unit; 301 - First timing controller; 302 - Second timing controller; 400 - Analog - to - digital converter; 500 - Color filter array; 510 - Filtering unit; 511 - First filtering element; 512 - Second filtering element; 40 - Camera module; 41 - Lens; 42 - Voice coil motor; 43 - Infrared filter; 44 - Image sensor; 45 - Connecting component; 46 - Base; 47 - Circuit board; 50 - Electronic device; 501 - Processor; 502 - Image sensor. Detailed implementation manners

[0022] Embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the drawings below are exemplary and are only used to explain the present application, and should not be construed as a limitation of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0023] The terms "first" and "second" in the description and claims of the present application may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality of" is two or more. In addition, "and / or" in the description and claims means at least one of the connected objects. The character " / ", generally represents an "or" relationship between the associated objects before and after.

[0024] In the description of the present application, it should be understood that the orientation or positional relationship indicated by terms such as "inner" and "outer" is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the present application.

[0025] In the description of the present application, it should be noted that unless otherwise clearly specified and limited, the terms "connected" and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0026] The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, rather than being intended to limit the present application. The terms related to the embodiments of the present application are explained below.

[0027] Image sensor: A device that converts an optical signal into an electrical signal and generates an image based on the electrical signal. The image sensor may include a pixel array, a color filter array, and a microlens array stacked in sequence. The pixel array may include a plurality of pixel circuits, and each pixel circuit may include a plurality of photoelectric conversion elements. The color filter array may include a plurality of filter units, and each filter unit may include a plurality of filter elements. The microlens array may include a plurality of microlenses. A pixel circuit is disposed opposite to a filter unit, one photoelectric conversion element in a pixel circuit is disposed opposite to one filter element in a filter unit, and one filter unit is disposed opposite to one microlens. The type of the image sensor may include a color image sensor, a multispectral sensor, an event sensor, a polarization sensor, or an integrated sensor of several types, etc. The specific type and function of the image sensor are not limited in the embodiments of the present application. In the image sensor, the size and shape of the pixel circuit, the resolution size, the arrangement pattern of the filter elements, the microlens pattern and the number of layers, the number, type and shape of the photoelectric conversion elements in the pixel circuit, the exposure method, etc. can all be set according to actual needs, and the embodiments of the present application do not limit this.

[0028] Dynamic Range (DR): The dynamic range is a very important performance metric in imaging. It describes the ability to simultaneously represent the highlight and shadow details in an image. If the dynamic range is relatively narrow, it is easy to have undesirable effects such as overexposure in the bright areas or complete blackness in the dark areas. The mathematical definition of the dynamic range is 20log(S / N), where S is the maximum unsaturated current of the image sensor, which can also be said to be the current when the image sensor just saturates, and N is the base current of the image sensor. The unit of the dynamic range is dB. The dynamic range of the human eye can reach about 100 dB, and the dynamic range of a camera is generally greater than 80 dB.

[0029] High Dynamic Range (HDR): To increase the brightness sampling range during shooting and the brightness level perception of the output image, high dynamic range has become a core requirement in imaging. HDR retains the details of the bright and dark parts by combining two images with different exposure ratios. It is suitable for high-contrast shooting subjects and is used to retain a wide range of details from the bright to the dark parts for high-contrast scenes and other shooting subjects. The HDR function is a very important function of mobile phone cameras. The solutions to implement the HDR function include: multi-frame HDR fusion, interleaved exposure fusion, interlaced exposure fusion, single-frame HDR, etc. The mainstream method of single-frame HDR is DCG HDR, and its advantage is that it does not lose time resolution and avoids motion blur in HDR images.

[0030] Conversion Gain (CG): In the pixel circuit of an image sensor, the conversion gain is the ratio of the electrons e generated by the photoelectric conversion module based on the photoelectric effect to the voltage signal V converted by the floating diffusion capacitance C FD The mathematical definition of the conversion gain is . In the pixel circuit of an image sensor, according to the conversion gain formula and the capacitance formula it is found that under the condition that the electrons e generated by the photoelectric conversion module based on the photoelectric effect remain unchanged, when the floating diffusion capacitance C FD is larger, the conversion gain CG is smaller, and the voltage signal V converted from the charge quantity Q accumulated by the electrons e generated by the photoelectric effect is smaller. Based on this, by changing the size of the floating diffusion capacitance C FD , the conversion gain CG of the image sensor can be changed, so that the image sensor has multiple conversion gains CG.

[0031] Dual Conversion Gain (DCG): In the pixel circuit of an image sensor, by adding a capacitor and a switching switch, the pixel circuit is equipped with two capacitors, thereby obtaining the DCG function. DCG is a dual-gain method at the pixel level, which divides the gain into two levels: High Conversion Gain (HCG) and Low Conversion Gain (LCG). The LCG mode corresponds to bright scenes, with a larger capacitor, lower conversion gain, and lower sensitivity; the HCG mode corresponds to low-light scenes, with a smaller capacitor, higher conversion gain, and higher sensitivity. The image sensor combines the LCG image and the HCG image through the DCG pixel circuit to retain the details of the bright and dark parts and achieve the HDR function.

[0032] Pixel circuit: The pixel circuit may include a photoelectric conversion module, a capacitor module, an output module, and a reset switch; the photoelectric conversion module includes a plurality of photoelectric conversion elements and a plurality of transfer transistors, one photoelectric conversion element is connected in series with one transfer transistor, and the plurality of transfer transistors are connected in parallel; the capacitor module may include at least two capacitors and a switching switch, the at least two capacitors are connected in parallel, and the switching switch is connected in series with one of the capacitors; the output module may include a source follower and a column selection signal switch; the reset switch may include a reset triode. The various pixel data output by the pixel circuit are voltage data.

[0033] Timing control unit: The timing control unit is used to control the order of closing and opening of each switch in the pixel circuit. Among them, the timing control unit can also change the order of closing and opening of each switch in the pixel circuit under the control of the image processing unit.

[0034] Analog to Digital Converter (ADC): The analog-to-digital converter is used to convert the pixel data output by the pixel circuit from an analog voltage signal to a digital signal, and the converted digital signal can be output to the image processing unit for processing.

[0035] Image Signal Processor (ISP): The image processing unit is used to perform a series of algorithms and processing steps on the original image data output by the image sensor, and convert the original image data into a high-quality image.

[0036] The following will describe in detail the image sensor, electronic device, and image processing method provided by the embodiments of the present application with reference to the accompanying drawings.

[0037] As Figure 1As shown in the figure, an embodiment of the present application provides an image sensor 10, which may include: a pixel circuit 100, an image processing unit (ISP) 200, and a timing control unit 300; the pixel circuit 100 is respectively connected to the image processing unit 200 and the timing control unit 300, and the image processing unit 200 is connected to the timing control unit 300; The pixel circuit 100 includes a photoelectric conversion module 101 and a capacitor module 102. The photoelectric conversion module 101 includes a first switch 1013 and a second switch 1014. The capacitor module 102 includes a switching switch 1023; the timing control unit 300 is respectively connected to the first switch 1013, the second switch 1014, and the switching switch 1023; When the first switch 1013, the second switch 1014, and the switching switch 1023 are all in the conducting state, the first pixel data output by the pixel circuit 100 is used to generate a first image; When the first switch 1013 and the switching switch 1023 are in the conducting state and the second switch 1014 is in the off state, the second pixel data output by the pixel circuit 100 is used to focus on the first photographed object in the first image.

[0038] In some embodiments of the present application, as Figure 1 shown, the photoelectric conversion module 101 may include a first photoelectric conversion element 1011, a second photoelectric conversion element 1012, a first switch 1013, and a second switch 1014. The first photoelectric conversion element 1011 and the second photoelectric conversion element 1012 are connected in parallel. The first end of the first photoelectric conversion element 1011 is grounded, the second end of the first photoelectric conversion element 1011 is connected to the first end of the first switch 1013, and the second end of the first switch 1013 is connected to the first end of the output module 103; the first end of the second photoelectric conversion element 1012 is grounded, the second end of the second photoelectric conversion element 1012 is connected to the first end of the second switch 1014, and the second end of the second switch 1014 is connected to the first end of the output module 103.

[0039] Wherein, the first switch 1013 is used to control the transfer of a first electron to the capacitor module 102, and the first electron e1 is an electron generated by the first photoelectric conversion element 1011 based on the photoelectric effect.

[0040] Wherein, the second switch 1014 is used to control the transfer of a second electron to the capacitor module 102, and the second electron e2 is an electron generated by the second photoelectric conversion element 1012 based on the photoelectric effect.

[0041] In this way, some embodiments of the present application can convert the light sensed by the photoelectric conversion elements, such as the first photoelectric conversion element 1011 and the second photoelectric conversion element 1012 in the photoelectric conversion module 101, into electrons. Subsequently, transfer switches, such as the first switch 1013 and the second switch 1014 in the photoelectric conversion module 101, can be used to transfer the electrons generated by the photoelectric effect to the capacitor module 102.

[0042] In some embodiments of the present application, as Figure 1 shown, the capacitor module 102 includes a first capacitor 1021, a second capacitor 1022, and a switching switch 1023. The first capacitor 1021 is connected in parallel with the switching switch 1023, and the switching switch 1023 is connected in series with the second capacitor 1022. The first end of the first capacitor 1021 is grounded, the second end of the first capacitor 1021 is connected to the first end of the output module 103, the first end of the second capacitor 1022 is grounded, the second end of the second capacitor 1022 is connected to the first end of the switching switch 1023, and the second end of the switching switch 1023 is connected to the first end of the output module 103.

[0043] Among them, the switching switch 1023 is used to switch the capacitance of the capacitor module 102, thereby switching the conversion gain mode of each pixel circuit 100 in the image sensor 10.

[0044] In some embodiments of the present application, when the switching switch 1023 is in the conducting state, the first capacitor 1021 and the second capacitor 1022 form a parallel relationship, and the capacitance value C FD of the capacitor module 102 is the sum of the capacitance value C1 of the first capacitor 1021 and the capacitance value C2 of the second capacitor 1022. In this case, the image sensor has a first conversion gain CG1, , where e can be regarded as the electrons generated by the photoelectric conversion module 101 based on the photoelectric effect.

[0045] In some embodiments of the present application, when the switching switch 1023 is in the off state, the capacitance value C FD of the capacitor module 102 is the capacitance value C1 of the first capacitor 1021. In this case, the image sensor has a second conversion gain CG2, , where e can be regarded as the electrons generated by the photoelectric conversion module 101 based on the photoelectric effect.

[0046] In some embodiments of the present application, when the image sensor has the above two conversion gains, the first conversion gain CG1 is less than the second conversion gain CG2. The first conversion gain CG1 can be referred to as the low conversion gain LCG, and the second conversion gain CG2 can be referred to as the high conversion gain HCG.

[0047] In some embodiments of the present application, as Figure 1 shown, the timing control unit 300 is respectively connected to the third terminal of the first switch 1013, the third terminal of the second switch 1014, and the third terminal of the switching switch 1023. The timing control unit is used to control the closing and opening sequence of the first switch 1013, the second switch 1014, and the switching switch 1023 in the pixel circuit 100.

[0048] Among them, when the first switch 1013 is in the conducting state and the second switch 1014 is in the non-conducting state, the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect can be the first electrons e1, and the first electrons e1 are the electrons generated by the first photoelectric conversion element 1011 based on the photoelectric effect.

[0049] Among them, when the first switch 1013 and the second switch 1014 are both in the conducting state, the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect can be the sum of the first electrons e1 and the second electrons e2. The first electrons e1 are the electrons generated by the first photoelectric conversion element 1011 based on the photoelectric effect, and the second electrons e2 are the electrons generated by the second photoelectric conversion element 1012 based on the photoelectric effect.

[0050] In some embodiments of the present application, when the first switch 1013, the second switch 1014, and the switching switch 1023 are all in the conducting state, the electric charge accumulated by the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect is converted into a first voltage signal V1 by the capacitor module 102. Since the first switch 1013 and the second switch 1014 are in the conducting state, ; since the switching switch 1023 is in the conducting state, the capacitance value of the capacitor module 102 , the first voltage signal , where is the electric charge accumulated by the first electrons e1 and the second electrons e2. The output module 103 is used to read out the first voltage signal V1 of the capacitor module 102 as the first pixel data, and the first pixel data output from the output terminal Vout of the pixel circuit 100 can be used to generate a first image.

[0051] In some embodiments of the present application, when the first switch 1013 and the switching switch 1023 are in the conducting state and the second switch 1014 is in the non-conducting state, the electric charge accumulated by the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect is converted into a second voltage signal V2 by the capacitor module 102. Since the first switch 1013 is in the conducting state and the second switch 1014 is in the non-conducting state, ; since the switching switch 1023 is in the conducting state, the capacitance value of the capacitor module 102 ; the second voltage signal ; wherein, is the accumulated charge amount of the first electron e1. The output module 103 is configured to read out the second voltage signal V2 of the capacitor module 102 as the second pixel data. The second pixel data output from the output terminal Vout of the pixel circuit 100 is used to focus on the first photographed object in the first image.

[0052] In practical applications, the first conversion gain of the image sensor is used for the pixel circuit 100 to output the first pixel data and the second pixel data. Both the first pixel data and the second pixel data can be obtained based on the first conversion gain. The first pixel data can be referred to as the image data of LCG, and the first image generated based on the first pixel data can be referred to as the LCG image. The second pixel data can be referred to as the phase detection data of LCG.

[0053] According to the image sensor provided by the embodiments of the present application, since the pixel circuit 100 in the image sensor 10 can output the first pixel data for generating the first image and the second pixel data for focusing on the first photographed object in the first image, the first image can be generated by using the first pixel data, and the first photographed object in the first image can be effectively focused by using the second pixel data, so that the picture of the first photographed object in the first image is clear, thereby improving the picture quality of the first image.

[0054] In some embodiments of the present application, in order to enable the pixel circuit 100 to output the first pixel data and the second pixel data, some embodiments of the present application can use two timing controllers to control the closing and opening sequence of the first switch 1013, the second switch 1014, and the switching switch 1023. As Figure 2 shown, the timing control unit 300 can include a first timing controller 301 and a second timing controller 302; the image processing unit 200 is connected to the first timing controller 301, and the first timing controller 301 is connected to the switching switch 1023; the second timing controller 302 is respectively connected to the first switch 1013 and the second switch 1014.

[0055] In some embodiments of the present application, as Figure 2 shown, the first timing controller 301 is connected to the third terminal of the switching switch 1023, and the first timing controller 301 is used to control the closing and opening sequence of the switching switch 1023. The second timing controller 302 is respectively connected to the third terminal of the first switch 1013 and the third terminal of the second switch 1014, and the second timing controller 302 is used to control the closing and opening sequence of the first switch 1013 and the second switch 1014.

[0056] Among them, the control timing of the timing control unit 300 includes: a first time point, a second time point, and a third time point; the first time point is the same as the second time point, and the third time point is later than the second time point; Among them, the first time point is the starting time point when the first timing controller 301 controls the switching switch 1023 to conduct, the second time point is the starting time point when the second timing controller 302 controls the first switch 1013 to conduct, and the third time point is the starting time point when the second timing controller 302 controls the second switch 1014 to conduct.

[0057] In this way, at the first time point, the first timing controller 301 closes the switching switch 1023, and at the same time the second timing controller 302 closes the first switch 1013, so that the first switch 1013 and the switching switch 1023 are in the conducting state and the second switch 1014 is in the off state, and the second pixel data output by the output terminal Vout of the pixel circuit 100; at the third time point, the second timing controller 302 closes the second switch 1014 again, so that the first switch 1013, the second switch 1014, and the switching switch 1023 are all in the conducting state, and the first pixel data output by the output terminal Vout of the pixel circuit 100. Thus, when the switching switch 1023 is controlled by the first timing controller 301, the pixel circuit 100 outputs the second pixel data and the first pixel data in sequence.

[0058] In some embodiments of the present application, in order to improve the accuracy of the first pixel data or the second pixel data in the first conversion gain mode, the first timing controller 301 can also control the analog-to-digital converter (ADC) to collect the first pixel data or the second pixel data in the first conversion gain mode multiple times, which is convenient for the image processing unit 200 to perform average calculation on the first pixel data or the second pixel data in the first conversion gain mode later, so as to improve the accuracy of the first pixel data or the second pixel data in the first conversion gain mode.

[0059] For example, as Figure 2 shown, the image sensor 10 may further include an analog-to-digital converter (ADC) 400. The first end of the analog-to-digital converter 400 is connected to the pixel circuit 100, the second end of the analog-to-digital converter 400 is connected to the image processing unit 200, and the third end of the analog-to-digital converter 400 is connected to the first timing controller 301.

[0060] Among them, the pixel circuit 100 is connected to the image processing unit 200 via the analog-to-digital converter 400, and the first timing controller 301 is connected to the control end of the analog-to-digital converter 400.

[0061] Among them, the analog-to-digital converter 400 is used to convert the pixel data output from the output terminal of the pixel circuit 100 from an analog voltage signal into a digital signal, and the converted digital signal can be output to the image processing unit for processing.

[0062] In a specific example, the first timing controller 301 can control the analog-to-digital converter 400 to collect the first pixel data in the first conversion gain mode once, and collect the second pixel data in the first conversion gain mode twice, facilitating the image processing unit 200 to perform an average calculation on the two second pixel data in the first conversion gain mode, thereby improving the accuracy of the second pixel data in the first conversion gain mode, and thus improving the accuracy of focusing on the first photographed object in the first image using the second pixel data.

[0063] In another specific example, the first timing controller 301 can control the analog-to-digital converter 400 to collect the first pixel data in the first conversion gain mode twice, and collect the second pixel data in the first conversion gain mode once, facilitating the image processing unit 200 to perform an average calculation on the two first pixel data in the first conversion gain mode, thereby improving the accuracy of the first pixel data in the first conversion gain mode, and thus improving the accuracy of generating the first image using the second pixel data.

[0064] In this way, since the first timing controller 301 is connected to the analog-to-digital converter 400, the first timing controller 301 can also control the analog-to-digital converter 400 to collect the first pixel data or the second pixel data in the first conversion gain mode multiple times, improving the accuracy of the pixel data in the first conversion gain mode.

[0065] In addition, in order to further improve the display effect of the image, when generating the first image using the first pixel data and effectively focusing on the first photographed object in the first image using the second pixel data, the pixel circuit 100 provided in some embodiments of the present application can also output third pixel data for generating a second image and fourth pixel data for focusing on the second photographed object in the second image.

[0066] For example, in some embodiments of the present application, when the first switch 1013 and the second switch 1014 are in the conducting state and the switching switch 1023 is in the off state, the third pixel data output by the pixel circuit 100 is used to generate the second image. When the first switch 1013 is in the conducting state and the second switch 1014 and the switching switch 1023 are in the off state, the fourth pixel data output by the pixel circuit 100 is used to focus on the second photographed object in the second image.

[0067] For example, when the first switch 1013 and the second switch 1014 are in the conducting state and the switching switch 1023 is in the off state, the amount of charge accumulated by the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect is converted into a third voltage signal V3 by the capacitor module 102. Since the first switch 1013 and the second switch 1014 are in the conducting state, ; since the switching switch 1023 is in the off state, the capacitance value of the capacitor module 102 , the third voltage signal , where is the amount of charge accumulated by the first electron e1 and the second electron e2. The output module 103 is used to read out the third voltage signal V3 of the capacitor module 102 as the third pixel data, and the third pixel data output from the output terminal Vout of the pixel circuit 100 is used to generate a second image.

[0068] For example, when the first switch 1013 and the switching switch 1023 are in the conducting state and the second switch 1014 is in the off state, the amount of charge accumulated by the electrons e generated by the photoelectric conversion module 101 based on the photoelectric effect is converted into a fourth voltage signal V4 by the capacitor module 102. Since the first switch 1013 is in the conducting state and the second switch 1014 is in the off state, ; since the switching switch 1023 is in the off state, the capacitance value of the capacitor module 102 ; the fourth voltage signal ; where is the amount of charge accumulated by the first electron e1. The output module 103 is used to read out the fourth voltage signal V4 of the capacitor module 102 as the fourth pixel data, and the fourth pixel data output from the output terminal Vout of the pixel circuit 100 is used to focus on the second object to be photographed in the second image.

[0069] Among them, the first object to be photographed and the second object to be photographed may be the same object to be photographed, or the first object to be photographed and the second object to be photographed may be different objects to be photographed. This application does not make specific restrictions on this. For example, in practical applications, the first object to be photographed may be an object to be photographed in a high-brightness area, and the second object to be photographed may be an object to be photographed in a shadow area. The first object to be photographed and the second object to be photographed are different objects to be photographed.

[0070] In practical applications, the second conversion gain CG2 of the image sensor is used for the pixel circuit 100 to output the third pixel data and the fourth pixel data. In other words, both the third pixel data and the fourth pixel data can be obtained based on the second conversion gain CG2. When the second conversion gain CG2 is the high conversion gain HCG, the third pixel data can be referred to as the image data of HCG, the second image generated based on the third pixel data can be referred to as the HCG image, and the fourth pixel data can be referred to as the phase detection data of HCG.

[0071] According to the image sensor provided by the embodiments of the present application, since the pixel circuit 100 in the image sensor 10 can output the first pixel data for generating the first image, the second pixel data for focusing on the first photographed object in the first image, the third pixel data for generating the second image, and the fourth pixel data for focusing on the second photographed object in the second image, the first image can be generated using the first pixel data, the first photographed object in the first image can be effectively focused using the second pixel data, the second image can be generated using the third pixel data, and the second photographed object in the second image can be effectively focused using the fourth pixel data, further improving the display effect of the image.

[0072] In some embodiments of the present application, in order for the pixel circuit 100 to output the third pixel data and the fourth pixel data, the second timing controller 302 is connected to the switching switch 1023, and the embodiments of the present application can adopt the second timing controller 302 to control the closing and opening sequence of the switching switch 1023.

[0073] For example, as Figure 2 shown, the second timing controller 302 is connected to the third terminal of the switching switch 1023, and the second timing controller 302 can be used to control the closing and opening sequence of the switching switch 1023. At the same time, the second timing controller 302 is respectively connected to the third terminal of the first switch 1013 and the third terminal of the second switch 1014. The second timing controller 302 is used to control the closing and opening sequence of the first switch 1013 and the second switch 1014.

[0074] In this case, the control timing of the timing control unit 300 further includes a fourth time point, and the control timing for the timing control unit 300 to execute processing can include: a second time point, a third time point, and a fourth time point; the third time point is later than the second time point; the fourth time point is later than the third time point.

[0075] Among them, the second time point is the starting time point when the second timing controller 302 controls the first switch 1013 to conduct, the third time point is the starting time point when the second timing controller 302 controls the second switch 1014 to conduct, and the fourth time point is the starting time point when the second timing controller 302 controls the changeover switch 1023 to conduct.

[0076] In this way, at the second time point, the second timing controller 302 closes the first switch 1013. At this time, the first switch 1013 is in the conducting state, and the second switch 1014 and the changeover switch 1023 are in the off state, and the fourth pixel data output by the output terminal Vout of the pixel circuit 100; at the third time point, the second timing controller 302 then closes the second switch 1014. At this time, the first switch 1013 and the second switch 1014 are in the conducting state, and the changeover switch 1023 is in the off state, and the third pixel data output by the output terminal Vout of the pixel circuit 100; at the fourth time point, the second timing controller 302 finally closes the changeover switch 1023. At this time, the first switch 1013, the second switch 1014, and the changeover switch 1023 are all in the conducting state, and the first pixel data output by the output terminal Vout of the pixel circuit 100. Thus, when the changeover switch 1023 is controlled by the second timing controller 302, the second timing controller 302 closes the first switch 1013, the second switch 1014, and the changeover switch 1023 in sequence, so that the pixel circuit 100 outputs the fourth pixel data, the third pixel data, and the first pixel data in sequence.

[0077] In some embodiments of the present application, for different lighting scenarios, in order to focus on the photographed objects in the first image and the second image respectively, the pixel circuit 100 may have a first reading mode and a second reading mode; the first reading mode is the mode in which the changeover switch 1023 is controlled by the first timing controller 301; the second reading mode is the mode in which the changeover switch 1023 is controlled by the second timing controller 302; Among them, the pixel circuit 100 is used to output the first pixel data and the second pixel data in the first reading mode; the pixel circuit 100 is used to output the first pixel data, the third pixel data, and the fourth pixel data in the second reading mode.

[0078] In this way, the pixel circuit 100 can output first pixel data for generating a first image and second pixel data for focusing on a first photographed object in the first image in the first reading mode, so as to achieve focusing on the first photographed object in the first image; the pixel circuit 100 can output third pixel data for generating a second image and fourth pixel data for focusing on a second photographed object in the second image in the second reading mode, so as to achieve focusing on the second photographed object in the second image, and can focus on the photographed objects in the first image and the second image respectively for different illumination scenarios.

[0079] For example, in practical applications, when the pixel circuit 100 starts to work, it can be in the second reading mode. The switching switch 1023 is controlled by the second timing controller 302. The second timing controller 302 closes the first switch 1013, the second switch 1014 and the switching switch 1023 in sequence, so that the pixel circuit 100 outputs the fourth pixel data, the third pixel data and the first pixel data in sequence. The first pixel data is used to generate a first image, the third pixel data is used to generate a second image, and the fourth pixel data is used to focus on the second photographed object in the second image.

[0080] For an illumination scenario where it is necessary to focus on the first photographed object in the first image, after the pixel circuit 100 outputs the above three kinds of pixel data in the second reading mode, it is still difficult to focus on the first photographed object in the first image. In this case, the control priority of the first timing controller 301 over the switching switch 1023 can be higher than that of the second timing controller 302 over the switching switch 1023. Among them, the first control instruction output by the first timing controller 301 to the switching switch 1023 can be used to overwrite the second control instruction output by the second timing controller 302 to the switching switch 1023. In this way, the first timing controller 301 can take over the control right of the switching switch 1023 from the second timing controller 302, and the first timing controller 301 controls the closing and opening sequence of the switching switch 1023. At this time, the pixel circuit 100 switches from the second reading mode to the first reading mode, so that the pixel circuit 100 outputs the first pixel data for generating a first image and the second pixel data for focusing on the first photographed object in the first image in the first reading mode.

[0081] For example, in some embodiments of the present application, when the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, the image processing unit 200 can be used to control the pixel circuit 100 to switch from the second reading mode to the first reading mode.

[0082] For example, the third pixel data includes a third voltage signal V3. The voltage signal value indicated by the third pixel data may be a third digital signal obtained after the third voltage signal V3 is processed by analog-to-digital conversion by the analog-to-digital converter 400. The third digital signal is a binary data. The overexposure threshold may include a first preset data. The image processing unit 200 is configured to determine that the third digital signal is greater than the first preset data. The first preset data may be 920 LSB (Least Significant Bit) or other numbers, which is not specifically limited in this application. By determining that the third digital signal is greater than the first preset data of 920 LSB by the image processing unit 200, it can be determined that the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, and thus it is determined that the third pixel data is overexposed invalid data. In this case, it is the illumination scenario where the first photographed object in the first image needs to be focused. Based on this, when the image processing unit 200 determines that the third digital signal is greater than the first preset data, it outputs a mode switching signal to the first timing controller 301 in the timing control unit 300, so that the pixel circuit 100 switches from the second reading mode to the first reading mode.

[0083] In addition, in some embodiments of the present application, when the voltage signal value indicated by the first pixel data is greater than the second preset data, the image processing unit 200 is configured to control the pixel circuit 100 to switch from the second reading mode to the first reading mode. Wherein, the second preset data may be determined based on the first preset data included in the overexposure threshold.

[0084] It should be noted that the first pixel data and the third pixel data have a proportional relationship. The third voltage signal V3 included in the third pixel data and the first voltage signal V1 included in the first pixel data have a proportional relationship. Furthermore, the third digital signal obtained after the third voltage signal V3 is processed by analog-to-digital conversion by the analog-to-digital converter 400 and the first digital signal obtained after the first voltage signal V1 is processed by analog-to-digital conversion by the analog-to-digital converter 400 have a proportional relationship. For example, the value of the third digital signal is four times the value of the first digital signal. The overexposure threshold may include a first preset data, and the value of the first preset data may be four times the value of the second preset data. In some embodiments of the present application, it can also be determined by the image processing unit 200 that the first digital signal obtained by analog-to-digital conversion of the first voltage signal V1 is greater than the second preset data of 230 LSB, thereby determining that the third digital signal is greater than the first preset data of 920 LSB, determining that the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, determining that the third pixel data is overexposed invalid data, and outputting a mode switching signal to the first timing controller 301 in the timing control unit 300, so that the pixel circuit 100 switches from the second reading mode to the first reading mode.

[0085] Among them, the embodiments of the present application do not limit the proportional relationship between the capacitance value C1 of the first capacitor 1021 and the capacitance value C2 of the second capacitor 1022. For the convenience of description, the present application is described according to a ratio of 1:3. If C1:C2 = 1:3, then C1:(C1 + C2) = 1:4, CG1:CG2 = 1:4, and the first voltage signal V1 indicated by the first pixel data: the third voltage signal V3 indicated by the third pixel data = 1:4.

[0086] The first voltage signal V1 is subjected to analog-to-digital conversion processing by the analog-to-digital converter 400 to obtain a first digital signal. The first digital signal is a binary data. The image processing unit 200 determines that the first digital signal is greater than a second preset data. The second preset data can take 230 LSB or other numbers, and the present application does not make specific limitations. By determining that the first digital signal is greater than the second preset data 230 LSB through the image processing unit 200, it can be determined that the third digital signal is greater than the first preset data 920 LSB, and thus it is determined that the third pixel data is overexposed invalid data. In this case, it is the illumination scene where the first photographed object in the first image needs to be focused. Based on this, when the first digital signal obtained by analog-to-digital conversion of the first voltage signal V1 is greater than the second preset data, the image processing unit 200 can also output a mode switching signal to the first timing controller 301 in the timing control unit 300, so that the pixel circuit 100 switches from the second reading mode to the first reading mode.

[0087] In some embodiments of the present application, when the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, the second end of the image processing unit 200 outputs a mode switching signal, and the mode switching signal is used to control the pixel circuit 100 to switch from the second reading mode to the first reading mode.

[0088] For example, the first timing controller 301 uses the mode switching signal output by the image processing unit 200 as an input signal to determine whether to switch the reading mode of the pixel circuit 100. If necessary, the first timing controller 301 completes the switching of the reading mode by controlling the closing and opening sequence of the switching module 1023, switches the pixel circuit 100 from the second reading mode to the first reading mode, and simultaneously controls the analog-to-digital converter 400 to cooperate with the data output in the first reading mode.

[0089] Taking the output of the analog-to-digital converter 400 as 10 bits as an example, if the image processing unit 200 determines that the third digital signal indicated by the third pixel data is greater than 920 LSB, it switches to the first reading mode, otherwise it maintains the second reading mode. If CG1:CG2 = 1:4, the image processing unit 200 can also switch to the first reading mode by analyzing that the first digital signal indicated by the first pixel data is greater than 230 LSB, otherwise it always uses the second reading mode.

[0090] In practical applications, taking the first image as an LCG image as an example, the LCG image is an image that reflects a high-brightness scene (such as a sky, a strong light source, etc.), and the second pixel data is used to focus on the photographed object in the high-brightness scene. To implement the phase detection autofocus function, in the optoelectronic conversion module 101, the number of the first optoelectronic conversion elements 1011 is the same as that of the second optoelectronic conversion elements 1012, and the first optoelectronic conversion elements 1011 and the second optoelectronic conversion elements 1012 are arranged symmetrically left and right. For example, in the optoelectronic conversion module 101, the number of the first optoelectronic conversion elements 1011 is two, the number of the second optoelectronic conversion elements 1012 is two, and the two first optoelectronic conversion elements 1011 and the two second optoelectronic conversion elements 1012 are arranged symmetrically left and right.

[0091] As Figure 3A shown, in the optoelectronic conversion module 101, the first optoelectronic conversion element may include two photodiodes: a first photodiode PD1 and a second photodiode PD2; the second optoelectronic conversion element may include two photodiodes: a third photodiode PD3 and a fourth photodiode PD4; the first switch may include two transfer transistors: a first transfer transistor TG1 and a second transfer transistor TG2; the second switch may include two transfer transistors: a third transfer transistor TG3 and a fourth transfer transistor TG4.

[0092] In addition, the pixel circuit 100 further includes a reset switch 104. The first end of the reset switch 104 is connected to the first power supply voltage terminal VDD1, and the second end of the reset switch 104 is connected to the first end of the output module 103; the output module 103 includes a source follower 1031, a column selection signal switch 1032, and a reference power supply 1033. The first end of the source follower 1031 is connected to the second end of the reset switch 104, the second end of the source follower 1031 is connected to the second power supply voltage terminal VDD2, the third end of the source follower 1031 is connected to the first end of the column selection signal switch 1032, the second end of the column selection signal switch 1032 is grounded via the reference power supply 1033, and the second end of the column selection signal switch 1032 is connected to the analog-to-digital converter 400. Among them, the source follower 1031 is used to record the voltage signal generated by the capacitor module 102. When the column selection signal switch 1032 is closed, the voltage signal is output through the Vout node. Furthermore, based on this voltage signal, the exposure image corresponding to the optoelectronic conversion module 101 after photosensitivity is obtained.

[0093] The image processing unit 200 is used to focus on the object in the first image based on the difference between the first pixel data and the second pixel data and the second pixel data. For example, the first pixel data includes LCG image data, the second pixel data includes first PDAF data of LCG, the image processing unit 200 obtains second PDAF data of LCG based on the difference between the LCG image data and the first PDAF data of LCG, and focuses on the photographed object in the LCG image based on the first PDAF data of LCG and the second PDAF data of LCG; wherein the first PDAF data is one of the left PDAF data and the right PDAF data, and the second PDAF data is the other of the left PDAF data and the right PDAF data.

[0094] If the reading mode does not need to be switched, the pixel circuit 100 will output pixel data according to the second reading mode. The working process of the pixel circuit 100 in the second reading mode is as follows: First, the first transfer transistor TG1 and the third transfer transistor TG3 are closed, so that the photosensitive electrons in the first photodiode PD1 and the third photodiode PD3 flow into the first capacitor 1021; at this time, the photosensitive electrons in PD1 and PD3 form a fourth voltage signal on the first capacitor 1021, and the fourth voltage signal can be output as the fourth pixel data, which is the left PDAF data of HCG; Then, the second transfer tube TG2 and the fourth transfer tube TG4 are closed to allow the photosensitive electrons in the second photodiode PD2 and the fourth photodiode PD4 to flow into the first capacitor 1021. At this time, the photosensitive electrons in the second photodiode PD2 and the fourth photodiode PD4 are added to the first capacitor 1021 to form a new third voltage signal on the first capacitor 1021. The third voltage signal can be output as the third pixel data, which is the image data of HCG.

[0095] Finally, the switching switch 1023 is closed to allow the first capacitor 1021 and the second capacitor 1022 to form a parallel relationship, and the total electrons will be distributed on the two capacitors to form a new first voltage signal. This first voltage signal can be output as the first pixel data, which is the image data of the LCG.

[0096] The above voltage signals are followed by the source follower 1031 and selected by the column selection signal switch 1032 to be read out to the analog-to-digital converter 400 and converted into digital signals. Later, through the processing of the image processing unit 200, the image data of HCG is subtracted from the left PDAF data of HCG, and the right PDAF data of HCG can be obtained. Therefore, in the first reading mode, the HCG image will have PDAF data for focusing. However, the LCG image cannot obtain PDAF data.

[0097] If it is necessary to switch the reading mode, the pixel circuit 100 is switched from the second reading mode to the first reading mode. The working process of the pixel circuit 100 in the first reading mode is as follows: First, close the first transfer transistor TG1 and the third transfer transistor TG3. At the same time, close the switching switch 1023. The first capacitor 1021 and the second capacitor 1022 form a parallel relationship, allowing the photosensitive electrons in the first photodiode PD1 and the third photodiode PD3 to flow into the first capacitor 1021 and the second capacitor 1022. At this time, the photosensitive electrons of PD1 and PD3 form a second voltage signal on the first capacitor 1021 and the second capacitor 1022. This second voltage signal can be output as the second pixel data, and the second pixel data is the left PDAF data of the LCG. Then, close the second transfer transistor TG2 and the fourth transfer transistor TG4, allowing the photosensitive electrons in the second photodiode PD2 and the fourth photodiode PD4 to flow into the first capacitor 1021 and the second capacitor 1022. At this time, the photosensitive electrons in the second photodiode PD2 and the fourth photodiode PD4 are added to the first capacitor 1021 and the second capacitor 1022, forming a new first voltage signal on the first capacitor 1021 and the second capacitor 1022. This first voltage signal can be output as the first pixel data, and the first pixel data is the image data of the LCG. Control the analog-to-digital converter 400 to repeat reading the image data of the LCG once.

[0098] During the later processing by the image processing unit 200, subtracting the left PDAF data of the LCG from the image data of the LCG can obtain the right PDAF data of the LCG. The image data of the LCG read one more time can be subjected to multiple averaging operations to make the image data of the LCG more accurate and the image quality better.

[0099] The pixel circuit 100 sequentially outputs the left PDAF data of the HCG, the image data of the HCG, and the image data of the LCG in the second readout mode. After each pixel circuit 100 outputs data for the first time according to the second readout mode, the image data of the HCG or the image data of the LCG is fed back to the image processing unit 200 through the analog-to-digital converter 400 for judgment. Taking the overexposure threshold of 920 LSB as an example, the image processing unit 200 can switch to the first reading mode by judging that the image data of the HCG is greater than 920 LSB, otherwise, it always uses the second reading mode. If CG1:CG2 = 1:4, the image processing unit 200 can also switch to the first reading mode by judging that the LCG image data in the pixel data is greater than 230 LSB, otherwise, it always uses the second reading mode.

[0100] Such as Figure 3BAs shown in the figure, the image sensor provided by the embodiment of the present application further includes a color filter array 500. The color filter array 500 includes a plurality of filter units 510. One filter unit 510 includes at least one first filter element 511 and at least one second filter element 512. Among them, one filter unit 510 can be disposed opposite to one pixel circuit 100. One first filter element 511 is disposed opposite to one first photoelectric conversion element 1011, and one second filter element 512 is disposed opposite to one second photoelectric conversion element 1012 (not shown in the figure).

[0101] As Figure 3B shown in the figure, the color filter array 500 shows twelve filter units 510. Each filter unit 510 includes two first filter elements 511 and two second filter elements 512. Among them, the types of each filter element in each filter unit 510 are the same. R1 and R2 represent red filter elements, Gr1, Gr2, Gb1, Gb2, Gb3 and Gb4 represent green filter elements, and B1, B2, B3 and B4 represent blue filter elements.

[0102] Among them, as Figure 3B shown in the figure, the marked shaded areas are the dark areas of the image. For example, the areas where the 4 red filter elements R1 are located are dark areas, the areas where the 4 blue filter elements B1 are located are dark areas, and the areas where the 4 green filter elements Gb4 are located are dark areas. The unmarked shaded areas are the bright areas of the image. For example, the areas where the 4 red filter elements R2 are located are bright areas.

[0103] Since the areas where the four red filter elements R1 are located are dark areas, the left PDAF data of HCG, the image data of HCG, and the image data of LCG output by the pixel circuits 100 corresponding to the four red filter elements R1 are all valid in the second readout mode. However, the areas where the four red filter elements R2 are located are bright areas, resulting in the image data of HCG output by the pixel circuits 100 corresponding to the four red filter elements R2 exceeding 920 LSB in the second readout mode, which is overexposed data. Therefore, it is not necessary for the pixel circuits corresponding to the four red filter elements R2 to output all the data in the HCG mode, and the image data and left PDAF data in the LCG mode can be output by switching the readout mode. Based on this, in the image sensor, after each pixel circuit 100 outputs data for the first time according to the second readout mode, the image data of HCG or the image data of LCG is fed back to the image processing unit 200 through the analog-to-digital converter 400 for judgment. When the image data of HCG in the pixel data is greater than 920 LSB, or the image data of LCG is greater than 230 LSB, the first readout mode is switched, and the left PDAF data of LCG, the image data of LCG, and the image data of LCG are read out in sequence, which can avoid outputting invalid overexposed data of HCG and enable the pixel circuit 100 to obtain the PDAF data of LCG; and the image data of LCG is output twice, and multiple average noise reduction processes can be performed to improve the image quality.

[0104] The image sensor 10 provided by the embodiment of the present application can obtain images with a large dynamic range in a high dynamic range scenario. At the same time, by optimizing the closing and opening sequence of each switch in the pixel circuit 100, both the HCG image and the LCG image can have omnidirectional full-pixel PDAF information, enabling the camera to have good HDR image quality in a high dynamic range scenario and quickly and accurately focus on the photographed objects in bright and dark scenes, improving the user's camera usage experience.

[0105] In addition, the present application also provides an imaging module 40 including the above image sensor, as Figure 4 shown, the imaging module 40 includes a lens (Lens) 41, a voice coil motor (Voice Coil Motor) 42, an infrared filter (IR Filter) 43, the image sensor 44 provided by the present application, a connecting component 45, a base 46, and a circuit board 47.

[0106] It should be noted that the imaging module provided by the embodiment of the present application includes the image sensor provided in any of the above embodiments and can implement all the functions of the image sensor. To avoid repetition, details are not described herein again.

[0107] Among them, the lens 41 is used for light collection and focusing. The lens 41 is wrapped and fixed by the voice coil motor 42. The upper and lower ends of the voice coil motor are connected to the elastic pieces. During focusing, an electromagnetic force is generated by the motor through energization, and this force finally balances with the elastic force of the elastic piece. The position of the motor can be controlled by the magnitude of the energization, and then the lens is pushed to the in-focus position by the motor. The function of the infrared filter 43 is to filter out unnecessary light projected onto the image sensor. The light passing through the infrared filter 43 can be sensed by the image sensor, preventing the image sensor from generating false colors or ripples, so as to improve its resolution and color reproducibility.

[0108] Among them, the voice coil motor may include an upper cover, an upper spring piece, a lower spring piece, a housing, a coil, a magnet, a moving part, and a terminal. Among them, the upper cover plays a role in protecting the motor; the upper spring piece generates a force on the motor when deformed, and the sum of it and the lower spring piece balances the electromagnetic force; the housing is the main frame of the fixed part of the motor, has a magnetic conduction effect, and can improve the effective utilization rate of the magnet; when the coil is energized, an upward thrust is generated under the magnetic field of the magnet, driving other components of the moving part to move together; the magnet generates a magnetic field, so that the energized coil generates an electromagnetic force under its magnetic field, and the moving part carrier drives the lens to move together; the lower spring piece generates a force on the motor when deformed, and the sum of it and the upper spring piece balances the electromagnetic force; the base is directly assembled with the motor and the circuit board; the mobile phone supplies power to the motor through the terminal.

[0109] In addition, the embodiment of the present application also provides an electronic device 50, as Figure 5 shown, the electronic device 50 includes a processor 501 and an image sensor 502.

[0110] It should be noted that the electronic device provided by the embodiment of the present application includes the image sensor provided by any of the above embodiments, and can implement all functions of the image sensor. To avoid repetition, it will not be elaborated here.

[0111] In the embodiments of the present application, the electronic device may be a terminal or other devices other than terminals. Exemplarily, the electronic device may be a smart watch, a mobile phone, a tablet computer, a laptop computer, a handheld computer, an in-vehicle electronic device, a Mobile Internet Device (MID), an augmented reality (AR) device, a virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc. It may also be a server, a Network Attached Storage (NAS), a personal computer (PC), etc. The embodiments of the present application do not make specific limitations.

[0112] In addition, the embodiments of the present application further provide an image processing method, which is executed by the electronic device provided in any of the above embodiments.

[0113] As Figure 6 shown, the image processing method provided by the embodiments of the present application may include: Step 610: Obtain first pixel data and second pixel data output by a pixel circuit; Step 620: Calculate the difference between the voltage signal value indicated by the first pixel data and the voltage signal value indicated by the second pixel data; Step 630: Generate a first image based on the first pixel data; Step 640: Focus on a first photographed object in the first image based on the difference and the second pixel data.

[0114] Among them, various pixel data output by the pixel circuit are all voltage data. For example, the first pixel data and the second pixel data may be voltage data. The first pixel data may include a first voltage signal, and the second pixel data may include a second voltage signal. In step 620, the difference between the first voltage signal and the second voltage signal may be determined, and in step 640, the first photographed object in the first image may be focused based on the difference and the second voltage signal.

[0115] For example, for a pixel circuit in an image sensor, the pixel circuit corresponds to as Figure 3BIn the area where the four red filter elements R2 shown are located, the voltage signal value indicated by the first pixel data output by the pixel circuit is 250 LSB, the voltage signal value indicated by the second pixel data is 100 LSB, and the difference between the voltage signal value indicated by the first pixel data and the voltage signal value indicated by the second pixel data is 150 LSB. Specifically, the image processing unit can calculate the phase difference result of the red channel based on the difference of 150 LSB and the voltage signal value of 100 LSB indicated by the second pixel data. Based on the phase difference result, the focus offset is determined, and based on the focus offset, the first object to be photographed in the first image is focused. Among them, the first object to be photographed can be an object to be photographed such as a portrait, a building, an item, etc., and the first object to be photographed can be located at various different positions in the first image.

[0116] Among them, the pixel circuit can be the pixel circuit in the image sensor provided in any of the above embodiments. To avoid repetition, it will not be elaborated here.

[0117] According to the image processing method provided by the embodiments of the present application, the first image can be generated using the first pixel data, and the first object to be photographed in the first image can be effectively focused using the second pixel data, so that the picture of the first object to be photographed in the first image is clear, thereby improving the picture quality of the first image.

[0118] In some embodiments of the present application, as Figure 7 shown, the image processing method provided by the embodiments of the present application may include: Step 710: Obtain the third pixel data output by the pixel circuit in the second read mode; Step 720: When the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, switch the pixel circuit from the second read mode to the first read mode; Among them, the first pixel data and the second pixel data are obtained in the first read mode; Step 730: Obtain the first pixel data and the second pixel data output by the pixel circuit in the first read mode; Step 740: Calculate the difference between the voltage signal value indicated by the first pixel data and the voltage signal value indicated by the second pixel data; Step 750: Generate the first image based on the first pixel data; Step 760: Focus on the first object to be photographed in the first image based on the difference and the second pixel data.

[0119] Among them, the specific content of steps 730 to 760 can refer to the specific content of steps 610 to 640, and steps 710 and 720 can be executed before step 730.

[0120] Wherein, in the first reading mode, the pixel circuit outputs first pixel data and second pixel data; in the second reading mode, the pixel circuit outputs first pixel data, third pixel data, and fourth pixel data.

[0121] For example, after the pixel circuit outputs the first pixel data, the third pixel data, and the fourth pixel data according to the second readout mode, the third pixel data is fed back to the image processing unit 200 through the analog-to-digital converter 400 for judgment. When the third digital voltage signal indicated by the third pixel data is greater than 920 LSB, it indicates that the third pixel data is overexposed invalid data, and then the first reading mode is switched. The pixel circuit sequentially outputs the second pixel data and the first pixel data, which can avoid outputting overexposed invalid data and enable the pixel circuit to obtain the second pixel data for focusing on the first photographed object in the first image.

[0122] According to the image processing method provided by the embodiments of the present application, when the third pixel data is overexposed invalid data, the pixel circuit can be switched from the second reading mode to the first reading mode, which can avoid outputting overexposed invalid data and enable the pixel circuit to obtain the second pixel data for focusing on the first photographed object in the first image, and effectively focus on the first photographed object in the first image by using the second pixel data, thereby improving the display effect of the image.

[0123] In addition, when the voltage signal value indicated by the third pixel data is less than the overexposure threshold, it indicates that the third pixel data is valid data, and the pixel circuit can maintain the second readout mode without switching the reading mode.

[0124] When the pixel circuit sequentially outputs the fourth pixel data, the third pixel data, and the first pixel data in the second readout mode, the image processing method provided by the embodiments of the present application may further include: Obtain the fourth pixel data, the third pixel data, and the first pixel data output by the pixel circuit in the second readout mode; Determine the difference between the voltage signal value indicated by the third pixel data and the voltage signal value indicated by the fourth pixel data; Generate a first image based on the first pixel data; Generate a second image based on the third pixel data; Focus on the second photographed object in the second image based on the difference between the voltage signal value indicated by the third pixel data and the voltage signal value indicated by the fourth pixel data and the fourth pixel data.

[0125] For example, for a pixel circuit in an image sensor, the pixel circuit corresponds to Figure 3BIn the region where the four blue filter elements B1 shown are located, the voltage signal value indicated by the third pixel data output by the pixel circuit is 800 LSB, the voltage signal value indicated by the fourth pixel data is 300 LSB, and the difference between the voltage signal value indicated by the third pixel data and the voltage signal value indicated by the fourth pixel data is 500 LSB. The image processing unit can calculate the phase difference result of the blue channel based on the calculated difference of 500 LSB and the voltage signal value of 300 LSB indicated by the fourth pixel data. Based on the phase difference result, the focus offset is determined, and based on the focus offset, the second photographed object in the second image is focused. Among them, the second photographed object can be an object to be photographed such as a portrait, a building, an item, etc., and the second photographed object can be located at various different positions in the second image.

[0126] In this way, in a low-light scene, the embodiment of the present application can generate a second image by using the third pixel data and effectively focus on the second photographed object in the second image by using the fourth pixel data, so that the image of the second photographed object in the second image is clear, thereby improving the image quality of the second image.

[0127] In this way, in a high dynamic range scene, the embodiment of the present application can obtain an image with a large dynamic range, enabling both the HCG image and the LCG image to have omnidirectional full-pixel PDAF information. In a high dynamic range scene, the camera can not only have good HDR image quality but also quickly and accurately focus on the photographed objects in bright and dark scenes, enhancing the user experience of using the camera.

[0128] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "schematic embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0129] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and purposes of the present application. The scope of the present application is defined by the claims and their equivalents.

Claims

1. An image sensor, characterized in that, It includes a pixel circuit, an image processing unit, and a timing control unit; the pixel circuit is respectively connected to the image processing unit and the timing control unit, and the image processing unit is connected to the timing control unit; The pixel circuit includes a photoelectric conversion module and a capacitor module, the photoelectric conversion module includes a first switch and a second switch, and the capacitor module includes a switching switch; the timing control unit is respectively connected to the first switch, the second switch, and the switching switch; When the first switch, the second switch, and the switching switch are all in the conducting state, the first pixel data output by the pixel circuit is used to generate a first image; When the first switch and the switching switch are in the conducting state and the second switch is in the off state, the second pixel data output by the pixel circuit is used to focus on a first photographed object in the first image.

2. The image sensor according to claim 1, wherein The timing control unit includes a first timing controller and a second timing controller; the image processing unit is connected to the first timing controller, and the first timing controller is connected to the switching switch; the second timing controller is respectively connected to the first switch and the second switch.

3. The image sensor according to claim 2, wherein The first time point is the same as the second time point, and the third time point is later than the second time point; Wherein, the first time point is the starting time point when the first timing controller controls the switching switch to conduct, the second time point is the starting time point when the second timing controller controls the first switch to conduct, and the third time point is the starting time point when the second timing controller controls the second switch to conduct.

4. The image sensor according to claim 3, wherein The second timing controller is connected to the switching switch; The fourth time point is later than the third time point; wherein, the fourth time point is the starting time point when the second timing controller controls the switching switch to conduct.

5. The image sensor according to claim 4, characterized in that, The control priority of the first timing controller over the switching switch is higher than that of the second timing controller over the switching switch; Wherein, the first control instruction output by the first timing controller to the switching switch is used to overwrite the second control instruction output by the second timing controller to the switching switch.

6. The image sensor according to claim 2, characterized in that, The image sensor further includes an analog-to-digital converter, a first end of the analog-to-digital converter is connected to the pixel circuit, a second end of the analog-to-digital converter is connected to the image processing unit, and a third end of the analog-to-digital converter is connected to the first timing controller.

7. The image sensor according to claim 2, wherein, When the first switch and the second switch are in the conducting state and the switching switch is in the off state, the third pixel data output by the pixel circuit is used to generate a second image; When the first switch is in the conducting state and the second switch and the switching switch are in the off state, the fourth pixel data output by the pixel circuit is used to focus on a second photographed object in the second image.

8. The image sensor according to claim 7, wherein, The pixel circuit has a first read mode and a second read mode; the first read mode is a mode in which the switching switch is controlled by a first timing controller; the second read mode is a mode in which the switching switch is controlled by the second timing controller; wherein, the pixel circuit is used to output the first pixel data and the second pixel data in the first read mode; the pixel circuit is used to output the first pixel data, the third pixel data and the fourth pixel data in the second read mode.

9. The image sensor according to claim 8, wherein, When the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, the image processing unit is used to control the pixel circuit to switch from the second read mode to the first read mode.

10. The image sensor according to claim 7, wherein when the switching switch is in the on state, the image sensor has a first conversion gain, and when the switching switch is in the off state, the image sensor has a second conversion gain, and the first conversion gain is less than the second conversion gain; the first conversion gain of the image sensor is used for the pixel circuit to output the first pixel data and the second pixel data, and the second conversion gain of the image sensor is used for the pixel circuit to output the third pixel data and the fourth pixel data.

11. An electronic device, characterized in that, comprising a processor and the image sensor according to any one of claims 1-10.

12. An image processing method, performed by the electronic device according to claim 11, characterized in that, The method includes: acquiring first pixel data and second pixel data output by a pixel circuit; calculating a difference between a voltage signal value indicated by the first pixel data and a voltage signal value indicated by the second pixel data; generating a first image based on the first pixel data; focusing on a first object to be photographed in the first image based on the difference and the second pixel data.

13. The method according to claim 12, characterized in that, Before acquiring the first pixel data and the second pixel data output by the pixel circuit, the method further includes: acquiring third pixel data output by the pixel circuit in the second read mode; when the voltage signal value indicated by the third pixel data is greater than the overexposure threshold, switching the pixel circuit from the second read mode to the first read mode; wherein, the first pixel data and the second pixel data are acquired in the first read mode.

Images