Image sensors, camera modules and electronic devices
By introducing a pixel comparison module and a processing module into the image sensor, the edge detection function of the image sensor is realized, which solves the problem of single function in the existing technology and improves the multifunctionality of the image sensor.
Patent Information
- Application Number
- CN202611338178.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-31
- Publication Date
- 2026-11-17
AI Technical Summary
Existing image sensors based on LOFIC pixel circuits have relatively limited functionality, only able to output image signals and lacking edge detection capabilities.
A pixel comparison module is introduced into the image sensor to generate an edge detection signal by comparing the electrical signals of adjacent pixel units. This signal is then combined with the processing module to generate an image data signal, thereby achieving the edge detection function.
Image sensors not only output image data, but also have edge detection capabilities, which enriches the functions of image sensors and improves the accuracy of edge detection.
Smart Images

Figure CN123123754A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of electronic equipment technology, specifically relating to an image sensor, a shooting module, and an electronic device. Background Technology
[0002] Currently, HDR (High Dynamic Range) technology based on LOFIC (Lateral Overflow Integration Capacitor) pixel circuits is widely used.
[0003] However, within a LOFIC pixel circuit, the LOFIC capacitor waits for the PPD (Pinned-Photodiode) to fill its well before accepting electrons overflowing from the PPD during each frame's exposure period. The LOFIC capacitor's function is solely to expand the PPD's full-well capacity during the exposure period; it remains idle after the previous frame is read until it accepts electrons overflowing from the PPD. Furthermore, within a LOFIC pixel circuit, the electrical signals accumulated in the PPD and LOFIC capacitor are only read and output as image signals in each frame. Thus, the function of an image sensor based on a LOFIC pixel circuit is relatively limited. Summary of the Invention
[0004] The purpose of this application is to provide an image sensor, a shooting module, and an electronic device that enables the image sensor to have both image output and edge detection functions, thereby enriching the functionality of the image sensor.
[0005] In a first aspect, embodiments of this application provide an image sensor, comprising: a pixel array including: M pixel units, each pixel unit being used to acquire light signals and output electrical signals; M being a positive integer; N pixel comparison modules, where N is a positive integer and N is less than M; each pixel comparison module being connected to multiple adjacent pixel units; each pixel comparison module being used to compare the electrical signals output by multiple adjacent pixel units and output a pixel comparison signal; a processing module being connected to each pixel unit and each pixel comparison module; the processing module being used to generate an image data signal based on the electrical signals and to generate an edge detection signal based on the pixel comparison signal; the image data signal being used to generate an image frame; and the edge detection signal being used to indicate the edge position information of each pixel unit in the image frame.
[0006] Secondly, embodiments of this application provide a shooting module, including: an image sensor as described in the first aspect.
[0007] Thirdly, embodiments of this application provide an electronic device, including: the shooting module of the second aspect.
[0008] The image sensor provided in this application includes a pixel array and a processing module. The pixel array includes M pixel units and N pixel comparison modules. M and N are both positive integers, and N is less than M. Pixel units are used to collect light signals and output electrical signals. Each pixel comparison module is connected to multiple adjacent pixel units. The pixel comparison module compares the electrical signals output by multiple adjacent pixel units and outputs a pixel comparison signal. The processing module is connected to each pixel unit and each pixel comparison module. The processing module generates an image data signal based on the electrical signals and generates an edge detection signal based on the pixel comparison signal. The image data signal is used to generate an image frame. The edge detection signal indicates the edge position information of each pixel unit in the image frame. By adding pixel comparison modules between the pixel units, the image sensor can output both image data signals and edge detection signals simultaneously. This enriches the functionality of the image sensor, enabling it to perform both image output and edge detection functions. Attached Figure Description
[0009] Figure 1 Schematic diagrams of the structure of an image sensor provided for some embodiments of this application;
[0010] Figure 2 Partial schematic diagrams of image sensors provided for some embodiments of this application;
[0011] Figure 3 Partial schematic diagrams of image sensors provided for some embodiments of this application;
[0012] Figure 4 Partial schematic diagrams of image sensors provided for some embodiments of this application;
[0013] Figure 5 Partial schematic diagrams of image sensors provided for some embodiments of this application;
[0014] Figure 6 A schematic diagram of pixel units provided for some embodiments of this application;
[0015] Figure 7 A schematic diagram of pixel units provided for some embodiments of this application;
[0016] Figure 8 A schematic diagram of pixel units provided for some embodiments of this application;
[0017] Figure 9 Timing diagrams of the image sensor operation provided for some embodiments of this application;
[0018] Figure 10A schematic diagram of electron flow direction of a pixel unit provided for some embodiments of this application;
[0019] Figure 11 A schematic diagram of electron flow direction of a pixel unit provided for some embodiments of this application;
[0020] Figure 12 A schematic diagram of electron flow direction of a pixel unit provided for some embodiments of this application;
[0021] Figure 13 Timing diagrams of pixel units provided for some embodiments of this application;
[0022] Figure 14 Schematic diagrams of the working principle of pixel units provided for some embodiments of this application;
[0023] Figure 15 Schematic diagrams of the working principle of pixel units provided for some embodiments of this application;
[0024] Figure 16 Schematic diagrams of the working principle of pixel units provided for some embodiments of this application;
[0025] Figure 17 Schematic diagrams of the working principle of pixel units provided for some embodiments of this application;
[0026] Figure 18 Schematic diagrams of the structure of an image sensor provided for some embodiments of this application;
[0027] Figure 19 A schematic diagram of a pixel layer chip provided for some embodiments of this application;
[0028] Figure 20 Schematic diagram of pixel circuit mezzanine provided for some embodiments of this application;
[0029] Figure 21 A schematic diagram of a logic layer chip provided for some embodiments of this application;
[0030] Figure 22 Timing diagrams of the image sensor operation provided for some embodiments of this application;
[0031] Figure 23 Schematic diagrams of edge detection of image sensors provided for some embodiments of this application;
[0032] Figure 24 Schematic diagrams of edge detection of image sensors provided for some embodiments of this application;
[0033] Figure 25 Schematic diagrams of edge detection of image sensors provided for some embodiments of this application;
[0034] Figure 26 Structural block diagrams of the shooting module provided for some embodiments of this application;
[0035] Figure 27 Structural block diagrams of electronic devices provided for some embodiments of this application.
[0036] Figure label:
[0037] 100 - Image sensor, 102 - Pixel array, 104 - Pixel unit, 106 - Pixel comparator module, 108 - Processing module, 110 - Signal comparison circuit, 112 - Row comparator, 114 - Column comparator, 116 - Signal buffer circuit, 118 - First driver, 120 - Second driver, 122 - Column parallel analog-to-digital converter module, 124 - Buffer and column scanner module, 126 - Image signal processing module, 128 - Pixel layer chip, 130 - Pixel circuit mezzanine, 132 - Logic layer chip, C LOFIC - Integrating capacitor, C DCG -First floating diffusion capacitance, C FD - Second floating diffusion capacitor, T1 - Row selection transistor, T2 - Second transistor, T3 - Third transistor, T4 - Fourth transistor, T5 - Fifth transistor, T6 - Sixth transistor, VDD - Power supply, PPD - Photodiode, 200 - Camera module, 300 - Electronic equipment. Detailed Implementation
[0038] The embodiments of this application will now be described in detail. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0039] The terms "first" and "second" in the specification and claims of this application may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise stated, "a plurality of" means two or more.
[0040] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "connected" and "linked" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.
[0041] The following is combined Figures 1-27 The image sensor, imaging module, and electronic device according to embodiments of this application are described in detail.
[0042] like Figure 1 As shown, this application embodiment provides an image sensor 100. The image sensor 100 includes a pixel array 102 and a processing module 108.
[0043] The pixel array 102 includes M pixel units 104 and N pixel comparison modules 106.
[0044] Where M and N are both positive integers, and N is less than M.
[0045] Optionally, the pixel unit 104 is used to collect optical signals and output electrical signals.
[0046] Optionally, pixel unit 104 may specifically be a LOFIC pixel circuit, such as... Figures 6 to 8 As shown, the LOFIC pixel circuit includes a photodiode PPD and an integrating capacitor C. LOFIC The first transistor is the selection transistor T1, the second transistor T2, the third transistor T3, the fourth transistor T4, the fifth transistor T5, and the sixth transistor T6.
[0047] The LOFIC pixel circuit includes three configurations. In configuration 1, such as... Figure 6 As shown, the first terminal of the row selection transistor T1 is used to output a photoelectric signal. The second terminal of the row selection transistor T1 is connected to the first terminal of the third transistor T3. The second terminal of the third transistor T3 is connected to the power supply VDD. The control terminal of the third transistor T3 is connected to the first terminals of both the fourth transistor T4 and the fifth transistor T5. The second terminal of the fourth transistor T4 is connected to the cathode of the photodiode PPD. The second terminal of the fifth transistor T5 is connected to the first terminal of the second transistor T2. The second terminal of the second transistor T2 is connected to the integrating capacitor C. LOFIC The first terminal of the transistor and the first terminal of the sixth transistor T6 are both connected. The second terminal of the sixth transistor T6 is connected to the power supply VDD. The integrating capacitor C LOFICBoth the second terminal of the fifth transistor T5 and the anode of the photodiode PPD are grounded. A first floating diffusion capacitance C is formed between the second terminal of the fifth transistor T5 and the first terminal of the second transistor T2. DCG A second floating diffusion capacitor C is formed between the control terminal of the third transistor T3, the first terminal of the fourth transistor T4, and the first terminal of the fifth transistor T5. FD .
[0048] In configuration 2, such as Figure 7 As shown, the first terminal of the row selection transistor T1 is used to output a photoelectric signal. The second terminal of the row selection transistor T1 is connected to the first terminal of the third transistor T3. The second terminal of the third transistor T3 is connected to the power supply VDD. The control terminal of the third transistor T3 is connected to the first terminals of the fourth transistor T4, the fifth transistor T5, and the second transistor T2. The second terminal of the fourth transistor T4 is connected to the cathode of the photodiode PPD. The second terminal of the fifth transistor T5 is connected to the first terminal of the sixth transistor T6. The second terminal of the second transistor T2 is connected to the integrating capacitor C. LOFIC The first terminal is connected, the second terminal of the sixth transistor T6 is connected to the power supply VDD, and the integrating capacitor C... LOFIC Both the second terminal of the fifth transistor T5 and the anode of the photodiode PPD are grounded. A first floating diffusion capacitance C is formed between the second terminal of the fifth transistor T5 and the first terminal of the sixth transistor T6. DCG A second floating diffusion capacitor C is formed between the control terminal of the third transistor T3, the first terminal of the fourth transistor T4, the first terminal of the second transistor T2, and the first terminal of the fifth transistor T5. FD .
[0049] In configuration 3, such as Figure 8 As shown, the first terminal of the row selection transistor T1 is used to output a photoelectric signal. The second terminal of the row selection transistor T1 is connected to the first terminal of the third transistor T3. The second terminal of the third transistor T3 is connected to the power supply VDD. The control terminal of the third transistor T3 is connected to the first terminals of both the fourth transistor T4 and the fifth transistor T5. The second terminal of the fourth transistor T4 is connected to the cathode of the photodiode PPD and the first terminal of the second transistor T2. The second terminal of the fifth transistor T5 is connected to the first terminal of the sixth transistor T6. The second terminal of the second transistor T2 is connected to the integrating capacitor C. LOFIC The first terminal is connected, the second terminal of the sixth transistor T6 is connected to the power supply VDD, and the integrating capacitor C... LOFIC Both the second terminal of the fifth transistor T5 and the anode of the photodiode PPD are grounded. A first floating diffusion capacitance C is formed between the second terminal of the fifth transistor T5 and the first terminal of the sixth transistor T6. DCGA second floating diffusion capacitor C is formed between the control terminal of the third transistor T3, the first terminal of the fourth transistor T4, and the first terminal of the fifth transistor T5. FD .
[0050] Optionally, the integrating capacitor C LOFIC It is used to receive and buffer electrons overflowing from the photodiode PPD. When the second transistor T2 is closed, electrons will freely enter and exit the integrating capacitor C. LOFIC And charge and discharge it.
[0051] Optionally, the integrating capacitor C LOFIC As an independent capacitor device, the integrating capacitance C LOFIC It has independent positive and negative plates and dielectric filling material. Therefore, the integrating capacitor C LOFIC Its capacitance is enormous, typically the second floating diffusion capacitor C. FD Hundreds of times the capacity.
[0052] Based on the integrating capacitor C LOFIC LOFIC pixel circuitry can enable HDR technology. Specifically, such as... Figures 9 to 12 As shown, during the exposure time of one frame, if pixel unit 104 encounters strong light, the photodiode PPD within pixel unit 104 will fill its well after a certain period of time, meaning that electrons will fill the built-in capacitance of the photodiode PPD. After the photodiode PPD is filled, it will continue to generate electrons, but excess electrons can flow out of the photodiode PPD through a partially closed transistor.
[0053] Specifically, in configuration 1, such as Figure 10 As shown, when the fourth transistor T4, the fifth transistor T5, and the second transistor T2 are closed, the overflowing electrons will be buffered by the second floating diffusion capacitor C. FD First floating diffusion capacitor C DCG With integrating capacitor C LOFIC In a super-large capacitor formed by parallel connections. In configuration 2, as... Figure 11 As shown, when the fourth transistor T4 and the second transistor T2 are closed, the overflowing electrons will be buffered by the second floating diffusion capacitor C. FD With integrating capacitor C LOFIC In a super-large capacitor formed by parallel connections. In configuration 3, such as Figure 12 As shown, when the second transistor T2 is closed, the overflowing electrons will only be buffered in the integrating capacitor C. LOFIC middle.
[0054] Based on this, during the reading time of one frame, the second transistor T2 is turned off, and electrons in the photodiode PPD will be introduced into the second floating diffusion capacitor C. FD and the first floating diffusion capacitor C DCGThe reading is then performed by the third transistor T3, and the second transistor T2 closes again, allowing the third transistor T3 to read the integrating capacitor C. LOFIC The voltage drop generated by the overflowing electrons serves as the electrical signal of pixel unit 104 under strong light.
[0055] Among them, electrons generated by the photodiode PPD after photosensitive activity overflow into the integrating capacitor C after the trap is full. LOFIC In the middle. During the reading time of one frame, the integrating capacitor C LOFIC It can still accept all the electrons transferred out by the photodiode PPD. Therefore, the integrating capacitor C LOFIC Its capacity is much greater than the full-well capacity of a photodiode (PPD).
[0056] In particular, to implement LOFIC HDR technology based on LOFIC pixel circuitry, the image sensor 100 needs to perform multiple signal reads during the read-out phase of each frame.
[0057] Specifically, taking configuration 1 as an example, such as Figures 13 to 17 As shown, the working timing of the LOFIC pixel circuit includes an exposure stage, a readout stage, and a reset stage. The readout stage includes a noise sampling stage, a noise + signal sampling stage, and a LOFIC signal sampling stage.
[0058] In the noise sampling phase, such as Figure 14 As shown, the fourth transistor T4 and the second transistor T2 are off, the fifth transistor T5 is on, and the third transistor T3 samples the output V. PIX The voltage is an LCG (Low Conversion Gain) noise signal, conducted by column wires to the downstream circuitry of this column, such as an ADC (Analog-to-Digital Converter), for recording and processing. Based on this, the fifth transistor T5 is turned off, and the third transistor T3 samples the output V. PIX The voltage is an HCG (High Conversion Gain) noise signal, which is also transferred to the back-end circuit for processing and recording via the column wires.
[0059] Optionally, in the noise + signal sampling stage, such as Figure 15 As shown, when the fourth transistor T4 is closed, electrons are transferred from the photodiode PPD to the second floating diffusion capacitor C. FD In the middle, the third transistor T3 samples the output V PIXThe voltage is an HCG signal, which is conducted by the column wires to the back-end circuit of this column for recording and processing, while simultaneously performing CDS (Correlated Double Sampling) noise reduction on the HCG. Based on this, the fifth transistor T5 closes, and electrons are transferred from the photodiode PPD to the second floating diffusion capacitor C. FD With the first floating diffusion capacitor C DCG In the middle. At this time, the third transistor T3 samples the output V. PIX The voltage is an LCG signal, which is transmitted by the column wires to the back-end circuit of this column for recording and processing, while simultaneously performing CDS noise reduction on the LCG.
[0060] Optionally, during the LOFIC signal sampling stage, such as Figure 16 As shown, when the second transistor T2 is closed, electrons are transferred from the photodiode PPD to the second floating diffusion capacitor C. FD Integrating capacitor C LOFIC With the first floating diffusion capacitor C DCG Within the super-large capacitor formed. At this time, the V output sampled by the third transistor T3. PIX The voltage is a LOFIC signal, which is conducted by the column conductor to the back-end circuit of this column for processing. Since the LOFIC noise signal was not sampled in the previous steps, the LOFIC signal in this stage contains noise. After the LOFIC signal is sampled, the LOFIC pixel circuit enters a selectable idle time to wait for the next frame to start.
[0061] Based on this, during the reset phase, such as Figure 17 As shown, at the start of the next frame, the sixth transistor T6 closes, the power supply VDD clears all electrons in the LOFIC pixel circuit, and the photodiode PPD is reset to its clamping voltage. After the reset and clearing are complete, the LOFIC pixel circuit enters the exposure period of the next frame.
[0062] Optionally, such as Figure 2 As shown, each pixel comparison module 106 is connected to multiple adjacent pixel units 104.
[0063] Optionally, the pixel comparison module 106 is used to compare the electrical signals output by multiple adjacent pixel units 104 and output a pixel comparison signal.
[0064] Optionally, the processing module 108 is connected to each pixel unit 104 and each pixel comparison module 106.
[0065] Optionally, the processing module 108 is used to generate an image data signal based on an electrical signal and to generate an edge detection signal based on a pixel comparison signal.
[0066] In the case where the pixel array 102 contains m rows and n columns of pixel units 104, since the pixel comparison module 106 compares the electrical signals output by adjacent pixels, the resolution of the edge detection signal is m-1 rows and n-1 columns, where m and n are both positive integers.
[0067] Optionally, the image data signal is used to generate an image frame.
[0068] Optionally, the edge detection signal is used to indicate the edge position information of each pixel unit 104 in the image frame.
[0069] Optionally, the edge detection signal can provide metadata support for various types of machine recognition, motion monitoring, and image algorithm applications, so as to provide users with more novel computational imaging applications without affecting the image quality of the image sensor 100.
[0070] This allows the image sensor 100 to have both image output and edge detection capabilities, enriching its functionality. Furthermore, embedding the pixel comparison module 106 between the pixel units 104 within the pixel array 102 does not alter the structure and operation of the LOFIC pixel circuit, facilitating pixel miniaturization and customized design.
[0071] Optionally, such as Figure 1 As shown, the processing module 108 includes a column-parallel analog-to-digital converter module 122, a buffer and column scanner module 124, and an image signal processing module 126.
[0072] Specifically, the image signal processing module 126 can be an ISP (Image Signal Processor). The ISP can be integrated inside the image sensor 100 or located outside the image sensor 100, without any specific restrictions.
[0073] Optionally, the electrical signal is transmitted in parallel to the parallel analog-to-digital converter module 122 for parallel signal processing and converted into a digital signal. The digital signal is then scanned column by column by the buffer and column scanner module 124 and output to the image signal processing module 126. The image signal processing module 126 processes the digital signal and outputs the image data signal.
[0074] Optionally, the pixel comparison signal is transmitted to the column parallel analog-to-digital converter module 122 in a column parallel manner for parallel signal processing, converted into a digital signal, and scanned and output by the buffer and column scanner module 124 as an edge detection signal.
[0075] In practical applications, for each pixel unit 104, the electrical signal output for imaging can be called the imaging signal, and the electrical signal for pixel comparison can be called the afterimage signal. After each pixel unit 104 outputs the imaging signal to the processing module 108, the afterimage signal is then input to the pixel comparison module 106 for comparison to obtain the pixel comparison signal.
[0076] In addition, the pixel comparison module 106 is relatively independent of the circuit structure in each pixel unit 104.
[0077] Therefore, the stacked structure of the image sensor 100 can adopt Figure 18 The three-layer stack shown specifically includes a pixel layer chip 128, a pixel circuit mezzanine 130, and a logic layer chip 132 in the image sensor 100.
[0078] Among them, such as Figure 19 As shown, the pixel layer chip 128 adopts the same layout structure as a conventional LOFIC pixel chip, including an optical structure and a pixel array 102. The output signal of the pixel array 102 is transmitted to the corresponding module in the bottom logic layer chip 132 through a TSV (Through-Silicon Via) or Cu-Cu (copper-copper) bonding structure located at the edge.
[0079] Optionally, such as Figure 20 As shown, the pixel circuit mezzanine 130 is used to house the newly added pixel comparison module 106. The pixel comparison module 106 is electrically connected to the pixel layer chip 128 via Cu-Cu bonding. The output signal of the pixel comparison module 106 is also transmitted to the corresponding module in the bottom logic layer chip 132 via TSV or Cu-Cu bonding structure.
[0080] Optionally, such as Figure 21 As shown, the logic layer chip 132 includes a column parallel analog-to-digital converter module 122, a buffer and column scanner module 124, an image signal processing module 126, a first driver 118, and a second driver 120. The logic layer chip 132 is used to receive the output signals from the upper two layers and generate corresponding drive signals.
[0081] An image sensor 100 according to an embodiment of this application includes a pixel array 102 and a processing module 108. The pixel array 102 includes M pixel units 104 and N pixel comparison modules 106. M and N are both positive integers, and N is less than M. Pixel units 104 are used to acquire light signals and output electrical signals. Each pixel comparison module 106 is connected to multiple adjacent pixel units 104. The pixel comparison module 106 is used to compare the electrical signals output by multiple adjacent pixel units 104 and output a pixel comparison signal. The processing module 108 is connected to each pixel unit 104 and each pixel comparison module 106. The processing module 108 is used to generate an image data signal based on the electrical signals and to generate an edge detection signal based on the pixel comparison signal. The image data signal is used to generate an image frame. The edge detection signal is used to indicate the edge position information of each pixel unit 104 in the image frame. By adding pixel comparison modules 106 between the pixel units 104, the image sensor 100 can output both image data signals and edge detection signals simultaneously. This allows the image sensor 100 to have both image output and edge detection capabilities, thus enriching the functionality of the image sensor.
[0082] According to some embodiments of this application, optionally, the pixel comparison signal includes a row comparison signal and a column comparison signal.
[0083] Based on this, such as Figure 3 As shown, the pixel comparison module 106 includes a signal comparison circuit 110, which includes a row comparator 112 and a column comparator 114.
[0084] The row comparator 112 is connected to two adjacent pixel units 104 in the same row.
[0085] Optionally, the column comparator 114 is connected to two adjacent pixel units 104 in the same column.
[0086] Optionally, the row comparator 112 is used to compare the electrical signals output by two adjacent pixel units 104 row by row, and output a row comparison signal.
[0087] For example, the row comparator 112 is used to compare the electrical signal output by the pixel unit 104 in the Xth row and Yth column with the electrical signal output by the pixel unit 104 in the Xth row and Y+1th column. Here, X and Y are both positive integers.
[0088] Optionally, the column comparator 114 is used to compare the electrical signals output by two adjacent pixel units 104 column by column, and output a column comparison signal.
[0089] For example, column comparator 114 is used to compare the electrical signal output by pixel unit 104 in row X and column Y, and the electrical signal output by pixel unit 104 in row X+1 and column Y.
[0090] Optionally, all row comparison signals and column comparison signals are transmitted to the processing module 108 through a shared column wire. If the row comparison signals and column comparison signals are analog signals, they need to be converted into digital signals by the column parallel analog-to-digital converter module 122. Otherwise, if the row comparison signals and column comparison signals are digital signals or high / low level signals, such as binary signals 1 or 0, they are directly buffered by the buffer and column scanner module 124 and scanned out as edge detection signals.
[0091] According to the image sensor 100 of this application embodiment, the pixel comparison signal includes a row comparison signal and a column comparison signal. The pixel comparison module 106 includes a signal comparison circuit 110, which includes a row comparator 112 and a column comparator 114. The row comparator 112 is connected to two adjacent pixel units 104 in the same row. The column comparator 114 is connected to two adjacent pixel units 104 in the same column. The row comparator 112 is used to compare the electrical signals output by two adjacent pixel units 104 row by row and output a row comparison signal. The column comparator 114 is used to compare the electrical signals output by two adjacent pixel units 104 column by column and output a column comparison signal. This achieves pixel comparison in both row and column directions, thereby enabling edge detection in both row and column directions and improving the accuracy of edge detection.
[0092] According to some embodiments of this application, optionally, such as Figure 3 As shown, each pixel unit 104 includes an integrating capacitor C. LOFIC .
[0093] Optionally, both row comparator 112 and column comparator 114 are connected to the integrating capacitor C in pixel unit 104. LOFIC connect.
[0094] Based on this, the electrical signal includes the integrating capacitor C. LOFIC The first voltage signal on.
[0095] Among them, due to the integrating capacitor C LOFIC During the exposure period of each frame, the photodiode PPD must be fully charged before the overflowing electrons can be received. Therefore, the pixel comparison period can exist within the reading period of each frame or across frames in the next frame. As long as it does not occupy the time period after the photodiode PPD is fully charged in the exposure period of the next frame, it will not affect the application of LOFIC HDR technology.
[0096] Additionally, in practical applications, if the LOFIC HDR function of the image sensor 100 is not activated, or the integrating capacitor C... LOFIC If not used as a LOFIC function, there is no impact; in this case, the pixel comparison period can extend into any period of the next frame.
[0097] For example, such as Figure 22 As shown, the operation of the image sensor 100 includes the following stages during each frame reading period:
[0098] Noise reduction sampling stage: As mentioned above, LCG noise is sampled first, followed by HCG noise sampling.
[0099] Pixel signal reading stage: After noise reduction sampling is completed, HCG signal is read first. After HCG signal reading and analog-to-digital conversion are completed, LCG signal reading and analog-to-digital conversion are performed.
[0100] LOFIC signal readout stage: After the pixel signal readout is completed, the LOFIC signal is read and analog-to-digital conversion is performed. After the LOFIC signal readout is completed, the LOFIC signal is not cleared, but is continuously buffered in the integrating capacitor C. LOFIC The internal signal serves as a residual image signal.
[0101] Afterimage calculation stage (i.e., pixel comparison stage): After the pre-processing stage is completed, starting from the second row of pixel units 104, the afterimage signal of each pixel unit 104 is output to the corresponding pixel comparison module 106. Among these, in the integrating capacitor C... LOFIC The afterimage signal will be continuously buffered until the next row of pixel units 104 finishes outputting the afterimage signal before subsequent reset and clearing.
[0102] According to an embodiment of the image sensor 100 of this application, each pixel unit 104 includes an integrating capacitor C. LOFIC Both row comparator 112 and column comparator 114 are connected to the integrating capacitor C in pixel unit 104. LOFIC Connection. The electrical signal includes the integrating capacitor C. LOFIC The first voltage signal is applied. Thus, the integrating capacitor C in each pixel unit 104 is reused. LOFIC The first voltage signal is compared with the pixel to obtain the edge detection signal. While realizing the edge detection function, it does not affect the operation of the LOFIC pixel circuit, the implementation of LOFIC HDR technology and image quality.
[0103] According to some embodiments of this application, optionally, such as Figure 4 and Figure 5 As shown, the pixel comparison module 106 also includes a signal buffer circuit 116.
[0104] The signal buffer circuit 116 is connected between the pixel unit 104 and the signal comparison circuit 110.
[0105] Optionally, the signal buffer circuit 116 is used to buffer the electrical signal and preprocess the electrical signal.
[0106] The electrical signal can be an analog signal or a digital signal, and the preprocessing includes at least one of the following: amplification, noise reduction, and analog-to-digital conversion.
[0107] Alternatively, amplification can be achieved using an analog amplifier, noise reduction can be achieved using a filter circuit, and analog-to-digital conversion can be achieved using an ADC.
[0108] If the preprocessing includes analog-to-digital conversion, the electrical signal will be converted into a K-bit digital signal, such as a K-bit binary signal of 1 or 0, where K is a positive integer. Accordingly, the row comparator 112 and column comparator 114 mentioned above are also digital comparators. The electrical signal after the analog-to-digital conversion is completed will be cached in a cache submodule composed of static random output memory.
[0109] Based on this, by adding a signal buffer circuit 116 to the pixel comparison module 106, the pixel comparison module 106 can flexibly select and connect to multiple nodes in the pixel unit 104. Furthermore, by adding the signal buffer circuit 116, the LOFIC pixel circuit does not need to be modified, which is beneficial for the miniaturization design of pixels.
[0110] According to the image sensor 100 of this application embodiment, the pixel comparison module 106 further includes a signal buffer circuit 116. The signal buffer circuit 116 is connected between the pixel unit 104 and the signal comparison circuit 110. The signal buffer circuit 116 is used to buffer electrical signals and preprocess them. The preprocessing includes at least one of the following: amplification, noise reduction, and analog-to-digital conversion. Thus, by adding the signal buffer circuit 116 between the pixel unit 104 and the signal comparison circuit 110, buffering and preprocessing of the electrical signals output by the pixel unit 104 can be achieved, allowing the pixel comparison module 106 to flexibly select and connect to multiple nodes within the pixel unit 104.
[0111] According to some embodiments of this application, optionally, such as Figure 5 As shown, each pixel unit 104 includes an integrating capacitor C. LOFIC .
[0112] Optionally, the signal buffer circuit 116 and the integrating capacitor C in the pixel unit 104 LOFIC connect.
[0113] Based on this, the electrical signal includes the integrating capacitor C. LOFIC The first voltage signal on.
[0114] According to an embodiment of the image sensor 100 of this application, each pixel unit 104 includes an integrating capacitor C. LOFIC The signal buffer circuit 116 and the integrating capacitor C in the pixel unit 104 LOFIC Connection. The electrical signal includes the integrating capacitor C. LOFIC The first voltage signal is applied. Thus, the integrating capacitor C in each pixel unit 104 is reused. LOFIC The first voltage signal is compared with the pixels to obtain the edge detection signal. While realizing the edge detection function, it does not affect the operation of the LOFIC pixel circuit, the implementation of LOFICHDR technology and image quality.
[0115] According to some embodiments of this application, optionally, such as Figure 4 As shown, each pixel unit 104 includes a row selection transistor T1, i.e., the first transistor.
[0116] Optionally, the signal buffer circuit 116 is connected to the row selection transistor T1 in the pixel unit 104.
[0117] Based on this, the electrical signal includes the second voltage signal output by the row selection transistor T1.
[0118] According to the image sensor 100 of this application embodiment, each pixel unit 104 includes a row selection transistor T1. A signal buffer circuit 116 is connected to the row selection transistor T1 in the pixel unit 104. The electrical signal includes a second voltage signal output by the row selection transistor T1. Thus, pixel comparison is performed based on the second voltage signal output by the row selection transistor T1 in each pixel unit 104 to obtain an edge detection signal. While achieving edge detection, this does not affect the operation of the LOFIC pixel circuit, the implementation of LOFIC HDR technology, or image quality.
[0119] According to some embodiments of this application, optionally, when the electrical signal amplitude of the first pixel unit is greater than the electrical signal amplitude of the adjacent pixel unit 104, the pixel comparison signal is a first mode signal; when the electrical signal amplitude of the first pixel unit is less than or equal to the electrical signal amplitude of the adjacent pixel unit 104, the pixel comparison signal is a second mode signal.
[0120] The first pixel unit is any one of the M pixel units 104.
[0121] Optionally, the first mode signal may be the difference between the electrical signal amplitude of the first pixel unit and the electrical signal amplitude of the adjacent pixel unit 104, such as 1.5V, or it may be a high-level signal or a binary signal 1, without any specific limitations.
[0122] Optionally, the second mode signal may be the difference between the electrical signal amplitude of the first pixel unit and the electrical signal amplitude of the adjacent pixel unit 104, such as -1.2V, or it may be a low-level signal or a binary signal 0, without any specific restrictions.
[0123] According to the image sensor 100 of this application embodiment, when the electrical signal amplitude of the first pixel unit is greater than the electrical signal amplitude of the adjacent pixel unit 104, the pixel comparison signal is a first mode signal; when the electrical signal amplitude of the first pixel unit is less than or equal to the electrical signal amplitude of the adjacent pixel unit 104, the pixel comparison signal is a second mode signal. The first pixel unit is any one of the M pixel units 104. Thus, since the electrical signal amplitude of a pixel unit is related to the light signal intensity, and the light signal intensity can reflect the occlusion of an object, by comparing the electrical signal amplitudes of any pixel unit with those of its adjacent pixel units, the positional relationship between any pixel unit and the object can be determined, thereby achieving object edge detection and improving the accuracy of edge detection.
[0124] According to some embodiments of this application, optionally, for the same row of pixel units 104:
[0125] When the row comparison signal between the first pixel unit and the left pixel unit 104 is a second mode signal, and the row comparison signal between the first pixel unit and the right pixel unit 104 is a first mode signal, the edge detection signal corresponding to the first pixel unit is a first value.
[0126] Optionally, when the row comparison signal between the first pixel unit and the left pixel unit 104 is a first mode signal, and the row comparison signal between the first pixel unit and the right pixel unit 104 is a second mode signal, the edge detection signal corresponding to the first pixel unit is a second value.
[0127] Optionally, if the row comparison signals of the first pixel unit and the pixel units 104 on the left and right sides are both second mode signals, the edge detection signal corresponding to the first pixel unit is a third value.
[0128] Wherein, the first value indicates that the first pixel unit is located at the bottom or right edge of the object in the image frame, the second value indicates that the first pixel unit is located at the top or left edge of the object in the image frame, and the third value indicates that the first pixel unit is not located at the edge of the object in the image frame.
[0129] According to the image sensor 100 of this application embodiment, for pixel units 104 in the same row: when the row comparison signal between the first pixel unit and the pixel unit 104 to its left is a second mode signal, and the row comparison signal between the first pixel unit and the pixel unit 104 to its right is a first mode signal, the edge detection signal corresponding to the first pixel unit is a first value; when the row comparison signal between the first pixel unit and the pixel unit 104 to its left is a first mode signal, and the row comparison signal between the first pixel unit and the pixel unit 104 to its right is a second mode signal, the edge detection signal corresponding to the first pixel unit is a second value; when the row comparison signals between the first pixel unit and the pixel units 104 on both the left and right sides are both second mode signals, the edge detection signal corresponding to the first pixel unit is a third value. The first value indicates that the first pixel unit is located at the lower or right edge of an object in the image frame; the second value indicates that the first pixel unit is located at the upper or left edge of an object in the image frame; and the third value indicates that the first pixel unit is not located at the edge of an object in the image frame. In this way, edge detection in the row direction is achieved by the image sensor 100, enriching the functionality of the image sensor 100.
[0130] According to some embodiments of this application, optionally, for the same column of pixel units 104:
[0131] When the column comparison signal between the first pixel unit and the upper pixel unit 104 is a second mode signal, and the column comparison signal between the first pixel unit and the lower pixel unit 104 is a first mode signal, the edge detection signal corresponding to the first pixel unit is a first value.
[0132] Optionally, when the column comparison signal between the first pixel unit and the upper pixel unit 104 is a first mode signal and the column comparison signal between the first pixel unit and the lower pixel unit 104 is a second mode signal, the edge detection signal corresponding to the first pixel unit is a second value.
[0133] Optionally, if the column comparison signals of the first pixel unit and the pixel units 104 on the upper and lower sides are both second mode signals, the edge detection signal corresponding to the first pixel unit is a third value.
[0134] In practical applications, edge detection signals can specifically be binary digital signals with positive and negative values. The binary value represents the edge strength, indicating whether the edge is located at the edge of an object, while the positive or negative sign indicates the edge direction.
[0135] Optionally, when the pixel array 102 contains m rows and n columns of pixel units 104, except for the pixel units 104 in the 1st row, mth row, 1st column and nth column which only participate in partial direction comparison, the remaining pixel units 104 are compared with their four adjacent pixel units 104 above, below, left and right.
[0136] For example, the first value is +1, the second value is -1, and the third value is 0, such as... Figure 23 As shown, pixels A and B within a given frame are compared with their neighboring pixels. The comparison results for pixel A are shown in Table 1 below, and the comparison results for pixel B are shown in Table 2 below.
[0137] Table 1
[0138] Adjacent pixel positions superior Down Left right Pixel Comparison Signal 0 1 0 1
[0139] Table 2
[0140] Adjacent pixel positions superior Down Left right Pixel Comparison Signal 1 0 1 0
[0141] As shown in Table 1, the edge detection signal of pixel A is +1 in both the column and row directions, while the edge detection signal of pixel B is -1 in both the column and row directions.
[0142] Based on this, edge detection is performed along both the row and column directions. While outputting image data signals, the image sensor 100 can also output the first edge detection result along the row direction, such as... Figure 24 As shown, and the second edge detection result in the output column direction, as follows: Figure 25 As shown.
[0143] According to the image sensor 100 of this application embodiment, for the same column of pixel units 104: when the column comparison signal between the first pixel unit and the upper pixel unit 104 is a second mode signal, and the column comparison signal between the first pixel unit and the lower pixel unit 104 is a first mode signal, the edge detection signal corresponding to the first pixel unit is a first value; when the column comparison signal between the first pixel unit and the upper pixel unit 104 is a first mode signal, and the column comparison signal between the first pixel unit and the lower pixel unit 104 is a second mode signal, the edge detection signal corresponding to the first pixel unit is a second value; when the column comparison signals between the first pixel unit and the pixel units 104 on both the upper and lower sides are both second mode signals, the edge detection signal corresponding to the first pixel unit is a third value. In this way, edge detection in the column direction of the image sensor 100 is realized, enriching the functionality of the image sensor 100, and, combined with edge detection in the row direction of the image sensor 100, improving the accuracy of edge detection.
[0144] According to some embodiments of this application, optionally, such as Figure 1 As shown, the image sensor 100 also includes a first driver 118 and a second driver 120.
[0145] The first driver 118 is connected to each pixel unit 104.
[0146] Optionally, the second driver 120 is connected to the pixel comparison module 106.
[0147] Optionally, the first driver 118 is used to drive the pixel unit 104 to work.
[0148] Optionally, the image sensor 100 employs a rolling shutter readout method, the schematic diagram of which is shown below. Figure 9 As shown, in each frame, each pixel unit 104 undergoes three steps: reset, exposure, and readout. Each row of pixel units 104 is activated, operated, or reset simultaneously, but the operation steps of each row of pixel units 104 have a time difference from the operation steps of other rows of pixel units 104.
[0149] Optionally, the second driver 120 is used to drive the pixel comparison module 106 to operate.
[0150] Optionally, the second driver 120 can be a standalone module or embedded in the first driver 118.
[0151] Specifically, the second driver 120 provides control signals required for driving and operating each row of pixel comparison modules 106 via row conductors. The control signals output by the second driver 120 can control a single row of pixel comparison modules 106 individually, or simultaneously control multiple rows of pixel comparison modules 106. Pixel comparison modules 106 on the same row share control conductors and control signals.
[0152] In practical applications, since the added edge detection function is an additive feature, the pixel comparison module 106 can be turned off by the second driver 120. After the pixel comparison module 106 is turned off, the image sensor 100 is a conventional CIS chip based on LOFIC pixel circuitry.
[0153] The image sensor 100 according to an embodiment of this application further includes a first driver 118 and a second driver 120. The first driver 118 is connected to each pixel unit 104. The second driver 120 is connected to each pixel comparison module 106. The first driver 118 drives the pixel unit 104 to operate. The second driver 120 drives the pixel comparison module 106 to operate. This achieves independent driving of pixel output and pixel comparison, ensuring the stability of the operating timing of the image sensor 100, and preventing interference between the image output function and the edge detection function of the image sensor 100.
[0154] According to some embodiments of this application, optionally, such as Figure 26As shown, this application embodiment also provides a shooting module 200. The shooting module 200 includes the image sensor 100 from any of the above embodiments. The shooting module 200 provided in this application embodiment includes the image sensor 100 from any of the above embodiments and achieves the same technical effect; therefore, to avoid repetition, it will not be described again here.
[0155] According to some embodiments of this application, optionally, such as Figure 27 As shown in the figure, this application embodiment also provides an electronic device 300. The electronic device 300 includes the shooting module 200 in the above embodiments. The electronic device 300 provided in this application embodiment includes the shooting module 200 in the above embodiments and can achieve the same technical effect; therefore, to avoid repetition, it will not be described again here.
[0156] It should be noted that the electronic device 300 in the embodiments of this application includes mobile electronic devices and non-mobile electronic devices.
[0157] In practical applications, the electronic device 300 can be a terminal or other devices besides a terminal. For example, the electronic device 300 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. The embodiments of this application do not specifically limit it.
[0158] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0159] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
Claims
1. An image sensor, characterized by, include: Pixel array, including: There are M pixel units, which are used to collect optical signals and output electrical signals; M is a positive integer. There are N pixel comparison modules, where N is a positive integer and N is less than M; each pixel comparison module is connected to multiple adjacent pixel units; the pixel comparison module is used to compare the electrical signals output by the multiple adjacent pixel units and output a pixel comparison signal. A processing module is connected to each pixel unit and each pixel comparison module; the processing module is used to generate an image data signal based on the electrical signal and to generate an edge detection signal based on the pixel comparison signal; the image data signal is used to generate an image frame; the edge detection signal is used to indicate the edge position information of each pixel unit in the image frame.
2. The image sensor of claim 1, wherein, The pixel comparison signal includes a row comparison signal and a column comparison signal; the pixel comparison module includes a signal comparison circuit; the signal comparison circuit includes: A row comparator is connected to two adjacent pixel units in the same row; the row comparator is used to compare the electrical signals output by two adjacent pixel units row by row, and output a row comparison signal; A column comparator is connected to two adjacent pixel units in the same column; the column comparator is used to compare the electrical signals output by the two adjacent pixel units column by column and output a column comparison signal.
3. The image sensor of claim 2, wherein, The pixel unit includes an integrating capacitor; the electrical signal includes a first voltage signal across the integrating capacitor; the row comparator and the column comparator are both connected to the integrating capacitor in the pixel unit.
4. The image sensor of claim 2, wherein, The pixel comparison module further includes: A signal buffer circuit is connected between the pixel unit and the signal comparison circuit; the signal buffer circuit is used to buffer the electrical signal and preprocess the electrical signal. The preprocessing includes at least one of the following: amplification, noise reduction, and analog-to-digital conversion.
5. The image sensor of claim 4, wherein, The pixel unit includes an integrating capacitor; the electrical signal includes a first voltage signal across the integrating capacitor; the signal buffer circuit is connected to the integrating capacitor in the pixel unit.
6. The image sensor of claim 4, wherein, The pixel unit includes a row selection transistor; the electrical signal includes a second voltage signal output by the row selection transistor; and the signal buffer circuit is connected to the row selection transistor in the pixel unit.
7. The image sensor of claim 2, wherein, When the electrical signal amplitude of the first pixel unit is greater than the electrical signal amplitude of the adjacent pixel unit, the pixel comparison signal is a first mode signal; when the electrical signal amplitude of the first pixel unit is less than or equal to the electrical signal amplitude of the adjacent pixel unit, the pixel comparison signal is a second mode signal; the first pixel unit is any one of the M pixel units.
8. The image sensor of claim 7, wherein, For pixel units in the same row: When the row comparison signal between the first pixel unit and the left pixel unit is the second mode signal, and the row comparison signal between the first pixel unit and the right pixel unit is the first mode signal, the edge detection signal corresponding to the first pixel unit is the first value. When the row comparison signal between the first pixel unit and the left pixel unit is the first mode signal, and the row comparison signal between the first pixel unit and the right pixel unit is the second mode signal, the edge detection signal corresponding to the first pixel unit is the second value. When the row comparison signals of the first pixel unit and the pixel units on the left and right sides are both the second mode signals, the edge detection signal corresponding to the first pixel unit is the third value; Wherein, the first value indicates that the first pixel unit is located at the bottom or right edge of the object in the image frame; the second value indicates that the first pixel unit is located at the top or left edge of the object in the image frame; and the third value indicates that the first pixel unit is not located at the edge of the object in the image frame.
9. The image sensor of claim 8, wherein, For the same column of pixel units: When the column comparison signal between the first pixel unit and the upper pixel unit is the second mode signal, and the column comparison signal between the first pixel unit and the lower pixel unit is the first mode signal, the edge detection signal corresponding to the first pixel unit is the first value. When the column comparison signal between the first pixel unit and the upper pixel unit is the first mode signal, and the column comparison signal between the first pixel unit and the lower pixel unit is the second mode signal, the edge detection signal corresponding to the first pixel unit is the second value. When the column comparison signals of the first pixel unit and the pixel units on the top and bottom sides are both the second mode signal, the edge detection signal corresponding to the first pixel unit is the third value.
10. The image sensor according to any one of claims 1 to 9, wherein, The image sensor also includes: A first driver is connected to each pixel unit; the first driver is used to drive the pixel unit to work. The second driver is connected to each pixel comparison module; the second driver is used to drive the pixel comparison module to work.
11. A camera module, characterized by, The image sensor included in any one of claims 1 to 10.
12. An electronic device, comprising: Includes the shooting module as described in claim 11.
