Image processor, camera module and electronic device

Abstract

The application discloses an image processor, a camera module and an electronic device, and belongs to the technical field of image processing. The image processor comprises a pixel layer and a circuit layer. The pixel layer comprises a first bonding area and a pixel array. The pixel array is in communication connection with the first bonding area. The pixel array comprises N rows of pixel units and M columns of pixel units. The circuit layer comprises a second bonding area, an in-pixel cache module, a conversion gain selection logic module and a cache row decoding driving module. The second bonding area is in communication connection with the in-pixel cache module, the conversion gain selection logic module and the cache row decoding driving module respectively. The in-pixel cache module comprises in-pixel cache devices. Each in-pixel cache device is in bonding connection with one pixel unit. The first bonding area is in bonding connection with the second bonding area.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processor, a camera module, and an electronic device. Background Technology

[0002] In image sensors (Complementary Metal-Oxide Semiconductor, CMOS), the dynamic range of an image is generally adjusted by changing the pixel exposure time of all pixels and making overall adjustments to the pixel signal gain.

[0003] In related technologies, the stacked structure of image sensors often makes it difficult to adjust individual pixels. For exposure in Dual Conversion Gain (DCG-HDR) mode, the image sensor often needs to first expose in High Conversion Gain (HCG) mode, then read the image and expose in Low Conversion Gain (LCG) mode. This method not only places high demands on computing power, but the synthesized HDR image often suffers from brightness / color stratification and signal-to-noise ratio (SNR) differences due to processing defects, resulting in some pixels being locally overexposed or underexposed. Summary of the Invention

[0004] The purpose of this application is to provide an image processor, camera module, and electronic device that can solve the problem that the stacked structure of image sensors often makes it difficult to adjust individual pixels.

[0005] In a first aspect, embodiments of this application provide an image processor, including: a pixel layer and a circuit layer;

[0006] The pixel layer includes: a first bonding region and a pixel array, wherein the pixel array is communicatively connected to the first bonding region, and the pixel array includes N rows of pixel units and M columns of pixel units;

[0007] The circuit layer includes: a second bonding region, an intra-pixel cache module, a conversion gain selection logic module, and a cache line decoding driver module. The second bonding region is communicatively connected to the intra-pixel cache module, the conversion gain selection logic module, and the cache line decoding driver module, respectively. The intra-pixel cache module includes an intra-pixel cache device, and each intra-pixel cache device is bonded to one pixel unit. The first bonding region is bonded to the second bonding region.

[0008] The conversion gain selection logic module is used to determine the dual gain mode selection signal corresponding to each pixel unit based on the first pixel output signal of each pixel unit.

[0009] The intra-pixel buffer device is used to write the dual-gain mode selection signal of the pixel unit corresponding to the intra-pixel buffer device when it receives the buffer control signal sent by the buffer line decoding driver module, where M and N are both positive integers.

[0010] Secondly, embodiments of this application provide a camera module, including the image processor as described in the first aspect.

[0011] Thirdly, embodiments of this application provide an electronic device including a camera module as described in the second aspect.

[0012] In this embodiment of the application, by providing a conversion gain selection logic module in the circuit layer, the dual gain mode selection signal corresponding to each pixel unit can be determined according to the first pixel output signal of each pixel unit. Furthermore, the circuit layer is also provided with an intra-pixel buffer device for writing the dual gain mode selection signal of the pixel unit corresponding to the intra-pixel buffer device. This can effectively realize dual gain high dynamic range for pixel-by-pixel control of the pixel array in the stack structure of the image sensor. Attached Figure Description

[0013] Figure 1 This is one of the schematic diagrams of an image processor provided in an embodiment of this application;

[0014] Figure 2 This is a second schematic diagram of an image processor provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] The image processor, camera module, and electronic device provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0018] Figure 1 One of the schematic diagrams of the image processor provided in the embodiments of this application is shown below. Figure 1 As shown, it includes: pixel layer 1 and circuit layer 2;

[0019] The pixel layer 1 includes: a first bonding region 11 and a pixel array 12, wherein the pixel array 12 is communicatively connected to the first bonding region 11, and the pixel array 12 includes N rows of pixel units 1110 and M columns of pixel units 1110.

[0020] The circuit layer 2 includes a second bonding region 21, an intra-pixel cache module 22, a conversion gain selection logic module 23, and a cache line decoding driver module 24. The second bonding region 21 is communicatively connected to the intra-pixel cache module 22, the conversion gain selection logic module 23, and the cache line decoding driver module 24, respectively. The intra-pixel cache module 22 includes intra-pixel cache devices. Each intra-pixel cache device 221 is bonded to one pixel unit 1110. The first bonding region 11 is bonded to the second bonding region 21.

[0021] The conversion gain selection logic module 23 is used to determine the dual gain mode selection signal corresponding to each pixel unit 1110 based on the first pixel output signal of each pixel unit 1110.

[0022] Wherein, the in-pixel buffer device 221 is used to write the dual gain mode selection signal of the pixel unit 1110 corresponding to the in-pixel buffer device 221 when receiving the buffer control signal sent by the buffer line decoding driver module 24, where M and N are both positive integers.

[0023] Specifically, in the embodiments of this application, the pixel layer and the circuit layer can be composed of silicon wafer layers. The pixel layer is specifically a pixel silicon wafer layer, and all pixel units are disposed on the pixel layer. The pixel layer can be manufactured using front-side illumination (FSI) or back-side illumination (BSI) processes.

[0024] In this embodiment, the pixel layer is arranged with an effective pixel array, and a first bonding area can be set around the pixel array for signal routing, thereby realizing the connection between the pixel layer and the circuit layer.

[0025] In this embodiment of the application, the pixel array includes multiple pixel units, which may specifically include N pixel rows in the same row and M pixel columns in the same column, corresponding to NxM pixel units in each pixel array.

[0026] Specifically, in order to further save wiring space, M pixel units in the same pixel row will be connected to the first bonding region through the same connecting line, and N pixel units in the same pixel column will also be connected to the first bonding region through the same connecting line.

[0027] The circuit layer described in the embodiments of this application can be a multi-layer structure or a single-layer structure. For example, the circuit layer can be a single layer or a two-layer structure.

[0028] More specifically, the circuit layer in this embodiment includes a second bonding region, an intra-pixel cache module, a conversion gain selection logic module, and a cache line decoding driver module. The pixel cache module can be configured to correspond to the position of the pixel array in the pixel layer, so that each intra-pixel cache device can be bonded to a pixel unit. In this case, each pixel unit can directly read its corresponding dual-gain mode selection signal from its corresponding intra-pixel cache device.

[0029] In this embodiment, the conversion gain selection logic module and the cache line decoding driver module can be configured around the pixel cache module. To save wiring, the second bonding region can be configured between the pixel cache module and the conversion gain selection logic module and the cache line decoding driver module surrounding it. In this embodiment, the data of the conversion gain selection logic module and the cache line decoding driver module are transmitted to the pixel layer through the second bonding region via the first bonding region.

[0030] More specifically, the first pixel output signal of the pixel unit described in the embodiments of this application can be the pixel output signal output by the pixel unit in the current frame after reset and exposure processing, and the first pixel output signal of each pixel unit can be a different pixel output signal.

[0031] The conversion gain selection logic module described in this embodiment is specifically used to determine whether each pixel unit is suitable for high gain mode or low gain mode based on the first pixel output signal of each pixel unit, and then obtain the dual gain mode selection signal corresponding to each pixel unit.

[0032] In this embodiment, the dual-gain mode selection signal corresponding to each pixel unit can be a digital signal.

[0033] It can also be an analog signal. This dual-gain mode selection signal will control the pixel unit 5 to output the pixel signal according to the gain mode indicated by the dual-gain mode selection signal within this frame.

[0034] The cache line decoding driver module described in this application embodiment may specifically include a decoder and driver controlled by a control logic unit. The cache line decoding driver module can output a cache control signal for one or more pixel rows. After receiving the cache control signal, the pixel unit can activate the cache within the pixel unit.

[0035] The dual gain mode selection signal is stored and written into the buffer of the pixel unit so that when the pixel output signal is read later, the gain of the pixel output signal can be amplified according to the gain mode indicated by the dual gain mode selection signal.

[0036] In this embodiment, all signals required by the pixel unit originate from the underlying circuit layer. Specifically, signals in the circuit layer can be transmitted through the connection between the second bonding region and the first bonding region.

[0037] The signal is transmitted to the pixel layer. Furthermore, the pixel circuits of each pixel unit in the pixel layer can be signal-linked to the corresponding cache module circuits in the pixel-level cache module of the circuit layer via channels independent of the first bonding region 5 and the second bonding region.

[0038] In this embodiment, by providing a conversion gain selection logic module in the circuit layer, the dual gain mode corresponding to each pixel unit can be determined based on the first pixel output signal of each pixel unit.

[0039] The selection signal, and the circuit layer also includes an intra-pixel buffer device for writing the dual-gain mode selection signal of the pixel unit 0 corresponding to the intra-pixel buffer device, which can effectively optimize the image sensor stack structure.

[0040] Achieve dual-gain high dynamic range for pixel arrays with pixel-by-pixel control.

[0041] Optionally, the circuit layer includes: a first sub-circuit layer and a second sub-circuit layer, and the second bonding region includes a third bonding region and a fourth bonding region;

[0042] The first sub-circuit layer includes: the third bonding region and the intra-pixel cache module, wherein the third bonding region is communicatively connected to the intra-pixel cache module.

[0043] The second sub-circuit includes: the fourth bonding region, the conversion gain selection logic module, and the cache line decoding driver module, wherein the fourth bonding region is connected to the conversion gain selection logic module and the cache line decoding driver module respectively;

[0044] The third bonding region is bonded to both the first bonding region and the fourth bonding region.

[0045] Figure 2 This is a second schematic diagram of an image processor provided in an embodiment of this application, as shown below. Figure 2 As shown, it includes a pixel layer 1 and a circuit layer. The circuit layer further includes a first sub-circuit layer 201 and a second sub-circuit layer 202. The second bonding region may specifically include a third bonding region 210 and a fourth bonding region 211. The third bonding region 210 and the fourth bonding region 211 are respectively arranged in the first sub-circuit layer 201 and the second sub-circuit layer 202. In addition to the third bonding region 210, the first sub-circuit layer 201 only has an in-pixel buffer module 22, while the conversion gain selection logic module 23 and the buffer line decoding driver module 24 are both arranged in the second sub-circuit layer 202. That is to say, the main signals required by the pixel all come from the second sub-circuit layer.

[0046] More specifically, the third bonding region is bonded to the first bonding region and the fourth bonding region respectively, thereby enabling information exchange between the first sub-circuit layer, the second sub-circuit layer and the pixel layer. The positions of the third bonding region, the first bonding region and the fourth bonding region correspond to each other to facilitate their connection.

[0047] The fourth bonding region is connected to the conversion gain selection logic module and the cache line decoding driver module, respectively, thereby enabling the conversion gain selection logic module and the cache line decoding driver module to transmit information to the pixel layer or the first sub-circuit layer through the fourth bonding region.

[0048] More specifically, in the embodiments of this application, the first sub-circuit layer may specifically be a sub-circuit layer for information caching within a pixel.

[0049] In the embodiments of this application, a three-layer stack structure consisting of a pixel layer, a first sub-circuit layer, and a second sub-circuit layer is adopted. This structure enables dual-gain high dynamic range control of the pixel array on a pixel-by-pixel basis within the image sensor's stack structure. By dividing the circuit layer into a first sub-circuit layer and a second sub-circuit layer, the size of the circuit layer is effectively reduced, thereby effectively increasing the number of pixels and image resolution carried by the pixel sensor.

[0050] Optionally, the circuit layer further includes: a pixel signal processing module and a column control logic driving module;

[0051] The second bonding region is connected to both the pixel signal processing module and the column control logic driving module.

[0052] The column control logic driving module is used to control the pixel array to perform image processing, and the pixel signal processing module is used to output the pixel output signal after each pixel unit has been processed by the corresponding gain mode. The gain mode is determined according to the dual gain mode selection signal corresponding to each pixel unit.

[0053] Specifically, in the embodiments of this application, within each frame time, the column control logic driving module can control each pixel unit to perform reset, exposure and readout. After each pixel unit completes the reset and exposure, it will enter the readout stage. At this time, the pixel signal processing module will read the pixel output signal of each pixel unit.

[0054] During the process of reading the pixel output signal, the gain mode of the pixel unit is determined according to the dual gain mode selection signal cached by the buffer device in each pixel unit, that is, whether each pixel unit is suitable for high gain mode or low gain mode.

[0055] During the process of reading the pixel output signal, the signal is output according to the gain mode corresponding to the pixel unit, and finally the second pixel output signal after gain processing is output through the pixel signal processing module.

[0056] In this embodiment, the pixel array is controlled by the column control logic driving module to perform image processing, and the pixel signal processing module outputs the pixel output signal after each pixel unit has been processed by the corresponding gain mode, thereby ensuring that the image processor can perform the image processing function normally.

[0057] Optionally, the pixel array further includes: a signal readout module;

[0058] The N pixel units in the same column are connected to the signal reading module via a first connecting line;

[0059] The signal reading module is used to read the second pixel output signal of each pixel unit. The second pixel output signal is obtained by the pixel unit after performing signal gain processing according to the written dual gain mode selection signal.

[0060] In the embodiments of this application, during each frame time, after each pixel unit completes the reset and exposure, it will enter the reading stage, at which time the signal reading module will read the pixel output signal of each pixel unit.

[0061] During the process of reading the pixel output signal, the gain mode of the pixel unit is determined according to the dual gain mode selection signal cached by the buffer device in each pixel unit, that is, whether each pixel unit is suitable for high gain mode or low gain mode.

[0062] During the process of reading the pixel output signal, the signal is output according to the gain mode corresponding to the pixel unit, and finally the second pixel output signal after gain processing is obtained.

[0063] Optionally, the conversion gain selection logic module, the buffer row decoding driver module, the pixel signal processing module, and the column control logic driver module are all arranged around the pixel-in-pixel buffer module in the circuit layer.

[0064] Specifically, in this embodiment, since each of the intra-pixel cache devices in the intra-pixel cache module needs to be bonded to a pixel unit, that is, in the stack structure, the position of the intra-pixel cache module needs to correspond to the position of the pixel array in the pixel layer. In order not to affect the setting of other modules and for the convenience of wiring, the intra-pixel cache module can often be set in the central area of ​​the circuit layer, and then the conversion gain selection logic module, the cache row decoding driver module, the pixel signal processing module and the column control logic driver module are all arranged around the intra-pixel cache module.

[0065] In some optional embodiments, the in-pixel buffer module can be placed in a corner of the circuit layer, while the conversion gain selection logic module, buffer row decoding driver module, pixel signal processing module and column control logic driver module can be distributed in other locations. This application does not further limit this.

[0066] In this embodiment, the conversion gain selection logic module, the cache row decoding driver module, the pixel signal processing module, and the column control logic driver module are all arranged around the pixel-in-pixel cache module in the circuit layer, which can effectively save wiring space and reduce the size of the circuit layer.

[0067] Optionally, the circuit layer further includes an input / output module, which is connected to the second bonding region;

[0068] The input / output devices are connected to external devices via gold wire connections or solder ball arrays, and are used for signal output from the image processor and signal input from external devices.

[0069] Specifically, in the embodiments of this application, the external device may refer to other devices outside the image processing framework, such as flash memory devices or processors, etc.

[0070] In this embodiment, the signal input and output of the image processor are conducted through the input / output module, i.e., the I / O port, on the circuit layer. The connection method with the outside world is not limited; it can be a gold wire connection or a solder ball array connection.

[0071] Optionally, the circuit layer in this embodiment may also include conventional modules in an image processor, such as a phase-locked loop (PLL) module and a power management unit (PMU) module.

[0072] In this embodiment of the application, the input / output module enables normal signal interaction between the image processor and external devices, ensuring the normal operation of the image processor.

[0073] Optionally, the silicon wafer size of the circuit layer is larger than the silicon wafer size of the pixel layer.

[0074] Specifically, when the circuit layer is a single-layer structure, the single-layer circuit layer includes modules such as a second bonding region, an intra-pixel cache module, a conversion gain selection logic module, and a cache line decoding driver module. Since each intra-pixel cache device is bonded to a pixel unit in the pixel layer, the pixel array and the intra-pixel cache module may be the same size.

[0075] The circuit layer includes not only in-pixel buffer devices, but also many other logic devices, such as conversion gain selection logic modules and buffer line decoding driver modules. Therefore, when the circuit layer is a single-layer structure, the silicon wafer size corresponding to the circuit layer is often larger than the silicon wafer size of the pixel layer.

[0076] Optionally, when the circuit layer has a two-layer structure, the circuit layer includes: a first sub-circuit layer and a second sub-circuit layer. The pixel-level buffer module is located in the first sub-circuit layer, while other conversion gain selection logic modules and the buffer line decoding driver module are located in the second sub-circuit layer. In this case, the silicon wafer size of the first sub-circuit layer and the second sub-circuit layer can be the same as the silicon wafer size of the pixel layer.

[0077] Optionally, the bonding connection method is specifically: through-silicon via interconnect or copper-copper bonding connection method.

[0078] Specifically, in the embodiments of this application, the connection between the layers can be through a through-silicon via (TSV) or copper-copper bonding (Cu-Cu Hybrid Bonding), or it can be a new silicon-silicon signal connection bonding technology in the future.

[0079] In other words, the connection between the first bonding region and the second bonding region can be a through-silicon via (TSV) interconnect or a copper-copper bonding connection, and the connection between the intra-pixel cache device and the pixel unit can also be a TSV interconnect or a copper-copper bonding connection.

[0080] More specifically, the connection between the first sub-circuit layer and the second sub-circuit layer can also be a through-silicon via (TSV) interconnect or a copper-copper bonding connection.

[0081] In the embodiments of this application, the effective connection of each layer in the pixel processor can be ensured by the interconnection method of through silicon via or copper-copper bonding.

[0082] Optionally, in the image processor of this application embodiment, for applications that do not require adaptive DCG-HDR, this image processor architecture also includes traditional pixel signal processing modules and links, which users can choose according to their needs.

[0083] Optionally, embodiments of this application also provide a camera module including the above-mentioned image sensor. This camera module can perform pixel-by-pixel modulation during the implementation of DCG-HDR function, effectively ensuring the imaging effect of the output image.

[0084] Optionally, embodiments of this application also provide an electronic device, which includes the camera module described in the above embodiments. This electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. Embodiments of this application do not impose specific limitations.

[0085] The electronic device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0086] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0088] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processor, characterized in that, include: Pixel layer and circuit layer; The pixel layer includes: a first bonding region and a pixel array, wherein the pixel array is communicatively connected to the first bonding region, and the pixel array includes N rows of pixel units and M columns of pixel units; The circuit layer includes: a second bonding region, an intra-pixel cache module, a conversion gain selection logic module, a cache row decoding driver module, a pixel signal processing module, and a column control logic driver module. The second bonding region is communicatively connected to the intra-pixel cache module, the conversion gain selection logic module, the cache row decoding driver module, the pixel signal processing module, and the column control logic driver module, respectively. The intra-pixel cache module includes an intra-pixel cache device, and each intra-pixel cache device is bonded to one pixel unit. The first bonding region is bonded to the second bonding region. The conversion gain selection logic module is used to determine the dual gain mode selection signal corresponding to each pixel unit based on the first pixel output signal of each pixel unit. The in-pixel buffer device is used to write the dual-gain mode selection signal of the pixel unit corresponding to the in-pixel buffer device when receiving the buffer control signal sent by the buffer line decoding driver module, where M and N are both positive integers; The column control logic driving module is used to control the pixel array to perform image processing, and the pixel signal processing module is used to output the pixel output signal after each pixel unit has been processed by the corresponding gain mode. The gain mode is determined according to the dual gain mode selection signal corresponding to each pixel unit. The pixel array further includes: a signal reading module; N pixel units in the same column are connected to the signal reading module via a first connection line; wherein, the signal reading module is used to read the second pixel output signal of each pixel unit, and the second pixel output signal is obtained by the pixel unit after performing signal gain processing according to the written dual gain mode selection signal.

2. The image processor according to claim 1, characterized in that, The circuit layer includes: a first sub-circuit layer and a second sub-circuit layer, and the second bonding region includes a third bonding region and a fourth bonding region; The first sub-circuit layer includes: the third bonding region and the intra-pixel cache module, wherein the third bonding region is communicatively connected to the intra-pixel cache module. The second sub-circuit includes: the fourth bonding region, the conversion gain selection logic module, and the cache line decoding driver module, wherein the fourth bonding region is connected to the conversion gain selection logic module and the cache line decoding driver module respectively; The third bonding region is bonded to both the first bonding region and the fourth bonding region.

3. The image processor according to claim 1, characterized in that, The conversion gain selection logic module, the buffer row decoding driver module, the pixel signal processing module, and the column control logic driver module are all arranged around the pixel-in-pixel buffer module in the circuit layer.

4. The image processor according to claim 1, characterized in that, The circuit layer further includes an input / output module, which is connected to the second bonding region. The input / output devices are connected to external devices via gold wire connections or solder ball arrays, and are used for signal output from the image processor and signal input from external devices.

5. The image processor according to claim 1, characterized in that, The silicon wafer size of the circuit layer is larger than that of the pixel layer.

6. The image processor according to claim 1 or 2, characterized in that, The bonding connection method is specifically: through-silicon via interconnect or copper-copper bonding connection method.

7. A camera module, characterized in that, Includes the image processor as described in any one of claims 1-6.

8. An electronic device, characterized in that, Includes the camera module as described in claim 7.

Images