Pixel fusion circuit, global image sensor device and intelligent terminal

Abstract

The utility model provides a pixel fusion circuit, a global image sensor device and an intelligent terminal, the pixel fusion circuit comprises M pixel circuits, at least one storage unit and N global switching devices, and M and N are positive integers; the pixel circuit and the storage unit are electrically connected with the ISP circuit unit module; the pixel circuits are arranged in one-to-one correspondence with the display pixels and output photoelectric signals of the display pixels; and the storage unit is electrically connected with the global switching device and is arranged corresponding to more than four adjacent same-color pixels in the display pixels, so that photoelectric signals fused by the more than four adjacent same-color pixels are recorded when global exposure is carried out in a multi-in-one mode. According to the invention, two sets of systems of roller shutter switch exposure and global exposure can be compatible, and rapid snapshot / preview / video can be carried out by adopting global exposure in a multi-in-one mode in which more than four adjacent same-color pixels are combined, so that the transmission bandwidth limitation is reduced, and the smooth video shooting and rapid object snapshot capabilities which are not possessed by a current CIS chip are improved.

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to a pixel fusion circuit, a global image sensor device and an intelligent terminal. Background Art

[0002] The current mobile phone CIS (CMOS image sensor) chip exposure adopts rolling shutter exposure, such as Figure 1 As shown, the rolling shutter switch exposure is read line by line. The CIS chip needs to traverse each line once to complete an exposure. The rolling shutter switch exposure can be used regardless of taking pictures / previewing / or video.

[0003] However, when using the rolling shutter switch to expose moving objects, existing CIS chips will cause motion blur and jello effect, and normal objects will be distorted. This is mainly because when the rolling shutter switch is exposed, there is a slight exposure time difference when the display pixels are read from top to bottom, resulting in different exposure times for the upper and lower parts of the picture, which leads to spatial distortion and motion artifacts.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Utility Model Content

[0005] In response to the above technical problems, the present application provides a pixel fusion circuit, a global image sensor device and an intelligent terminal, which are compatible with two systems of rolling switch exposure and global exposure. Global exposure can be used for fast capture / preview / video in all-in-1 mode (for example, 4-in-1, 9-in-1, 16-in-1), reducing transmission bandwidth limitations and improving the smooth video shooting and fast object capture capabilities that current CIS chips do not have.

[0006] The present application provides a pixel fusion circuit, comprising M pixel circuits, at least one storage unit and N global switch devices, where M and N are positive integers; the pixel circuit and the storage unit are both electrically connected to an ISP circuit unit module; the pixel circuit is arranged in a one-to-one correspondence with a display pixel, and outputs a photoelectric signal of the display pixel; the storage unit is electrically connected to the global switch device, and is also arranged in correspondence with more than 4 adjacent same-color pixels in the display pixel, so as to record the photoelectric signals fused by the more than 4 adjacent same-color pixels when global exposure is performed in an all-in-one mode.

[0007] Optionally, the display pixels are arranged in a Bayer array.

[0008] Optionally, the storage unit is arranged corresponding to 16 adjacent pixels of the same color, the 16 adjacent pixels of the same color form a monochrome unit, and four adjacent monochrome units form one display pixel.

[0009] Optionally, the pixel circuit is provided with a 4T-APS structure, and the 4T-APS structure includes a photodiode, a first transistor, a second transistor, a third transistor and a fourth transistor; the first transistor is connected between the photodiode and a suspended diffusion area, and the gate of the first transistor is used to receive a transmission signal; the second transistor is connected between a first power supply terminal and a suspended diffusion area, and the suspended diffusion area is a region for converting the charge from the photodiode into a voltage signal, the gate of the second transistor is used to receive a reset signal to reset the suspended diffusion area, and the first power supply terminal is used to receive a power supply voltage; the third transistor is a source follower, and is connected between the second power supply terminal and the fourth transistor, the gate of the third transistor is connected to the suspended diffusion area to amplify and output the voltage signal of the suspended diffusion area, and the second power supply terminal is used to receive the power supply voltage; the fourth transistor is connected between the third transistor and the output end, the gate of the fourth transistor is used to receive a selection signal, and the output end outputs the pixel's photoelectric signal Pix out.

[0010] Optionally, the pixel fusion circuit includes a stacked pixel unit circuit layer and a storage unit circuit layer; the pixel unit circuit layer is provided with a pixel unit circuit, and the pixel unit circuit includes a plurality of the pixel circuits; the storage unit circuit layer is located below the pixel unit circuit layer, the storage unit circuit layer is provided with a storage unit circuit, and the storage unit circuit includes the storage unit.

[0011] Optionally, a through silicon via is provided between the pixel unit circuit layer and the storage unit circuit layer.

[0012] Optionally, the storage unit circuit layer is arranged above the ISP unit circuit layer, and the ISP unit circuit layer is arranged above the Si substrate layer; the ISP unit circuit layer is provided with an ISP circuit unit module; and the Si substrate layer is provided with a Si substrate.

[0013] Optionally, a through silicon via is provided between the pixel unit circuit layer and the Si substrate layer.

[0014] The present application also provides a global image sensor device, comprising the pixel fusion circuit as described above.

[0015] The present application also provides a smart terminal, comprising the global image sensor device as described above.

[0016] The pixel fusion circuit, global image sensor device and intelligent terminal provided by the present application are provided with storage units corresponding to more than 4 adjacent pixels of the same color. The storage unit can store the photoelectric signals after the fusion of more than 4 adjacent pixels of the same color when the all-in-one mode, such as 16-in-1 mode, and the global switch device is turned on. It is compatible with two systems, namely rolling switch exposure and global exposure. In the full-frame full size mode and 4-in-1 mode, rolling switch exposure is used to take photos / preview / video. In the all-in-one mode (such as 16-in-1) in which more than 4 adjacent pixels of the same color are merged, global exposure can be used for snapshot / preview / video, thereby reducing the transmission bandwidth limitation and improving the smooth video shooting and fast object snapshot capabilities that the current CIS chip does not have. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0018] Figure 1 It is a schematic diagram of a read frame of a CIS chip exposure in the prior art;

[0019] Figure 2 It is a schematic diagram of the hardware structure of an intelligent terminal for implementing various embodiments of the present application;

[0020] Figure 3 It is a communication network system architecture diagram provided by an embodiment of the present application;

[0021] Figure 4 is a circuit structure diagram of a pixel fusion circuit according to the first embodiment;

[0022] Figure 5a , 5b and 5c is Figure 4 Schematic diagram of the Bayer array in three modes of the pixel fusion circuit;

[0023] Figure 6 is a partial structural schematic diagram of a pixel fusion circuit according to the second embodiment;

[0024] Figure 7 is a schematic diagram of a stacked structure of a pixel fusion circuit according to a third embodiment;

[0025] Figure 8 yes Figure 7 A connection diagram of the pixel fusion circuit when it is working;

[0026] Fig. 9 yes Figure 7 A partial connection diagram of the pixel fusion circuit when it is working;

[0027] Fig.10 It is a schematic diagram of the pixel fusion circuit provided in an embodiment of the present application before and after shooting.

[0028] The realization of the purpose, functional features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. The above-mentioned drawings have shown clear embodiments of this application, which will be described in more detail later. These drawings and textual descriptions are not intended to limit the scope of the concept of this application in any way, but to illustrate the concept of this application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0029] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0030] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0031] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate that there are the described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or", "and / or", "including at least one of the following" etc. used in this application can be interpreted as inclusive, or mean any one or any combination. For example, “comprising at least one of the following: A, B, C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”, and for another example, “A, B or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A and B and C”. An exception to this definition will only occur when a combination of elements, functions, steps or operations are inherently mutually exclusive in some manner.

[0032] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0034] The smart terminal may be implemented in various forms. For example, the smart terminal described in this application may include smart terminals such as mobile phones, tablet computers, laptop computers, PDAs, portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, etc., as well as fixed terminals such as digital TVs and desktop computers.

[0035] See also Figure 2, which is a schematic diagram of the hardware structure of a mobile terminal for implementing various embodiments of the present application, the mobile terminal 100 may include: RF (Radio Frequency, radio frequency) unit 101, WiFi module 102, audio output unit 103, A / V (audio / video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that Figure 1 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal. The mobile terminal may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] Combine the following Figure 2 The following is a detailed introduction to the various components of the mobile terminal:

[0037] The radio frequency unit 101 can be used for receiving and sending signals during information transmission or communication. Specifically, after receiving the downlink information of the base station, it is sent to the processor 110 for processing; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communications can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication), GPRS (General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution) and 5G, etc.

[0038] WiFi is a short-range wireless transmission technology. Mobile terminals can help users send and receive emails, browse web pages, and access streaming media through WiFi module 102. It provides users with wireless broadband Internet access. Figure 1 The WiFi module 102 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention.

[0039] The audio output unit 103 can convert the audio data received by the RF unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output it as sound when the mobile terminal 100 is in a call signal reception mode, a talk mode, a recording mode, a voice recognition mode, a broadcast reception mode, etc. Moreover, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, etc.

[0040] The A / V input unit 104 is used to receive audio or video signals. The A / V input unit 104 may include a graphics processor (GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The processed image frame can be displayed on the display unit 106. The image frame processed by the graphics processor 1041 can be stored in the memory 109 (or other storage medium) or sent via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in a telephone call mode, a recording mode, a voice recognition mode, and other operating modes, and can process such sound into audio data. The processed audio (voice) data can be converted into a format output that can be sent to a mobile communication base station via the radio frequency unit 101 in the case of a telephone call mode. The microphone 1042 can implement various types of noise elimination (or suppression) algorithms to eliminate (or suppress) noise or interference generated in the process of receiving and sending audio signals.

[0041] The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 1061 and / or the backlight when the mobile terminal 100 is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured on the mobile phone, such as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0042] The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0043] The user input unit 107 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 1071 or near the touch panel 1071 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 110, and can receive and execute the command sent by the processor 110. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may further include other input devices 1072. Optionally, the other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc., which are not specifically limited here.

[0044] Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits the information to the processor 110 to determine the type of touch event. Then, the processor 110 provides corresponding visual output on the display panel 1061 according to the type of touch event. Figure 1 In the figure, the touch panel 1071 and the display panel 1061 are used as two independent components to implement the input and output functions of the mobile terminal. However, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated to implement the input and output functions of the mobile terminal, which is not limited here.

[0045] The interface unit 108 serves as an interface through which at least one external device can be connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headphone port, etc. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and an external device.

[0046] The memory 109 can be used to store software programs and various data. The memory 109 can mainly include a program storage area and a data storage area. Optionally, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 109 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0047] The processor 110 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 109, and calling data stored in the memory 109, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 110.

[0048] The mobile terminal 100 may also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0049] although Figure 2 Not shown, the mobile terminal 100 may further include a Bluetooth module, etc., which will not be described in detail here.

[0050] In order to facilitate understanding of the embodiments of the present application, the communication network system on which the mobile terminal of the present application is based is described below.

[0051] See also Figure 3 , Figure 3 A communication network system architecture diagram is provided for an embodiment of the present application, wherein the communication network system is an LTE system of universal mobile communication technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203 and an operator's IP service 204, which are connected in sequence for communication.

[0052] Optionally, UE201 may be the above-mentioned terminal 100, which will not be described in detail here.

[0053] E-UTRAN 202 includes eNodeB 2021 and other eNodeBs 2022 , etc. Optionally, eNodeB 2021 may be connected to other eNodeBs 2022 via a backhaul (eg, an X2 interface), and eNodeB 2021 is connected to EPC 203 , and eNodeB 2021 may provide UE 201 with access to EPC 203 .

[0054] EPC203 may include MME (Mobility Management Entity) 2031, HSS (Home Subscriber Server) 2032, other MMEs 2033, SGW (Serving Gate Way) 2034, PGW (PDN Gate Way) 2035 and PCRF (Policy and Charging Rules Function) 2036, etc. Optionally, MME 2031 is a control node that processes signaling between UE 201 and EPC 203, and provides bearer and connection management. HSS 2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information such as service features and data rates. All user data can be sent through SGW2034, PGW2035 can provide IP address allocation and other functions for UE 201, PCRF2036 is the policy and charging control policy decision point for service data flow and IP bearer resources, which selects and provides available policy and charging control decisions for the policy and charging execution function unit (not shown in the figure).

[0055] The IP service 204 may include the Internet, an intranet, an IMS (IP Multimedia Subsystem) or other IP services.

[0056] Although the above introduction takes the LTE system as an example, those skilled in the art should know that the present application is not only applicable to the LTE system, but also to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, 5G and future new network systems (such as 6G), etc., without limitation here.

[0057] Based on the above-mentioned mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

[0058] The pixel fusion circuit provided in the present application, by setting storage units corresponding to more than 4 adjacent pixels of the same color, the storage units can store the photoelectric signals after the fusion of more than 4 adjacent pixels of the same color when in an all-in-1 mode such as a 16-in-1 mode and the global switch device is turned on; it is compatible with two systems of rolling switch exposure and global exposure. In the full-frame full size mode and the 4-in-1 mode, rolling switch exposure is used to take photos / preview / video. In the all-in-1 mode (such as 16-in-1) in which more than 4 units are merged, global exposure can be used for snapshot / preview / video, reducing the transmission bandwidth limitation and improving the smooth video shooting and fast object snapshot capabilities that the current CI S chip does not have.

[0059] First embodiment

[0060] Please refer to Figure 4 , Figure 4 3 is a circuit structure diagram of a pixel fusion circuit according to the first embodiment. The pixel fusion circuit in the embodiment of the present application includes M pixel circuits 310, at least one storage unit 320 and N global switch devices 330, where M and N are positive integers; the pixel circuit 310 and the storage unit 320 are both electrically connected to the ISP circuit unit module ( Figure 4 The pixel circuit 310 is arranged in one-to-one correspondence with the display pixel to output the photoelectric signal of the display pixel; the storage unit 320 is electrically connected to the global switch device 330, and the storage unit 320 is arranged in correspondence with more than 4 adjacent same-color pixels in the display pixel to record the photoelectric signal Vaa of more than 4 adjacent same-color pixels when global exposure is performed in the all-in-one mode.

[0061] Specifically, Figure 4Only the circuit structure corresponding to one pixel on the pixel unit is illustrated. The pixel is correspondingly provided with a pixel circuit 310. The pixel circuit 310 is provided with an APS (Active Pixel Sensor) structure. The APS structure includes multiple transistors and a photodiode, which can be used to process and convert the electrical signal generated by the pixel unit to achieve high-resolution and high-image quality image capture. When performing rolling shutter switch exposure, the ISP circuit unit module can perform image processing according to the photoelectric signal of the corresponding pixel provided by the pixel circuit 310. In addition, it should be noted that the pixel is provided with a corresponding storage unit 320, but the storage unit 320 is not only set in one-to-one correspondence with the pixel, but is set in correspondence with more than 4 adjacent pixels of the same color, and the setting position of the storage unit 320 for each pixel may not be unique, and the structure of the remaining adjacent pixels of the same color can be set with reference to the pixel. When performing global exposure in the multi-in-1 mode, N global switch devices 330 can be turned on accordingly according to the global control signal GS, more than 4 adjacent pixels of the same color can be pixel-fused, and the storage unit 320 can record the photoelectric signal Vaa of more than 4 adjacent pixels of the same color fused, without the need to record one piece of information after each frame is read in the traditional way. For example, one storage unit 320 can be set corresponding to 16 adjacent pixels of the same color, so that when performing global exposure in the 4-in-1 mode, 16 adjacent pixels of the same color can be pixel-fused, and the storage unit 320 records the photoelectric signal Vaa of 16 adjacent pixels of the same color fused.

[0062] Optionally, the pixel circuit 310 may be provided with a 4T-APS structure. Figure 4 As shown, the 4T-APS structure includes a photodiode PD, a first transistor T1 to a fourth transistor T4. The first transistor T1 is connected between the photodiode PD and the suspended diffusion region FD, and its gate is used to receive the transmission signal TG. The second transistor T2 is connected between the first power terminal and the suspended diffusion region FD, and the suspended diffusion region FD is a region for converting the charge from the photodiode PD into a voltage signal. The gate of the second transistor T2 is used to receive the reset signal RST to reset the suspended diffusion region FD, and the first power terminal is used to receive the power supply voltage Vp ix. The third transistor T3 is a source follower and is connected between the second power terminal and the fourth transistor T4. Its gate is connected to the suspended diffusion region FD to amplify and output the voltage signal of the suspended diffusion region FD, and the second power terminal receives the power supply voltage Vdd. The fourth transistor T4 is connected between the third transistor T3 and the output end, and its gate is used to receive the selection signal SEL, and the output end outputs the photoelectric signal Pix out of the pixel.

[0063] Optionally, in a plurality of pixel fusion circuits corresponding to more than 4 adjacent pixels of the same color, N photodiodes are connected one by one to the first ends of N global switch devices 330, and the second ends of the N global switch devices 330 are electrically connected to the same storage unit 320, so that when global exposure is adopted in the all-in-one mode, the N global switch devices 330 are simultaneously controlled to be turned on, and the charges of all more than 4 adjacent pixels of the same color are fused, and the storage unit 320 records the photoelectric signal Vaa of all more than 4 adjacent pixels of the same color fused. It can be understood that the photodiode can be a photodiode of an APS structure in the pixel circuit 310, and the APS structure can be used to process and convert the electrical signal generated by the pixel unit. In other variant implementations, a node of the APS structure can be connected to the global switch device 330, and it can still be used for charge fusion of more than 4 adjacent pixels of the same color, all of which belong to the protection scope of the present application.

[0064] Figure 5a , 5b and 5c is Figure 4 The following is a schematic diagram of the Bayer array in three modes of the pixel fusion circuit. The following only takes a CIS chip with 50 million pixels and 1.0 micron pixel size as an example to illustrate the working method in three modes, but it does not limit the pixels and pixel size of the CIS chip.

[0065] like Figure 5a As shown, in the full size (50M) mode, the pixels may be arranged in a Bayer array (i.e., RGGB array), but the present application is not limited thereto, and may also be other arrays, such as RGBW, nona-cell, etc. The Bayer array includes multiple pixel sets, each pixel set includes a pixel R, a pixel Gr, a pixel Gb, and a pixel B, for a total of 4 types of pixels. However, each pixel is only 1.0 micron in size, and there is not enough space to set a corresponding storage unit 320 for each pixel of a single pixel set. In the full size (50M) mode, rolling shutter switch exposure can be used for taking pictures / previewing / videoing, and the number of pixels is large, which occupies a large transmission bandwidth.

[0066] like Figure 5bAs shown, in the 4-in-1 (12.5M) mode, the pixels may be arranged in a Bayer array (i.e., RGGB array), but the present application is not limited thereto, and may also be other arrays, such as RGBW, nona-cell, etc. The Bayer array includes a plurality of pixel sets, each pixel set including four adjacent same-color pixels R, four adjacent same-color pixels Gr, four adjacent same-color pixels Gb, and four adjacent same-color pixels B, for a total of 4 types of pixels. However, each pixel is only 2.0 microns in size, and there is not enough space to set a corresponding storage unit 320 for each pixel of a single pixel set. In the 4-in-1 (12.5M) mode, rolling shutter switch exposure can be used for taking photos / previewing / videos, with a large number of pixels and a large transmission bandwidth.

[0067] like Figure 5c As shown, in the 16-in-1 (3.125M) mode, the pixels can be arranged in a Bayer array (i.e., RGGB array), but the present application is not limited thereto, and can also be other arrays, such as RGBW, nona-cell, etc. The Bayer array includes multiple pixel sets, each pixel set includes sixteen adjacent same-color pixels R, sixteen adjacent same-color pixels Gr, sixteen adjacent same-color pixels Gb, and sixteen adjacent same-color pixels B, for a total of four types of pixels. At this time, each pixel is 4.0 micron pixel size, and there is enough space to set a corresponding storage unit 320 for each pixel of a single pixel set, which is compatible with the preparation of a global exposure device. In the 16-in-1 (3.125M) mode, it has the functions of rolling shutter switch exposure and global exposure switching. Global exposure can be used for snapshot / preview / video. The storage unit 320 can be used to record the photoelectric signal of the moment during global exposure, which can be read out after processing by the ISP circuit unit module. The number of pixels after fusion is relatively reduced. Under low transmission bandwidth, global exposure can be used to shoot high-speed moving objects, and the snapshot capability is improved, and the adaptive recording capability is improved, which improves the smooth video shooting and fast object capture capabilities that the current CIS chip does not have.

[0068] It should be noted that, in the example of the embodiment of the present application, the storage unit 320 is set corresponding to 16 adjacent same-color pixels, so that when global exposure is performed in the 16-in-1 mode, the photoelectric signal Vaa of the fusion of 16 adjacent same-color pixels is recorded, but the present application is not limited to this. It is only necessary to first satisfy that the space of the CIS chip is sufficient to set a corresponding storage unit 320 for each pixel of a single pixel set, and then set the storage unit 320 according to the number of adjacent same-color pixels included in each pixel, that is, global exposure can be used for fast capture / preview / video in the multi-in-1 mode in which more than 4 adjacent same-color pixels are merged. For example, if the space of the CIS chip is sufficient to set a corresponding storage unit 320 for each pixel of a single pixel set including 9 adjacent same-color pixels, the storage unit 320 can be set to correspond to the 9 adjacent same-color pixels, so that when global exposure is performed in the 9-in-1 mode, the photoelectric signal Vaa of the fusion of 9 adjacent same-color pixels can be recorded, and so on, all belong to the protection scope of the present application.

[0069] Second embodiment

[0070] Figure 6 It is a partial structural diagram of a pixel fusion circuit according to the second embodiment. This embodiment provides a pixel fusion circuit, and its basic structure, principle and technical effects are the same as those of the first embodiment. For the sake of brief description, for parts not mentioned in this embodiment, reference may be made to the corresponding contents in the first embodiment. In this embodiment, the pixel fusion circuit includes a pixel unit circuit layer and a storage unit circuit layer arranged in a stacked manner; the pixel unit circuit layer is provided with a pixel unit circuit, and the pixel unit circuit includes M pixel circuits 310; the storage unit circuit layer is located below the pixel unit circuit layer, and the storage unit circuit layer is provided with a storage unit circuit, and the storage unit circuit includes a storage unit 320. Figure 6 As shown, through the stacked structure of this embodiment, the storage unit is arranged in the storage unit circuit layer and is located below, and the M pixel circuits 310 corresponding to more than 4 adjacent pixels of the same color are arranged in the pixel unit circuit layer and are located above, so that the storage unit 320 can be easily arranged corresponding to more than 4 adjacent pixels of the same color on the pixel unit, such as 16 adjacent pixels of the same color.

[0071] Optionally, please also refer to Figure 4 and Figure 6 A through silicon via (TSV) may be provided between the pixel unit circuit layer and the memory unit circuit layer, and the pixel unit circuit may be electrically connected to the memory unit circuit through the through silicon via (TSV) to achieve pixel fusion.

[0072] Third embodiment

[0073] Figure 7 FIG. 1 is a schematic diagram of a stacked structure of a pixel fusion circuit according to the third embodiment. Figure 7 As shown, this embodiment provides a pixel fusion circuit, and its basic structure, principle and technical effect are the same as those of the second embodiment. For the sake of brief description, for parts not mentioned in this embodiment, reference may be made to the corresponding contents in the second embodiment. In this embodiment, the pixel fusion circuit is provided with a stacked pixel unit circuit layer 410 and a storage unit circuit layer 420. In addition, the storage unit circuit layer 420 is also arranged above the ISP unit circuit layer 430, and the ISP unit circuit layer 430 is arranged above the Si substrate layer 440; the ISP unit circuit layer 430 is provided with an ISP circuit unit module; and the Si substrate layer 440 is provided with a Si substrate. That is to say, this embodiment is provided with a stacked structure of a pixel unit circuit layer 410, a storage unit circuit layer 420, an ISP unit circuit layer 430 and a Si substrate layer 440 from top to bottom, but the present application is not limited thereto. For example, in another embodiment, the storage unit circuit layer 420 and the ISP unit circuit layer 430 may also be arranged as the same layer.

[0074] Alternatively, if Figure 7 As shown, through silicon vias (TSVs) may be provided between the pixel unit circuit layer 410 and the Si substrate layer 440 , that is, the circuits of each layer may be interconnected through the through silicon vias (TSVs).

[0075] Figure 8 yes Figure 7 A connection diagram of the pixel fusion circuit when it is working. Figure 7 and Figure 8 In the image sensor, the pixels are arranged in an array, and the corresponding pixel circuits 310 in the pixel unit circuit layer 410 are arranged in an array accordingly. Then the M pixel circuits 310 arranged in the array can be used as a detection part, and the detection part can also include a vertical scanner 341 and a horizontal scanner 342 respectively connected to each pixel circuit 310. The vertical scanner 341 can be responsible for controlling and operating the exposure and readout of the entire array. The horizontal scanner 342 can be responsible for reading the photoelectric signals of the pixels in the pixel unit array row by row in order, and transmitting the photoelectric signals to the ISP circuit unit module. In addition, the detection part can be connected to the storage part, Figure 8 The storage part only illustrates the connection part of a pixel circuit 310, and the pixel circuit 310 is connected to the global switch device 330, and the global switch device 330 is electrically connected to the storage unit 320, so that when global exposure is performed in the all-in-one mode, the global switch device 330 is turned on, and the storage unit 320 can record the photoelectric signal Vaa of more than 4 adjacent pixels of the same color, and send it to the ISP circuit unit module.

[0076] Fig. 9 yes Figure 7 Partial connection diagram of the pixel fusion circuit when it is working. Figure 5c , Figure 7 , Figure 8 and Fig. 9 The pixel fusion circuit of this embodiment can be arranged in a global image sensing device, and the global image sensing device can be divided into a detection part, a storage part, a signal processing part and a support part, and the circuits of each part can be interconnected through silicon vias TSV. Among them, the detection part includes a pixel circuit 310 corresponding to each pixel, the storage part includes a plurality of storage units 320, the signal processing part includes an ISP circuit unit module 350, and the support part includes a Si substrate. Among them, the detection part can group pixels, and when the global image sensing device performs global exposure in an all-in-one mode such as a 16-in-1 mode, more than 4 adjacent pixels of the same color can be pixel fused, such as 16 adjacent pixels of the same color can be pixel fused, then the more than 4 adjacent pixels of the same color are grouped into one group, for example, Figure 5c The 16 adjacent same-color pixels in the image can be divided into a group, and the 16 adjacent same-color pixels are arranged in 4X4. Then the detection part groups the pixels, and each group can be 16 adjacent same-color pixels including pixels 1 to 4, pixels 5 to 8, pixels 9 to 12, and pixels 13 to 16. Among them, each storage unit 320 is respectively set corresponding to a group of adjacent same-color pixels, and each storage unit 320 can store the photoelectric signal after the fusion of a corresponding group of adjacent same-color pixels, such as the photoelectric signal after the fusion of 16 adjacent same-color pixels. Therefore, when using the multi-in-1 mode such as the 16-in-1 mode with global exposure, the storage unit 320 participates in the work, which records the photoelectric signal Vaa of more than 4 adjacent same-color pixels, and is read out after processing by the ISP circuit unit module 350, which can be used for fast capture, preview and high-speed video recording, improving the video smooth shooting and fast object capture capabilities that the current CIS chip does not have.

[0077] It is understood that the detection part and the signal processing part can be interconnected through silicon vias TSV ( Fig. 9 For example, when rolling shutter switch exposure is adopted in the conventional full size (50M) mode or 4-in-1 (12.5M) mode, the photoelectric signal of each pixel can be directly sent to the ISP circuit unit module, and the storage unit 320 can be correspondingly non-operating, and the pixel unit and the ISP circuit unit module operate, which is the same as the conventional CIS chip operating mode in the prior art.

[0078] Fig.10 3 is a schematic diagram of the pixel fusion circuit provided in the embodiment of the present application before and after shooting. The pixel fusion circuit of this embodiment is compatible with two systems of rolling shutter switch exposure and global exposure. When the all-in-one mode such as 16-in-1 mode is used for global exposure, the storage unit 320 participates in the work. The storage unit 320 records the photoelectric signals of all pixels at a moment, such as Fig.10As shown in the schematic diagram before and after shooting, the storage unit does not record the photoelectric signal before exposure, but records the photoelectric signals of all pixels for a moment after exposure, and is read out after being processed by the ISP circuit unit module, thereby realizing fast capture, preview and high-speed video recording.

[0079] The embodiment of the present application also provides a global image sensor device, including the above pixel fusion circuit. The implementation of the global image sensor device can refer to the above pixel fusion circuit embodiment, and the repeated parts will not be repeated.

[0080] An embodiment of the present application also provides a smart terminal, comprising the global image sensor device as described above.

[0081] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0082] The circuits in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0083] In the present application, the same or similar terminology concepts, technical solutions and / or application scenario descriptions are generally described in detail only the first time they appear. When they appear again later, they are generally not repeated for the sake of brevity. When understanding the technical solutions and other contents of the present application, for the same or similar terminology concepts, technical solutions and / or application scenario descriptions that are not described in detail later, reference can be made to the previous related detailed descriptions.

[0084] In the present application, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] The various technical features of the technical solution of the present application can be arbitrarily combined. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present application.

[0086] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, a data center, etc. that contains one or more available media integrated. The available medium can be a magnetic medium, (e.g., a floppy disk, a storage disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state storage disk Solid State Disk (SSD)), etc.

[0087] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A pixel fusion circuit, characterized in that: It includes M pixel circuits, at least one storage unit and N global switch devices, where M and N are positive integers; the pixel circuit and the storage unit are both electrically connected to the ISP circuit unit module; the pixel circuit is arranged in a one-to-one correspondence with the display pixel, and outputs the photoelectric signal of the display pixel; the storage unit is electrically connected to the global switch device, and is also arranged in correspondence with more than 4 adjacent same-color pixels in the display pixel, so as to record the photoelectric signal of the more than four adjacent same-color pixels fused when performing global exposure in the all-in-one mode.

2. The pixel fusion circuit according to claim 1, characterized in that: The display pixels are arranged in a Bayer array.

3. The pixel fusion circuit according to claim 1, characterized in that: The storage unit is arranged corresponding to 16 adjacent pixels of the same color, the 16 adjacent pixels of the same color form a monochrome unit, and four adjacent monochrome units form one display pixel.

4. The pixel fusion circuit according to claim 1, characterized in that: The pixel circuit is provided with a 4T-APS structure, and the 4T-APS structure includes a photodiode, a first transistor, a second transistor, a third transistor and a fourth transistor; the first transistor is connected between the photodiode and the suspended diffusion region, and the gate of the first transistor is used to receive a transmission signal; The second transistor is connected between the first power terminal and the suspended diffusion area, the suspended diffusion area is a region for converting the charge from the photodiode into a voltage signal, the gate of the second transistor is used to receive a reset signal to reset the suspended diffusion area, and the first power terminal is used to receive a power supply voltage; the third transistor is a source follower and is connected between the second power terminal and the fourth transistor, the gate of the third transistor is connected to the suspended diffusion area to amplify and output the voltage signal of the suspended diffusion area, and the second power terminal is used to receive the power supply voltage; the fourth transistor is connected between the third transistor and the output end, the gate of the fourth transistor is used to receive a selection signal, and the output end outputs the pixel's photoelectric signal Pix out.

5. The pixel fusion circuit according to any one of claims 1 to 4, characterized in that: The pixel fusion circuit includes a stacked pixel unit circuit layer and a storage unit circuit layer; the pixel unit circuit layer is provided with a pixel unit circuit, and the pixel unit circuit includes a plurality of the pixel circuits; the storage unit circuit layer is located below the pixel unit circuit layer, and the storage unit circuit layer is provided with a storage unit circuit, and the storage unit circuit includes the storage unit.

6. The pixel fusion circuit according to claim 5, characterized in that: A through silicon via is provided between the pixel unit circuit layer and the memory unit circuit layer, and the pixel circuit is electrically connected to the memory unit through the through silicon via.

7. The pixel fusion circuit according to claim 5, characterized in that: The storage unit circuit layer is arranged above the ISP unit circuit layer, and the ISP unit circuit layer is arranged above the Si substrate layer; the ISP unit circuit layer is provided with an ISP circuit unit module; and the Si substrate layer is provided with a Si substrate.

8. The pixel fusion circuit according to claim 7, characterized in that: A through silicon via is provided between the pixel unit circuit layer and the Si substrate layer.

9. A global image sensor device, characterized in that: The method comprises the pixel fusion circuit as claimed in any one of claims 1 to 8.

10. An intelligent terminal, characterized in that: Comprising the global image sensor device as claimed in claim 9.

Images