Image processing apparatus, electronic apparatus, and operation method for noise reduction

Abstract

Disclosed are an image processing apparatus, an electronic apparatus, and an operating method of an apparatus for noise reduction. The image processing apparatus includes an image sensor including a pixel array configured to generate a first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain from a pixel sharing a floating diffusion region, and configured to generate first image data and second image data based on the first pixel signal and the second pixel signal, respectively, and an image signal processor configured to generate an output image based on the first image data and the second image data. The image signal processor includes a nominalization circuit configured to nominalize the first image data based on a dynamic range of the second image data to generate third image data, and a blending circuit configured to generate the output image based on the second image data and the third image data.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to Korean Patent Application No. 10-2021-0083080, filed on June 25, 2021, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The exemplary embodiments of this disclosure described herein relate to electronic devices, and more specifically, to an image processing apparatus and method of operation thereof for noise reduction using dual-conversion gain. Background Technology

[0004] Image sensors include charge-coupled device (CCD) image sensors, complementary metal-oxide-semiconductor (CMOS) image sensors, and so on. A CMOS image sensor consists of pixels composed of CMOS transistors and converts light energy into electrical signals using photoelectric conversion elements (or devices) included in each pixel. The CMOS image sensor obtains information about the captured / imaged image by using the electrical signals generated by each pixel.

[0005] Meanwhile, with the increasing demand for CMOS image sensors, including those in smartphones, and the growing need for high-quality images, various techniques are being developed to reduce noise in output images. In particular, it is necessary to adequately guarantee the brightness of images captured in low-light environments with limited exposure time, and it is also necessary to reduce noise caused by magnification. Summary of the Invention

[0006] The exemplary embodiments of this disclosure provide an image processing apparatus and a method of operation thereof for reducing noise in an output image by using a double conversion gain.

[0007] According to an embodiment, an apparatus for noise reduction using dual conversion gain includes: an image sensor including a pixel array configured to generate a first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain from pixels sharing a floating diffusion region, and the image sensor being configured to generate first image data and second image data based on the first pixel signal and the second pixel signal, respectively; and an image signal processor generating an output image based on the first image data and the second image data. The image signal processor includes: a standardization circuit that standardizes the first image data based on the dynamic range of the second image data to generate third image data; and a mixing circuit that generates the output image based on the second image data and the third image data.

[0008] According to an embodiment, an apparatus for noise reduction using dual conversion gain includes: a lens that receives light reflected from an object; an image sensor including a pixel array configured to generate a first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain from pixels sharing a floating diffusion region, and the image sensor being configured to receive light from the lens and generate first image data and second image data based on the first pixel signal and the second pixel signal, respectively; an image signal processor that generates an output image based on the first image data and the second image data; and a main processor that generates a video stream based on the output image. The image signal processor includes: a normalization circuit that normalizes the first image data based on the dynamic range of the second image data to generate third image data; and a mixing circuit that generates the output image based on the second image data and the third image data.

[0009] According to an embodiment, an operating method for an apparatus for noise reduction using dual conversion gain includes: generating first image data corresponding to a first conversion gain and second image data corresponding to a second conversion gain; nominalizing the first image data based on the dynamic range of the second image data to generate third image data; generating an output image based on the second image data and the third image data; and generating a video stream based on the output image. Attached Figure Description

[0010] The above and other objects and features of this disclosure will become readily understood by referring to the detailed description of embodiments of this disclosure with reference to the accompanying drawings.

[0011] Figure 1 An example of the configuration of an image processing block according to some exemplary embodiments of this disclosure is shown.

[0012] Figure 2 It shows Figure 1 An example of the configuration of an image sensor.

[0013] Figure 3 It shows Figure 2 A circuit diagram of one of the pixel groups in a pixel array.

[0014] Figure 4A It shows when Figure 3 Circuit diagram of the floating diffusion region when the dual-conversion transistor is turned off.

[0015] Figure 4B It shows when Figure 3The circuit diagram of the floating diffusion region when the dual-conversion transistor is turned on.

[0016] Figure 5 It shows Figure 2 A circuit diagram of another example of one of the pixel groups in a pixel array.

[0017] Figure 6 An example is shown of determining whether to perform noise reduction using dual conversion gain, depending on the shutter speed and conversion gain of the image sensor 120.

[0018] Figure 7 It is shown according to the reference Figure 6 An example describing the output image of the result of determining whether to perform noise reduction using double conversion gain.

[0019] Figure 8 The nominalization and hybridization operations according to some example embodiments of this disclosure are conceptually illustrated.

[0020] Figure 9 Examples of configurations of electronic devices including an image processing block according to some exemplary embodiments of the present disclosure are shown.

[0021] Figure 10 This is a flowchart illustrating a method for noise reduction using double conversion gain according to some example embodiments of the present disclosure.

[0022] Figure 11 An example configuration of an electronic device including a camera module that implements a noise reduction apparatus according to some example embodiments of the present disclosure is shown.

[0023] Figure 12 It shows Figure 11 An example of the camera module configuration. Detailed Implementation

[0024] Below, exemplary embodiments of the present disclosure will be described in detail and clearly to enable those skilled in the art to readily practice the present disclosure.

[0025] In the detailed description, the components described by reference to terms such as "unit," "module," "block," "and," or "device," as well as the functional blocks shown in the accompanying drawings, will be implemented using, for example, hardware including logic circuitry; hardware / software combinations, such as a processor running software; or combinations thereof. For example, a processing circuitry system may more specifically include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field-programmable gate array (FPGA), a system-on-a-chip (SoC), a programmable logic unit, a microprocessor, an application-specific integrated circuit (ASIC), etc. For example, hardware may include circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, pressure sensors, inertial sensors, microelectromechanical systems (MEMS), passive components, or combinations thereof.

[0026] Figure 1 Examples of configurations of an image processing block 100 according to some exemplary embodiments of the present disclosure are shown. The image processing block 100 can be implemented as part of various electronic devices such as smartphones, digital cameras, laptops, and / or desktop computers. The image processing block 100 may include a lens 110, an image sensor 120, and an image signal processor 130.

[0027] Light can be reflected from objects such as the subject or scenery, and lens 110 can receive the reflected light. Image sensor 120 can generate an electrical signal based on the light received through lens 110. For example, image sensor 120 can be implemented using a complementary metal-oxide-semiconductor (CMOS) image sensor or the like. For example, image sensor 120 can be a multi-pixel image sensor with a dual-pixel structure or a four-unit structure.

[0028] Image sensor 120 may include a pixel array. Pixels in the pixel array can convert light into electrical signals to generate pixel signals. The ratio at which light is converted into electrical signals (e.g., voltage) is defined as the "conversion gain". In particular, the pixel array can generate pixel signals under both low and high conversion gain conditions by using variations in conversion gain, i.e., dual conversion gain.

[0029] Additionally, the image sensor 120 may include an analog-to-digital converter (ADC) circuit for performing correlated double sampling (CDS) on the pixel values ​​to convert them into electrical signals. Specifically, the ADC circuit of this disclosure can output first image data IDAT1 corresponding to a pixel signal generated from the pixel array under low conversion gain conditions, and can output second image data IDAT2 corresponding to a pixel signal generated from the pixel array under high conversion gain conditions. (Refer to...) Figure 2 A more detailed description of the configuration of the image sensor 120 will be provided, and references will be made. Figure 3, Figure 4A and Figure 4B The configuration of the pixel array associated with the dual conversion gain is described in more detail.

[0030] The image signal processor 130 can appropriately process the first image data IDAT1 and / or the second image data IDAT2 output from the image sensor 120 and thus can generate an output image IOUT associated with the captured object or landscape. For example, the output image IOUT can be a still image or it can be an image of each video frame that constitutes the video. To this end, the image signal processor 130 can perform various processes such as color correction, automatic white balance, gamma correction, color saturation correction, bad pixel correction, tone correction, noise reduction, etc.

[0031] The image processing block 100 of this disclosure can operate in a first operating mode or a second operating mode based on the ambient illumination of the object. For example, the image processing block 100 can operate in the first operating mode in a low-illuminance environment where the image sensor 120 fails to receive sufficient light, and in the second operating mode in a high-illuminance environment where the image sensor 120 receives sufficient light.

[0032] In the first operating mode, the image sensor 120 can output first image data IDAT1 corresponding to a low conversion gain and second image data IDAT2 corresponding to a high conversion gain, and the image signal processor 130 can perform noise reduction by using dual conversion gain. Specifically, in the first operating mode, the image signal processor 130 can blend the first image data IDAT1 corresponding to the low conversion gain and the second image data IDAT2 corresponding to the high conversion gain, thereby generating a noise-reduced output image IOUT. For this purpose, the image signal processor 130 may include a normalization circuit 131 and a blending circuit 132.

[0033] The brightness of the image indicated by the first image data IDAT1, corresponding to a low conversion gain, can be darker than the brightness of the image indicated by the second image data IDAT2, corresponding to a high conversion gain. In other words, the dynamic range of the first image data IDAT1 and the dynamic range of the second image data IDAT2 can be different. Therefore, in order to match the dynamic range of the first image data IDAT1 and the dynamic range of the second image data IDAT2, the standardization circuit 131 can standardize the brightness value of the first image data IDAT1 depending on the dynamic range of the second image data IDAT2, thereby generating the third image data IDAT3.

[0034] However, this disclosure is not limited thereto. For example, the standardization circuit 131 may standardize the brightness value of the second image data IDAT2 depending on the dynamic range of the first image data IDAT1, thereby generating the third image data IDAT3. In the following, for clarity, it is assumed that the standardization circuit 131 standardizes the brightness value of the first image data IDAT1 to generate the third image data IDAT3.

[0035] The mixing circuit 132 can mix the second image data IDAT2 and the third image data IDAT3 to generate a noise-reduced output image IOUT. For example, the output image IOUT can be generated by multiplying the value of the third image data IDAT3 together with a first weight and multiplying the value of the second image data IDAT2 together with a second weight. That is, the output image IOUT can indicate an image with reduced noise compared to the first image data IDAT1 through the third image data IDAT3 by using a double-conversion gain.

[0036] The first and second weights can be arbitrary values ​​to be assigned to the image data, either pre-determined or during computation. For example, the first weight can be set to "a", a value between "0" and "1", and the second weight can be set to "1-a", the value of the first weight subtracted from "1". However, this disclosure is not limited thereto. For example, the output image IOUT can be generated using various schemes based on the first image data IDAT1 to the third image data IDAT3.

[0037] For example, noise reduction using double conversion gain according to some example embodiments of this disclosure can be performed to reduce temporal noise in an image. However, this disclosure is not limited thereto. Noise reduction using double conversion gain can also be performed to reduce different types of noise (e.g., spatial noise and fixed pattern noise).

[0038] The noise reduction using double conversion gain described above can be performed relative to still images, and also relative to images constituting video frames. The signal-to-noise ratio (SNR) of the image sensor 120 can be improved by performing noise reduction using double conversion gain.

[0039] Meanwhile, in the second operating mode, the image sensor 120 can output only the first image data IDAT1 corresponding to the low conversion gain, and the image signal processor 130 can generate the output image IOUT by processing the first image data IDAT1 without going through the above-described operations of the standardization circuit 131 and the mixing circuit 132.

[0040] Whether the image processing block 100 operates in a first operating mode or a second operating mode can be automatically determined by the ambient illuminance of the object sensed by an illuminance sensor (not shown) placed outside the image processing block 100, or it can be determined in response to user input. (Refer to...) Figure 9 This will be described in more detail.

[0041] exist Figure 1 The diagram shows a lens 110 and an image sensor 120. However, in some example embodiments, the image processing block 100 may include multiple lenses and multiple image sensors. In this case, the multiple lenses may have different fields of view. Additionally, the multiple image sensors may have different functions, different performance, and / or different characteristics, and may each include pixel arrays with different configurations.

[0042] Figure 2 It shows Figure 1 An example of the configuration of image sensor 120. Image sensor 120 may include pixel array 121, row driver 122, ramp signal generator 123, ADC circuitry 124, timing controller 125, and buffer 126. Below, it will be compared with... Figure 1 Describe together Figure 2 .

[0043] Pixel array 121 may include a plurality of pixels arranged in a matrix along rows and columns. Each of the plurality of pixels may include a photoelectric conversion element. For example, the photoelectric conversion element may include a photodiode, a phototransistor, a photogate, a pinned photodiode, etc.

[0044] Pixel array 121 may include multiple pixel groups PG. Each pixel group PG may include two or more pixels, i.e., multiple pixels. The multiple pixels constituting a pixel group PG may share a single floating diffusion region or multiple floating diffusion regions. Figure 2 An example of a pixel array 121 including pixel groups PG arranged in four rows and four columns (i.e., 4×4 pixel groups PG) is shown. However, this disclosure is not limited to this, and different sizes of pixel groups and floating diffusion areas can be used.

[0045] A pixel group PG may include pixels of the same color. For example, a pixel group PG may include red pixels for converting red light into electrical signals, green pixels for converting green light into electrical signals, or blue pixels for converting blue light into electrical signals. For example, the pixels constituting pixel array 121 may be arranged in a four-Bayer pattern.

[0046] In some example embodiments, pixel array 121 may include a plurality of pixel groups PG, and at least one of the plurality of pixel groups PG may have a color filter. For example, pixel group PG may include a red color filter for converting light into a red spectrum, or a green color filter for converting light into a green spectrum, or a blue color filter for converting light into a blue spectrum.

[0047] The pixels of pixel array 121 can output pixel signals through column lines CL1 to CL4 depending on the intensity or amount of light received from the outside. For example, the pixel signal can be an analog signal corresponding to the intensity or amount of light received from the outside.

[0048] When the image processing block 100 operates in a first operating mode (e.g., in a low-light environment), the pixel array 121 can output pixel signals under low conversion gain conditions and can also output pixel signals under high conversion gain conditions. In operation, when the image processing block 100 operates in a second operating mode (e.g., in a high-light environment), the pixel array 121 can output pixel signals under low conversion gain conditions. The pixel signals can be provided to the ADC circuit 124 via a voltage buffer (e.g., a source follower) and then via column lines CL1 to CL4.

[0049] The row driver 122 can select and drive rows of the pixel array 121. The row driver 122 can decode the address and / or control signals generated by the timing controller 125 and can generate control signals for selecting and driving rows of the pixel array 121. For example, the control signals may include signals for selecting pixels, signals for driving dual-conversion transistors, signals for resetting the floating diffusion region, etc.

[0050] The ramp signal generator 123 can generate a ramp signal under the control of the timing controller 125. For example, the ramp signal generator 123 can operate in response to a control signal such as a ramp enable signal. When the ramp enable signal is activated, the ramp signal generator 123 can generate a ramp signal depending on preset (or desired) values ​​(e.g., start level, end level, and ramp rate). In other words, the ramp signal can be a signal that increases or decreases along a preset (or desired) ramp rate during a specific time period. The image signal can be provided to the ADC circuit 124.

[0051] The ADC circuit 124 can receive pixel signals from multiple pixels of the pixel array 121 via column lines CL1 to CL4, and can receive ramp signals from the ramp signal generator 123. The ADC circuit 124 can operate based on correlated double sampling (CDS) technology to obtain a reset signal and an image signal from the received pixel signals and extract the difference between the reset signal and the image signal as the valid signal component. The ADC circuit 124 may include multiple comparators COMP and multiple counters CNT.

[0052] In detail, each comparator COMP can compare the reset signal and the ramp signal of the pixel signal, compare the image signal and the ramp signal of the pixel signal, and perform correlated double sampling (CDS) on the comparison result. Each counter CNT can count the pulses of the signal that has undergone correlated double sampling and output the counting result as a digital signal.

[0053] For example, each counter CNT can include an up / down counter, a bit-by-bit inverting counter, etc. The operation of a bit-by-bit counter can be similar to that of an up / down counter. For instance, a bit-by-bit counter can perform only up counting and, upon inputting a specific signal, can perform a function to transform all internal bits of the counter to obtain the two's complement of 1. A bit-by-bit counter can perform a reset count, and then the reset count result can be inverted to convert it to the two's complement of 1, i.e., a negative value.

[0054] For example, in the first operating mode, each counter CNT can independently output a digital signal corresponding to low conversion gain and a digital signal corresponding to high conversion gain. In contrast, in the second operating mode, each counter CNT can generate a digital signal corresponding to low conversion gain. Figure 2 An example of an ADC circuit 124 including four comparators COMP and four counters CNT is shown, but this disclosure is not limited thereto.

[0055] The timing controller 125 can generate control signals and / or clocks for controlling the operation and / or timing of each of the line driver 122, the ramp signal generator 123, and the ADC circuit 124.

[0056] Buffer 126 may include a memory MEM and a sense amplifier SA. The memory MEM may store the digital signal output from the corresponding counter CNT of the ADC circuit 124. The sense amplifier SA may sense and amplify the digital signal stored in the memory MEM. The sense amplifier SA may output the amplified digital signal as image data IDAT1 or IDAT2.

[0057] For example, when the digital signal corresponds to a low conversion gain, the amplified digital signal can be output as the first image data IDAT1; when the digital signal corresponds to a high conversion gain, the amplified digital signal can be output as the second image data IDAT2. In other words, the sense amplifier SA can output both the first image data IDAT1 and the second image data IDAT2 in the first operating mode, while in the second operating mode it can output only the first image data IDAT1.

[0058] Figure 3 It shows Figure 2 A circuit diagram of an example of one of the pixel groups PG of the pixel array 121. Figure 4A It shows when Figure 3 The circuit diagram of the floating diffusion region FD1 of the dual-conversion transistor when DC is turned off. Figure 4B It shows when Figure 3 The circuit diagram of the floating diffusion regions FD1 and FD2 of the dual-conversion transistor when DC is turned on.

[0059] For example, a pixel group PG may include pixels PX1 to PX4, photoelectric conversion elements PD1 to PD4, transfer transistors Tx1 to Tx4, a reset transistor RST, a dual conversion transistor DC, a drive transistor Dx, and a selection transistor SEL. Figure 3 An example of a pixel group PG having a four-unit structure in which four pixels PX1 to PX4 each include photoelectric conversion elements PD1 to PD4 is shown, but this disclosure is not limited thereto. For example, the pixel group PG can be implemented with various different structures.

[0060] The first pixel PX1 may include a first photoelectric conversion element PD1 and a first transfer transistor Tx1, while each of the remaining pixels PX2, PX3, and PX4 may also include similar components / elements. Pixels PX1 to PX4 may share a reset transistor RST, a dual conversion transistor DC, a drive transistor Dx, and a selection transistor SEL. Additionally, pixels PX1 to PX4 may share a first floating diffusion region FD1.

[0061] The first floating diffusion region FD1 or the second floating diffusion region FD2 can accumulate (or integrate) the charge corresponding to the amount of incident light. When transfer transistors Tx1 to Tx4 are turned on by transfer signals VT1 to VT4 respectively, the first floating diffusion region FD1 or the second floating diffusion region FD2 can accumulate (or integrate) the charge supplied from photoelectric conversion elements PD1 to PD4. Because the first floating diffusion region FD1 is connected to the gate terminal of the drive transistor Dx, which operates as a source follower amplifier, a voltage corresponding to the charge accumulated in the first floating diffusion region FD1 can be formed. For example, the capacitance of the first floating diffusion region FD1 is described as the first capacitor CFD1.

[0062] The dual-conversion transistor DC can be driven by the dual-conversion signal VDC. When the dual-conversion transistor DC is off, the capacitance of the first floating diffusion region FD1 can correspond to the first capacitor CFD1. That is, the first floating diffusion region FD1 can have a capacitance value that corresponds to the first capacitor CFD1. In general environments, such as low-light environments, because the first floating diffusion region FD1 is not easily saturated, it is not necessary to increase the capacitance of the first floating diffusion region FD1 (i.e., CFD1). In this case, the dual-conversion transistor DC can be off. In some example embodiments, the dual-conversion transistor DC can be off, so in the first operating mode, the first capacitor CFD1 can correspond to the first floating diffusion region FD1.

[0063] However, in high-illuminance environments, the first floating diffusion region FD1 may easily saturate. To prevent or reduce saturation, the dual-conversion transistor DC can be turned on, electrically connecting the first floating diffusion region FD1 and the second floating diffusion region FD2. In this case, the capacitance (capacitance value) of the floating diffusion region FD1 can be increased to the sum of the first capacitance CFD1 and the second capacitance CFD2. In some example embodiments, the dual-conversion transistor DC can be turned on, so in the second operating mode, the first capacitance CFD1 can correspond to the first floating diffusion region FD1 and the second floating diffusion region FD2. In some example embodiments, at least one of the first floating diffusion region FD1 and the second floating diffusion region FD2 can be a capacitor.

[0064] The transfer transistors Tx1 to Tx4 can be driven by transfer signals VT1 to VT4 respectively, and can transfer the charge generated (or integrated) by photoelectric conversion elements PD1 to PD4 to the first floating diffusion region FD1 or the second floating diffusion region FD2. For example, the first terminals of the transfer transistors Tx1 to Tx4 can be connected to photoelectric conversion elements PD1 to PD4 respectively, and their second terminals can be connected together with the first floating diffusion region FD1.

[0065] The reset transistor RST can be driven by the reset signal VRST and can provide a power supply voltage VDD to the first floating diffusion region FD1 or the second floating diffusion region FD2. Therefore, the charge accumulated in the first floating diffusion region FD1 or the second floating diffusion region FD2 can be moved to the terminal of the power supply voltage VDD, and the voltage of the first floating diffusion region FD1 or the second floating diffusion region FD2 can be reset.

[0066] The driving transistor Dx amplifies the voltage of either the first floating diffusion region FD1 or the second floating diffusion region FD2 and generates a pixel signal PIX corresponding to the amplified result. The selection transistor SEL is driven by the selection signal VSEL and can select the pixels to be read row by row. When the selection transistor SEL is turned on, the pixel signal PIX can be output via the column line CL. Figure 2 The ADC circuit 124.

[0067] According to some exemplary embodiments of this disclosure, in a first operating mode (e.g., in a low-light environment), the pixel array 121 can turn the dual-conversion transistor DC on or off, thereby changing the conversion gain and thus enabling the output of pixel signals under low and high conversion gain conditions, respectively. Specifically, the low conversion gain condition may correspond to the case where the dual-conversion transistor DC is on (…). Figure 4B The high conversion gain condition can correspond to the case where the DC turn-off of the dual-conversion transistor is off. Figure 4A (thin solid line).

[0068] Therefore, the image sensor 120 can output first image data IDAT1 corresponding to low conversion gain and second image data IDAT2 corresponding to high conversion gain, and the image signal processor 130 can perform noise reduction using dual conversion gain based on the first image data IDAT1 and the second image data IDAT2.

[0069] Meanwhile, the above description is based on the assumption that the same exposure time is applied to the pixels constituting the pixel group PG. Figures 1 to 3 , Figure 4A and Figure 4B As given; in some example embodiments, different exposure times can be applied to the pixels constituting the pixel group PG. In other words, a first portion of the pixels can be driven (or controlled) during a long exposure time, while a second portion of the pixels can be driven (or controlled) during a short exposure time. For example, noise reduction using double conversion gain can be performed when the number of pixels driven during the long exposure time is the same as the number of pixels driven during the short exposure time, but this disclosure is not limited thereto.

[0070] Figure 5 It shows Figure 2A circuit diagram of another example of one of the pixel groups in a pixel array. (Reference) Figure 5 A pixel group PG may include three unit pixel groups UPG, and each unit pixel group UPG includes three pixels PX1 to PX3. The first pixel PX1 may include a first photoelectric conversion element PD1 and a first transfer transistor Tx1, while each of the remaining pixels PX2 and PX3 may also include similar components / elements. Pixels PX1 to PX3 may share a reset transistor RST, a dual conversion transistor DC, a drive transistor Dx, and a selection transistor SEL. Additionally, pixels PX1 to PX3 may share a first floating diffusion region FD1.

[0071] The expansion of the capacitance of the first floating diffusion region FD1 by the dual-conversion transistor DC, the operation of the transfer transistors Tx1 to Tx3, and the operation of the reset transistor RST, drive transistor Dx, and select transistor SEL can be almost identical to the reference. Figure 3 The descriptions are similar, therefore, additional descriptions will be omitted to avoid redundancy.

[0072] The second floating diffusion region FD2 can be electrically connected to the floating diffusion region of an adjacent unit pixel group (not shown) via a connecting line IL1. In this case, the capacitance of the first floating diffusion region FD1 can be further increased (or expanded). Although not shown in the figures, the unit pixel group UPG may also include a switching element (e.g., an element such as a dual-conversion transistor DC) for electrically connecting the second floating diffusion region FD2 to the floating diffusion region of the adjacent unit pixel group.

[0073] Like reference Figure 3 As described herein, according to some example embodiments of this disclosure, in a first operating mode (e.g., in a low-light environment), the pixel array 121 can turn the dual-conversion transistor DC on or off, thereby changing the conversion gain and enabling the output of pixel signals under both low and high conversion gain conditions. Therefore, the image sensor 120 can output first image data IDAT1 corresponding to low conversion gain and second image data IDAT2 corresponding to high conversion gain, and the image signal processor 130 can perform noise reduction using the dual-conversion gain based on the first image data IDAT1 and the second image data IDAT2.

[0074] in addition, Figure 5Each unit pixel group (UPG) can have a different conversion gain (e.g., a first conversion gain to a third conversion gain) depending on whether the corresponding dual-conversion transistor DC is turned on or off. For example, each of the first to third conversion gains can correspond to either a low conversion gain or a high conversion gain. In this case, the image sensor 120 can perform noise reduction using dual conversion gains based on first to third image data corresponding to the first to third conversion gains, respectively.

[0075] Figure 6 An example is shown of determining whether to perform noise reduction DC GNR using dual conversion gain, depending on the shutter time or conversion gain of the image sensor 120. Shutter time, or shutter speed, indicates the duration during which the image sensor 120 receives light, and conversion gain indicates the rate at which the image sensor 120 converts light into an electrical signal. Below, this will be discussed in conjunction with... Figure 1 Describe together Figure 6 .

[0076] For reference Figure 1 As described, in low-light environments where image sensor 120 fails to receive sufficient light, noise reduction using dual conversion gain can be performed by image signal processor 130. Specifically, in low-light environments where even increasing the shutter speed of image sensor 120 cannot adequately guarantee image brightness, the conversion gain of image sensor 120 can be increased to obtain a brighter image. With the increase in conversion gain, the signal-to-noise ratio of image sensor 120 can also increase. Particularly in the case of video shooting, because the shutter speed is limited to ensure a sufficient frame rate, improvements in signal-to-noise ratio when outputting video frames may become even more important.

[0077] Therefore, the image processing block 100 can operate in a first operating mode to improve the signal-to-noise ratio of the image sensor 120 while ensuring sufficient image brightness in low-light environments. In the first operating mode, the image sensor 120 can output image data corresponding to a low conversion gain (e.g., Figure 1 The first image data (IDAT1) and the image data corresponding to the high conversion gain (e.g., Figure 1 The second image data (IDAT2). Subsequently, the image signal processor 130 can refer to... Figure 1 The described standardization and hybrid operations (e.g., the operation of standardization circuit 131 and hybrid circuit 132) are used to perform noise reduction using double conversion gain.

[0078] In contrast, in high-illuminance environments where the image sensor 120 receives sufficient light and high conversion gain is not required, the image processing block 100 can operate in a second operating mode. In this second operating mode, the image sensor 120 can output only image data corresponding to low conversion gain (e.g., Figure 1 The first image data (IDAT1), and the image signal processor 130 may not execute the reference. Figure 1 The description includes standardization and hybrid operations.

[0079] Figure 7 It is shown according to the reference Figure 6 This describes an example of determining whether to perform noise reduction DCG NR using dual conversion gain on the output image. Below, it will be compared with... Figure 1 Describe together Figure 7 .

[0080] In low-light environments, that is, when the image processing block 100 operates in the first operating mode, the output image can be an image generated by mixing an image corresponding to low conversion gain and an image corresponding to high conversion gain. In contrast, in high-light environments, that is, when the image processing block 100 operates in the second operating mode, the output image can be an image generated based on an image corresponding to low conversion gain.

[0081] Figure 8 The nominalization and hybridization operations according to some example embodiments of this disclosure are conceptually illustrated. (See reference...) Figure 1 As described, in order to match the dynamic range of the first image data IDAT1 corresponding to low conversion gain and the dynamic range of the second image data IDAT2 corresponding to high conversion gain, the standardization circuit 131 can standardize the brightness value of the first image data IDAT1 depending on the dynamic range of the second image data IDAT2 and thus generate the third image data IDAT3.

[0082] The hybrid circuit 132 can output a noise-reduced output image IOUT by multiplying the third image data IDAT3 with a first weight determined in advance or during calculation, and multiplying the second image data IDAT2 with a second weight determined in advance or during calculation. For example, the first weight can be set to "a", which is a value between "0" and "1", and the second weight can be set to "1-a", which is the value of the first weight subtracted from "1".

[0083] Figure 9Examples of configurations of an electronic device 10 including an image processing block 100 according to some exemplary embodiments of the present disclosure are shown. The electronic device 10 may include the image processing block 100, a main processor 200, and a display device 300. Operation and reference of the image processing block 100 are also provided. Figure 1 The operations described are the same, therefore, additional descriptions will be omitted to avoid redundancy.

[0084] The main processor 200 can perform various operations for controlling the overall operation of the electronic device 10. For example, the main processor 200 can be implemented using a general-purpose processor, a special-purpose processor, or an application processor (AP), and can include one or more processor cores. The main processor 200 can control the image processing block 100 to obtain the output image IOUT.

[0085] Specifically, based on the ambient illuminance of the object to be sensed by an illuminance sensor (not shown), the main processor 200, according to some example embodiments of this disclosure, can determine whether the image processing block 100 operates in a first operating mode or a second operating mode. The main processor 200 can send a control signal CTRL to the image processing block 100 based on the sensing results of the illuminance sensor.

[0086] For example, the control signal CTRL can be controlled via a reference. Figure 4A and Figure 4B The described dual-conversion signal VDC controls the on or off state of the dual-conversion transistor DC. For example, the control signal CTRL can be enabled in response to a low-light environment. The image processing block 100 can operate in a first operating mode in response to the enabled control signal CTRL, and can operate in a second operating mode when the control signal CTRL is not enabled.

[0087] Additionally, the main processor 200 can receive user input to determine a first operating mode or a second operating mode based on the ambient illuminance of the object. Furthermore, the main processor 200 can determine the operation to activate the hybrid circuit 132 based on both the ambient illuminance of the object and the received user input.

[0088] For reference Figure 1 As described, the output image IOUT can be a still image or an image of each video frame that constitutes the video. The main processor 200 can store the still image or video stream based on the obtained output image IOUT in a storage device such as a UFS card, embedded UFS memory, SD card, SSD and / or eMMC, and can provide the still image or video stream to the display device 300.

[0089] The display device 300 can provide a user with a still image or video stream based on the output image IOUT. For example, the display device 300 may include an organic LED (OLED) display device, an active matrix OLED (AMOLED) display device, a light-emitting diode (LED) display device, a liquid crystal display (LCD) device, etc., but this disclosure is not limited thereto.

[0090] For example, when the image signal processor 130 performs noise reduction using double conversion gain, the still image or video stream provided to the display device 300 can also be a still image or video stream that has undergone the aforementioned noise reduction using double conversion gain. Furthermore, when the output image IOUT is the image constituting each video frame, the display device 300 can provide the user with a preview image based on the video stream of the output image IOUT.

[0091] Therefore, still images or video frames based on the output image IOUT, which has undergone noise reduction using double conversion gain, can be stored in a storage device by the main processor 200. Additionally, video frames based on the output image IOUT, which has undergone noise reduction using double conversion gain, can be provided to the user as preview images on the display device 300. In other words, noise reduction using double conversion gain according to some example embodiments of this disclosure can be performed to generate various types of images, such as still images, video streams, and preview images.

[0092] Figure 10 This is a flowchart illustrating a method for noise reduction using double conversion gain according to some example embodiments of the present disclosure. (Refer to...) Figure 1 and Figure 9 describe Figure 10 .

[0093] In operation S110, the image sensor 120 can generate first image data IDAT1 corresponding to low conversion gain and second image data IDAT2 corresponding to high conversion gain. In operation S120, the normalization circuit 131 of the image signal processor 130 can normalize the brightness value of the first image data IDAT1 depending on the dynamic range of the second image data IDAT2, thereby generating third image data IDAT3.

[0094] In operation S130, the mixing circuit 132 of the image signal processor 130 can mix the second image data IDAT2 and the third image data IDAT3 to generate an output image IOUT. For example, the output image IOUT can be generated by multiplying the value of the third image data IDAT3 with a first weight (e.g., "a" as a value between "0" and "1") and multiplying the value of the second image data IDAT2 with a second weight (e.g., "1-a"). That is, the output image IOUT can indicate an image with reduced noise compared to the first image data IDAT1 through the third image data IDAT3 by using a double-conversion gain.

[0095] In operation S140, the main processor 200 can generate a still image or video stream based on the output image IOUT, store the still image or video stream in a storage device, and provide the still image or video stream to the display device 300. Alternatively, if the output image IOUT is the image constituting each video frame, in operation S150, the display device 300 can provide the video stream based on the output image IOUT as a preview image to the user. That is, noise reduction using double conversion gain according to some example embodiments of this disclosure can be performed to generate various types of images, such as still images, video streams, and preview images, and thus, the noise ratio of the image sensor 120 can be improved.

[0096] Figure 11 An example configuration of an electronic device including a camera module that implements a noise reduction apparatus according to some example embodiments of the present disclosure is shown. Figure 12 It shows Figure 11 An example of the camera module configuration.

[0097] refer to Figure 11 The electronic device 1000 may include a camera module group 1100, an application processor 1200, a PMIC 1300, and an external memory 1400.

[0098] Camera module group 1100 may include multiple camera modules 1100a, 1100b, and 1100c. Figure 11 The diagram illustrates an electronic device comprising three camera modules 1100a, 1100b, and 1100c, but this disclosure is not limited thereto. In some example embodiments, the camera module group 1100 may be modified to include only two camera modules.

[0099] Below, will refer to Figure 12 A more comprehensive and detailed description of the configuration of camera module 1100b is provided, but the following description is equally applicable to the remaining camera modules 1100a and 1100c.

[0100] refer to Figure 12 The camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140, and a storage unit 1150.

[0101] The prism 1105 may include a reflective plane 1107 of light-reflecting material and may change the path of light "L" incident from the outside.

[0102] In some example embodiments, prism 1105 can change the path of light "L" incident in the first direction "X" to a second direction "Y" perpendicular to the first direction "X". Alternatively, prism 1105 can change the path of light "L" incident in the first direction "X" to a second direction "Y" perpendicular to the first direction "X" by rotating the reflective plane 1107 of the light-reflecting material about the central axis 1106 in direction "A" or by rotating the central axis 1106 in direction "B". In this case, OPFE 1110 can move in a third direction "Z" perpendicular to the first direction "X" and the second direction "Y".

[0103] In some example embodiments, as shown in the figure, the maximum rotation angle of prism 1105 in direction "A" may be equal to or less than 15 degrees in the positive A direction and greater than 15 degrees in the negative A direction, but this disclosure is not limited thereto.

[0104] In some example embodiments, prism 1105 can move within approximately 20 degrees in the positive or negative B direction, between 10 and 20 degrees, or between 15 and 20 degrees; here, prism 1105 can move at the same angle in the positive or negative B direction or can move at a similar angle within approximately 1 degree.

[0105] In some example embodiments, the prism 1105 can move the reflective plane 1107 of the light-reflecting material in a third direction (e.g., the Z direction) parallel to the direction in which the central axis 1106 extends.

[0106] For example, OPFE 1110 may include an optical lens consisting of "m" groups (m being a natural number). Here, the "m" lenses can be moved in the second direction "Y" to change the optical zoom ratio of camera module 1100b. For example, when the default optical zoom ratio of camera module 1100b is "Z", the optical zoom ratio of camera module 1100b can be changed to 3Z, 5Z, or a higher optical zoom ratio by moving the "m" optical lenses included in OPFE 1110. OPFE 1110 may also include "n" groups of optical lenses (n being a natural number) in front of the aforementioned "m" lenses.

[0107] Actuator 1130 can move OPFE 1110 or optical lens (hereinafter referred to as "optical lens") to a specific position. For example, actuator 1130 can adjust the position of the optical lens so that image sensor 1142 is placed at the focal length of the optical lens to obtain accurate sensing.

[0108] Image sensing device 1140 may include image sensor 1142, control logic 1144, and memory 1146. Image sensor 1142 can sense an image of a target using light "L" provided through an optical lens. Control logic 1144 can control the overall operation of camera module 1100b. For example, control logic 1144 can control the operation of camera module 1100b based on control signals provided through control signal line CSLb.

[0109] The memory 1146 can store information necessary for the operation of the camera module 1100b, such as calibration data 1147. Calibration data 1147 may include information necessary for the camera module 1100b to generate image data using externally supplied light "L". Calibration data 1147 may include, for example, information about the degree of rotation, information about the focal length, information about the optical axis, etc. In the case where the camera module 1100b is implemented as a multi-state camera where the focal length varies depending on the position of the optical lens, calibration data 1147 may include the focal length value for each position (or state) of the optical lens and information about autofocus.

[0110] The storage unit 1150 can store image data sensed by the image sensor 1142. The storage unit 1150 can be disposed outside the image sensing device 1140 and can be implemented in a form in which the storage unit 1150 and the sensor chip constituting the image sensing device 1140 are stacked. In some example embodiments, the storage unit 1150 can be implemented with an electrically erasable programmable read-only memory (EEPROM), but this disclosure is not limited thereto.

[0111] Let's refer to each other. Figure 11 and Figure 12 In some example embodiments, each of the plurality of camera modules 1100a, 1100b, and 1100c may include an actuator 1130. Therefore, the same calibration data 1147 or different calibration data 1147 may be included in the plurality of camera modules 1100a, 1100b, and 1100c depending on the operation of the actuators 1130 in the plurality of camera modules 1100a, 1100b, and 1100c.

[0112] In some example embodiments, one of the plurality of camera modules 1100a, 1100b and 1100c (e.g. 1100b) may be a camera module in the shape of a folded lens including the aforementioned prism 1105 and OPFE 1110, while the remaining camera modules (e.g. 1100a and 1100c) may be camera modules in the shape of a vertical shape excluding the aforementioned prism 1105 and OPFE 1110; however, this disclosure is not limited thereto.

[0113] In some example embodiments, one of the multiple camera modules 1100a, 1100b, and 1100c (e.g., 1100c) may be a depth camera of a vertical shape, for example, that extracts depth information using infrared (IR). In this case, the application processor 1200 may combine image data provided from the depth camera with image data provided from any other camera module (e.g., 1100a or 1100b) and may generate a three-dimensional (3D) depth image.

[0114] In some example embodiments, at least two camera modules (e.g., 1100a and 1100b) of the plurality of camera modules 1100a, 1100b and 1100c may have different fields of view. In this case, at least two camera modules (e.g., 1100a and 1100b) of the plurality of camera modules 1100a, 1100b and 1100c may include different optical lenses, but are not limited thereto.

[0115] Additionally, in some example embodiments, the fields of view of the multiple camera modules 1100a, 1100b, and 1100c may be different. In this case, the multiple camera modules 1100a, 1100b, and 1100c may include different optical lenses, but are not limited thereto.

[0116] In some example embodiments, multiple camera modules 1100a, 1100b, and 1100c can be configured to be physically separate from each other. That is, multiple camera modules 1100a, 1100b, and 1100c may not use the sensing area of ​​a single image sensor 1142, but rather multiple camera modules 1100a, 1100b, and 1100c may each include an independent image sensor 1142.

[0117] Return to Figure 11 The application processor 1200 may include an image processing device 1210, a memory controller 1220, and internal memory 1230. The application processor 1200 may be implemented separately from the multiple camera modules 1100a, 1100b, and 1100c. For example, the application processor 1200 and the multiple camera modules 1100a, 1100b, and 1100c may be implemented using separate semiconductor chips.

[0118] The image processing apparatus 1210 may include a plurality of sub-image processors 1212a, 1212b and 1212c, an image generator 1214 and a camera module controller 1216.

[0119] The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b and 1212c, the number of which corresponds to the number of the plurality of camera modules 1100a, 1100b and 1100c.

[0120] Image data generated from camera modules 1100a, 1100b, and 1100c can be provided to their respective sub-image processors 1212a, 1212b, and 1212c via separate image signal lines ISLa, ISLb, and ISLc. For example, image data generated from camera module 1100a can be provided to sub-image processor 1212a via image signal line ISLa, image data generated from camera module 1100b can be provided to sub-image processor 1212b via image signal line ISLb, and image data generated from camera module 1100c can be provided to sub-image processor 1212c via image signal line ISLc. This image data transmission can be performed, for example, using a camera serial interface (CSI) based on MIPI (Mobile Industrial Processor Interface), but this disclosure is not limited thereto.

[0121] Meanwhile, in some example embodiments, a sub-image processor can be configured to correspond to multiple camera modules. For example, sub-image processor 1212a and sub-image processor 1212c can be implemented as a single unit, rather than as shown below. Figure 12 As shown, they are separated from each other; in this case, one of the multiple image data provided from camera module 1100a and camera module 1100c can be selected by selecting an element (e.g., a multiplexer), and the selected image data can be provided to the integrated sub-image processor.

[0122] For example, each of the sub-image processors 1212a, 1212b, and 1212c can normalize the brightness value of the image data corresponding to the low conversion gain based on the dynamic range of the image data corresponding to the high conversion gain, and can perform noise reduction using dual conversion gain by mixing the normalized image data and the image data corresponding to the high conversion gain.

[0123] Image data provided to sub-image processors 1212a, 1212b, and 1212c can be provided to image generator 1214. Image generator 1214 can generate an output image by using the image data provided from sub-image processors 1212a, 1212b, and 1212c, depending on image generation information or a mode signal.

[0124] In detail, the image generator 1214 can generate an output image by combining at least a portion of image data generated from camera modules 1100a, 1100b, and 1100c, which have different fields of view, depending on the image generation information or the mode signal. Alternatively, the image generator 1214 can generate an output image by selecting one of the image data generated from camera modules 1100a, 1100b, and 1100c, which have different fields of view, depending on the image generation information or the mode signal.

[0125] In some example embodiments, the image generation information may include a zoom signal or zoom factor. Additionally, in some example embodiments, the mode signal may be, for example, a signal based on a mode selected by the user.

[0126] When the image generation information is a zoom signal (or zoom factor) and camera modules 1100a, 1100b, and 1100c have different fields of view, the image generator 1214 may perform different operations depending on the type of zoom signal. For example, when the zoom signal is a first signal, the image generator 1214 may merge image data output from camera module 1100a and image data output from camera module 1100c, and may generate an output image by using the merged image data and image data output from camera module 1100b that is not used in the merging operation. When the zoom signal is a second signal different from the first signal, without an image data merging operation, the image generator 1214 may select one of the image data output from camera modules 1100a, 1100b, and 1100c respectively, and may output the selected image data as the output image. However, this disclosure is not limited thereto, and the way image data is processed can be modified as necessary without limitation.

[0127] In some example embodiments, the image generator 1214 can generate merged image data with increased dynamic range by receiving multiple image data with different exposure times from at least one of a plurality of sub-image processors 1212a, 1212b, and 1212c and performing high dynamic range (HDR) processing on the multiple image data. Additionally, the image generator 1214 can receive image data that has undergone noise reduction using double conversion gain from at least one of the plurality of sub-image processors 1212a, 1212b, and 1212c.

[0128] The camera module controller 1216 can provide control signals to camera modules 1100a, 1100b, and 1100c respectively. The control signals generated by the camera module controller 1216 can be provided to the corresponding camera modules 1100a, 1100b, and 1100c respectively through the separate control signal lines CSLa, CSLb, and CSLc.

[0129] One of a plurality of camera modules 1100a, 1100b, and 1100c may be designated as the master camera (e.g., 1100b) depending on image generation information or mode signals including zoom signals, while the remaining camera modules (e.g., 1100a and 1100c) may be designated as slave cameras. The aforementioned designation information may be included in control signals, and the control signals including the designation information may be provided to the corresponding camera modules 1100a, 1100b, and 1100c respectively via separate control signal lines CSLa, CSLb, and CSLc.

[0130] The operation of a camera module as a master or slave device can be changed depending on the zoom factor or operating mode signal. For example, when the field of view of camera module 1100a is wider than that of camera module 1100b and the zoom factor indicates a low zoom ratio, camera module 1100b can operate as a master device, while camera module 1100a can operate as a slave device. Conversely, when the zoom factor indicates a high zoom ratio, camera module 1100a can operate as a master device, while camera module 1100b can operate as a slave device.

[0131] In some example embodiments, the control signals provided from the camera module controller 1216 to each of the camera modules 1100a, 1100b, and 1100c may include a synchronization enable signal. For example, when camera module 1100b is used as the main camera and camera modules 1100a and 1100c are used as slave cameras, the camera module controller 1216 may transmit a synchronization enable signal to camera module 1100b. Camera module 1100b, provided with the synchronization enable signal, may generate a synchronization signal based on the provided synchronization enable signal and may provide the generated synchronization signal to camera modules 1100a and 1100c via the synchronization signal line SSL. Camera modules 1100b, 1100a, and 1100c may be synchronized via the synchronization signal to transmit image data to the application processor 1200.

[0132] In some example embodiments, the control signals provided from the camera module controller 1216 to each of the camera modules 1100a, 1100b, and 1100c may include mode information based on a mode signal. Based on the mode information, the plurality of camera modules 1100a, 1100b, and 1100c may operate in a first operating mode and a second operating mode with respect to the sensing speed.

[0133] In the first operating mode, multiple camera modules 1100a, 1100b, and 1100c can generate image signals at a first speed (e.g., at a first frame rate), encode the image signals at a second speed (e.g., at a second frame rate higher than the first frame rate), and transmit the encoded image signals to the application processor 1200. In this case, the second speed can be 30 times or less than the first speed.

[0134] Application processor 1200 can store the received image signal, i.e., the encoded image signal, in internal memory 1230 provided therein or in external memory 1400 located outside application processor 1200. Subsequently, application processor 1200 can read and decode the encoded image signal from internal memory 1230 or external memory 1400, and can display image data generated based on the decoded image signal. For example, one of the sub-image processors 1212a, 1212b, and 1212c of image processing device 1210 can perform decoding and can also perform image processing on the decoded image signal.

[0135] In the second operating mode, multiple camera modules 1100a, 1100b, and 1100c can generate image signals at a third speed (e.g., a third frame rate image signal lower than the first frame rate) and transmit the image signals to the application processor 1200. The image signals provided to the application processor 1200 can be unencoded signals. The application processor 1200 can perform image processing on the received image signals or store the image signals in internal memory 1230 or external memory 1400.

[0136] PMIC 1300 can supply power, such as power supply voltage, to multiple camera modules 1100a, 1100b, and 1100c respectively. For example, under the control of application processor 1200, PMIC 1300 can supply a first power to camera module 1100a via power signal line PSLa, a second power to camera module 1100b via power signal line PSLb, and a third power to camera module 1100c via power signal line PSLc.

[0137] In response to a power control signal PCON from the application processor 1200, the PMIC 1300 can generate power corresponding to each of the plurality of camera modules 1100a, 1100b, and 1100c, and can adjust the power level. The power control signal PCON can include a power adjustment signal for each operating mode of the plurality of camera modules 1100a, 1100b, and 1100c. For example, the operating mode can include a low-power mode. In this case, the power control signal PCON can include information about the camera module operating in low-power mode and a set power level. The power levels supplied to the plurality of camera modules 1100a, 1100b, and 1100c respectively can be the same or different from each other. Additionally, the power level can be dynamically changed.

[0138] According to some exemplary embodiments of this disclosure, the signal-to-noise ratio of an image sensor can be improved. In particular, according to some exemplary embodiments of this disclosure, noise in the output signal can be effectively reduced in video recording mode.

[0139] Although this disclosure has been described with reference to some exemplary embodiments thereof, it will be apparent to those skilled in the art that various changes and modifications may be made to this disclosure without departing from the spirit and scope of the disclosure as set forth in the appended claims.

Claims

1. An image processing apparatus, the image processing apparatus comprising: An image sensor, the image sensor including a pixel array, the pixel array being configured to: Based on the light received from the object, a first pixel signal corresponding to the first conversion gain is generated from the pixels of the shared floating diffusion region, and Based on the light received reflected from the object, a second pixel signal corresponding to the second conversion gain is generated, and The image sensor is configured to generate first image data and second image data based on the first pixel signal and the second pixel signal, respectively. as well as An image signal processor configured to generate an output image based on the first image data and the second image data, the image signal processor comprising: A standardization circuit, configured to standardize the first image data based on the dynamic range of the second image data to generate third image data; as well as A hybrid circuit configured to generate the output image based on the second image data and the third image data in response to the ambient illumination of the object.

2. The image processing apparatus according to claim 1, wherein, The first pixel signal corresponds to the charge stored in the floating diffusion region having a first capacitance value, and The second pixel signal corresponds to the charge stored in the floating diffusion region having a second capacitance value.

3. The image processing apparatus according to claim 2, wherein, The pixel array includes a capacitor connected to the floating diffusion region such that the floating diffusion region has the second capacitance value.

4. The image processing apparatus according to claim 1, wherein, The hybrid circuit is configured as follows: Assign a first weight to the third image data, and A second weight is assigned to the second image data.

5. The image processing apparatus according to claim 4, wherein, Each of the first weight and the second weight has a value between "0" and "1", and the sum of the first weight and the second weight is "1".

6. The image processing apparatus according to claim 1, wherein, The pixels in the pixel array include a first pixel controlled based on a first exposure time and a second pixel controlled based on a second exposure time, and Wherein, based on the fact that the number of the first pixel and the number of the second pixel are equal, the mixing circuit is configured to generate the output image based on the second image data and the third image data.

7. An electronic device, the electronic device comprising: A lens, configured to receive light reflected from an object; An image sensor, the image sensor including a pixel array, the pixel array being configured to: A first pixel signal corresponding to a first conversion gain and a second pixel signal corresponding to a second conversion gain are generated from pixels in the shared floating diffusion region, and The image sensor is configured to receive light from the lens and generate first image data and second image data based on the first pixel signal and the second pixel signal, respectively. An image signal processor configured to generate an output image based on the first image data and the second image data; as well as The main processor is configured to generate a video stream based on the output image. The image signal processor includes: A standardization circuit, configured to standardize the first image data based on the dynamic range of the second image data to generate third image data; as well as A hybrid circuit configured to generate the output image based on the second image data and the third image data in response to the ambient illumination of the object.

8. The electronic device according to claim 7, wherein, The first pixel signal corresponds to the charge stored in the floating diffusion region having a first capacitance value, and The second pixel signal corresponds to the charge stored in the floating diffusion region having a second capacitance value.

9. The electronic device according to claim 8, wherein, The pixel array includes a capacitor connected to the floating diffusion region such that the floating diffusion region has the second capacitance value.

10. The electronic device according to claim 7, wherein, The hybrid circuit is configured to assign a first weight to the third image data and a second weight to the second image data.

11. The electronic device according to claim 10, wherein, Each of the first weight and the second weight has a value between "0" and "1", and the sum of the first weight and the second weight is "1".

12. The electronic device according to claim 7, wherein, The main processor is further configured to determine the operation to activate the hybrid circuit based on received user input.

13. The electronic device according to claim 7, wherein, The main processor is also configured to output the video stream as a noise-reduced preview image to the display device of the electronic device.

14. The electronic device according to claim 7, wherein, The pixels in the pixel array include a first pixel controlled based on a first exposure time and a second pixel controlled based on a second exposure time, and Based on the fact that the number of the first pixel and the number of the second pixel are equal, the mixing circuit is configured to generate the output image based on the second image data and the third image data.

15. A method of operating an apparatus for noise reduction, the method comprising: Generate first image data corresponding to a first conversion gain and second image data corresponding to a second conversion gain; The first image data is nominalized based on the dynamic range of the second image data to generate the third image data; An output image is generated based on the second image data and the third image data; Based on the ambient illumination of the object corresponding to the first image data and the second image data, it is determined whether to perform the generation of the third image data and the generation of the output image; as well as A video stream is generated based on the output image.

16. The method according to claim 15, wherein, The first conversion gain is less than the second conversion gain.

17. The method according to claim 15, wherein, Generating the output image includes: A first weight and a second weight are assigned to the third image data and the second image data, respectively.

18. The method according to claim 17, wherein, Each of the first weight and the second weight has a value between "0" and "1", and the sum of the first weight and the second weight is "1".

19. The method according to claim 15, further comprising: The video stream is output to a display device as a noise-reduced preview image.

Images