Image sensor, image sensing device including same, and operating method

Abstract

An image sensor includes: a pixel array arranged in a Bayer pattern and including pixels that generate charges according to received incident light; and analog logic configured to convert an analog signal output from at least one of the pixels into a first digital code using an analog-to-digital converter, and to convert the first digital code into a second digital code by adjusting the low-order bits of the first digital code in response to a control signal.

Description

Technical Field

[0001] The present invention concept is generally related to image sensors. Prior Art

[0002] Image sensing devices can be used in various electronic devices including, for example, mobile devices such as smart phones, tablets, digital cameras, and the like. The image sensing device can include a two-dimensional pixel arrangement configured to generate an analog electrical signal corresponding to the intensity (or luminance) of incident electromagnetic energy (e.g., visible light). Thereafter, an analog-to-digital converter (ADC) can be used to convert the analog signal into a corresponding digital signal.

[0003] Representative image sensors include charge coupled devices (CCDs) and complementary metal-oxide semiconductor (CMOS) image sensors. Compared to CMOS image sensors, CCDs exhibit low noise and excellent image quality. However, CMOS image sensors have a simple driving method and can be implemented using various scanning methods. In addition, CMOS image sensors can be easily integrated with signal processing circuits on a single semiconductor chip. This ability enables physical downsizing of the constituent electronic devices, and CMOS process technology can be interchanged with existing product fabrication lines to reduce overall manufacturing costs. CMOS image sensors also operate at relatively low power consumption, which allows for practical applications within a wide range of mobile devices.

[0004] CMOS image sensors include an ADC configured to convert an analog image signal into a corresponding digital signal. The pixel array of the image sensor generally includes pixels arranged in a two-dimensional (2D) matrix (e.g., columns and rows), where each pixel outputs an image signal corresponding to the received incident light. In this regard, each of the pixels accumulates a photoelectric charge corresponding to the level of incident light via a photodiode and outputs a pixel signal as an analog current that varies in relation to the accumulated photoelectric charge. It is this analog pixel signal that can be converted into a corresponding digital signal using an ADC. Therefore, a number of ADC circuits equal to the number of column lines in the pixel array are typically required.

[0005] In various applications of CMOS image sensors, various signal processing techniques can be used to generate high-quality image signals. SUMMARY OF THE INVENTION

[0006] Embodiments of the inventive concept provide an image sensing device capable of reducing power consumption of digital logic.

[0007] Embodiments of the inventive concept provide an image sensing device that can operate with reduced overall power consumption while maintaining an acceptable signal-to-noise ratio (SNR) and image quality.

[0008] In one aspect, embodiments of the inventive concept provide an image sensor including: a pixel array arranged in a Bayer pattern and including pixels that generate charges respectively according to received incident light; and analog logic configured to convert an analog signal output from at least one of the pixels into a first digital code using analog-to-digital conversion, and convert the first digital code into a second digital code by adjusting low-order bits of the first digital code in response to a control signal.

[0009] In another aspect, embodiments of the inventive concept provide a method of operating an image sensing device. The method includes: generating an analog signal corresponding to charges generated in response to incident light using at least one pixel; converting the analog signal from an output line of the at least one pixel to generate a first digital code; and adjusting low-order bits of the first digital code in response to a first control signal to generate a second digital code.

[0010] In another aspect, embodiments of the inventive concept provide an image sensing device including: a lens that receives incident light; a pixel array including pixels configured to generate charges in response to the incident light and output an analog signal corresponding to the charges; a column decoder configured to enable at least one of the pixels; first logic configured to receive the analog signal from an output line of the pixel enabled by the column decoder and convert the analog signal into an N-bit first digital code; second logic configured to process an N-bit second digital code to output image data; and a control circuit configured to control operations of the first logic and the second logic, wherein the second digital code includes the remaining (N-k) high-order bits other than k low-order bits of the first digital code, N and k are natural numbers, and N is greater than k.

[0011] However, the notable aspects and features of the inventive concept are not limited solely to those explicitly set forth herein, and other aspects, features, and technical advantages will be recognized by those skilled in the art upon consideration of the following detailed description. Brief Description of the Drawings

[0012] Upon consideration of several exemplary embodiments described with reference to the accompanying drawings, the foregoing and other aspects and features of the inventive concept will be fully understood, in which: FIG. 1 is a block diagram showing an image sensing device 1 according to an embodiment of the inventive concept. FIG. 2 is a conceptual diagram showing a part of an exemplary pixel array according to an embodiment of the inventive concept. FIG. 3 is a circuit diagram showing a pixel structure according to an embodiment of the inventive concept in one example. FIG. 4 is a conceptual diagram showing an output signal that may be provided by an analog-to-digital converter during operation of an image sensing device according to an embodiment of the inventive concept in one example. FIG. 5 is a block diagram showing a peripheral circuit of an image sensor according to an embodiment of the inventive concept in one example. FIG. 6 is a graph showing the number of bits of signal level reduction in one example of the operation of an image sensing device according to an embodiment of the inventive concept. FIG. 7 is a graph showing the number of gain output bits in one example of the operation of an image sensing device according to an embodiment of the inventive concept. FIG. 8 is a block diagram showing a peripheral circuit of an image sensor according to an embodiment of the inventive concept. FIG. 9 is a perspective view showing an image sensor 700 that may be used as an alternative form of the image sensor 200 in the image sensing device 1 shown in FIG. 1. FIG. 10 is a block diagram showing an electronic device including a multi-camera module according to an embodiment of the inventive concept. FIG. 11 is a block diagram further showing the multi-camera module shown in FIG. 10 in one example. Embodiments

[0013] Throughout the written description and all the drawings, the same reference numerals and labels are used to denote the same or similar elements and / or features. Terms such as "unit", "module", and similar terms used in the functional blocks shown in the detailed description or the drawings may be implemented in various ways in hardware, software, or a combination of hardware / software to perform the functions described.

[0014] FIG. 1 is a block diagram showing an image sensing device 1 according to an embodiment of the inventive concept.

[0015] Referring to FIG. 1, the image sensing device 1 can be implemented in various ways as, for example, a digital camera, a video camera, a mobile phone, a smart phone, a tablet personal computer (PC), a personal digital assistant (PDA), a mobile Internet device (MID), a wearable computer, an Internet of things (IoT) device, or an Internet of everything (IoE) device, etc., portable electronic devices.

[0016] In some embodiments, the image sensing device 1 may include a display unit 300, a digital signal processor (DSP) 150, and an image sensor 200 (e.g., a CMOS image sensor).

[0017] The image sensor 200 may include a pixel array 210, a column driver 220, a correlated double sampling (CDS) block 230, an analog digital converter (ADC) 100, a ramp generator 260, a timing generator 270, a control register block 280, and a buffer 290.

[0018] In this regard, the image sensor 200 can be used to image (or sense) an object 400 through a lens 500 under the control of the DSP 150. Here, the DSP 150 can output the image sensed by the image sensor 200 to the display unit 300.

[0019] In some embodiments, the image sensor 200 can receive a raw image from the pixel array 210 and perform analog binning using the ADC 100 and the buffer 290 to output the binned image to the DSP 150.

[0020] The display unit 300 includes a device capable of outputting or displaying an image. For example, the display unit 300 can be a computer, a mobile communication device, and other image output terminals.

[0021] The DSP 150 includes a camera control 152, an image signal processor (ISP) 151, and an interface (I / F) 153.

[0022] The camera control 152 can be used to control the operation of the control register block 280. The camera control 152 can use the inter-integrated circuit (I2C) to control the operation of the image sensor 200 (i.e., the control register block 280), but the scope of the inventive concept is not limited thereto.

[0023] The ISP 151 receives the image data output from the buffer 290, processes or disposes of the received image data so that the data can be easily viewed by humans, and outputs the processed or disposed image data to the display unit 300 via the I / F 153.

[0024] In some embodiments, the ISP 151 can process the image data output from the image sensor 200. The ISP 151 can output a digitally merged image to the display unit 300 as the final merged image. In such a case, according to some embodiments, the image output from the image sensor 200 can be the original image from the pixel array, or according to some embodiments, the image output from the image sensor 200 can be a merged image. Although various embodiments are possible, for the sake of convenience in the present written description, it is assumed that the image data is output from the image sensor 200.

[0025] Although the embodiment shown in FIG. 1 shows the ISP 151 located in the DSP 150, in other embodiments of the inventive concept, the ISP 151 can be located in the image sensor 200 in other ways. In some embodiments, the image sensor 200 and the ISP 151 can be implemented as a single semiconductor package (e.g., a multi-chip package (MCP)).

[0026] The pixel array 210 includes pixels arranged in a matrix (e.g., including columns and rows). Each of the pixels can include a photosensitive device (or a photoelectric conversion device) and a readout circuit that outputs a pixel signal (e.g., an analog signal) corresponding to the charge generated by the photosensitive device. The photosensitive device can be implemented as, for example, a photodiode or a pinned photodiode.

[0027] The column driver 220 can selectively enable each of the pixels. For example, the column driver 220 can drive the pixels implemented in the pixel array 210 in units of columns. For example, the column driver 220 can generate control signals capable of controlling the operations of the pixels included in each of the columns.

[0028] In response to various control signals, the pixel signals output from each of the pixels can be transmitted to the CDS block 230.

[0029] Therefore, the CDS block 230 can include a plurality of CDS circuits. In some embodiments, each of the CDS circuits can perform correlated double sampling on the pixel values respectively output by the row lines implemented in the pixel array 210 in response to at least one switching signal output from the timing generator 270, and can compare the correlated double sampled pixel values with the ramp signal output from the ramp generator 260 to output corresponding comparison signals.

[0030] The ADC block 100 can convert each of the output comparison signals into digital signals, and output the plurality of digital signals to the buffer 290.

[0031] The ramp generator 260 outputs a ramp signal Vramp to the CDS block 230. The ramp signal Vramp ramps from a reference level to be compared with the reset signal Vrst, rises to the reference level, and then ramps again from the reference level to be compared with the image signal Vim.

[0032] The timing generator 270 can be used to control the operations of the column driver 220, the ADC block 100, the CDS block 230, and the ramp generator 260 under the control of the control register block 280.

[0033] In this regard, the control register block 280 can control the operations of the timing generator 270, the ramp generator 260, and the buffer 290 in relation to the DSP 150.

[0034] The buffer 290 can transmit the image data corresponding to the digital signals provided by the ADC block 100 (e.g., the pixel array ADC output) to the DSP 150.

[0035] The original image data output from the pixel array 210 by the CDS block 230 and the ADC block 100 may be Bayer image data formed in a Bayer format. That is, the Bayer image data may be processed by the ISP 151 using red, green and blue (RGB) format data and output to the display unit 300.

[0036] FIG. 2 is a conceptual diagram showing a pixel array according to an embodiment of the inventive concept, and FIG. 3 is a circuit diagram further showing a pixel structure according to an embodiment of the inventive concept in one example.

[0037] Referring to FIG. 2, the pixel array 210 may be arranged in a Bayer pattern. The Bayer pattern includes rows in which R (red) and Gr (green) pixels are repeatedly arranged and rows in which Gb (green) and B (blue) pixels are repeatedly arranged. Therefore, the original image also has a Bayer pattern image, in which columns in which Gr pixels and R pixels are sequentially arranged and columns in which B pixels and Gb pixels are sequentially arranged are alternately arranged in the Bayer pattern image.

[0038] The original image may be operated in a kernel unit (K). According to some embodiments, the kernel may include at least one R (red) pixel, at least one Gr (green) pixel, at least one Gb (green) pixel, and at least one B (blue) pixel. Here, the original image is expressed as a "kernel" and, according to other embodiments, is alternatively referred to as a window or a unit.

[0039] An exemplary kernel has a size of 2×2, but the scope of the inventive concept is not limited thereto, and other embodiments of the inventive concept may use kernels determined to be of different sizes (e.g., 3×3, 2×4, 5×5, etc.).

[0040] Referring to FIG. 3, each pixel (e.g., one of P1 to P4) may include one or more photodiodes (e.g., PD1 to PD4), one or more transfer transistors (e.g., M1_1 to M1_4), a reset transistor (e.g., M2), a source follower transistor (e.g., M3), and a selection transistor (e.g., M4). Here, the various photodiodes may have different colors or the same color.

[0041] Control signals S_TG1 to S_TG4, S_RESET, and S_SEL capable of separately controlling the transistors M1 to M4 may be output from the column driver 220. The output signal of the selection transistor M4 is supplied to the row line.

[0042] In some embodiments, only one photodiode may be connected to transistors M2, M3, and M4. The pixel includes a photodiode and a transfer transistor. As another option, two or more photodiodes may share transistors M2, M3, and M4. When two or more pixels share transistors M2, M3, and M4, by sharing the floating diffusion region, the size of the photodiode PD region can be increased without increasing the size of the entire pixel.

[0043] FIG. 4 is a conceptual diagram showing an output signal that may be provided by an analog-to-digital converter during the operation of an image sensing device according to an embodiment of the inventive concept in one example, and FIG. 5 is a block diagram showing the peripheral circuit of an image sensor according to an embodiment of the inventive concept in one example.

[0044] Referring to FIGS. 4 and 5, when an analog signal is amplified, noise is also amplified. For example, when an analog signal to be input is converted into an N-bit signal, it may be assumed that the noise among the N-bit signals occupies approximately one bit. When the analog gain is high during signal conversion, the noise level also becomes higher. That is, the signal-to-noise ratio (SNR) can be calculated as follows: ... [Equation 1].

[0045] However, noise tends to be mainly distributed in a specific frequency band. Therefore, when the region corresponding to the frequency band in which the noise signal among the conversion signals is mainly distributed is locally clipped, the noise level can be reduced while maintaining the SNR as shown below: ... [Equation 2].

[0046] For example, in an illustrated example, when the analog signal is n = 13 bits and the noise level is 5 bits (a = 5), it is assumed that 3 bits (b = 3) are clipped. When the first digital code (N = 13) is 1010101110101, the 3 low-order bits 101 can be clipped to remove the noise, and then 1010101110 (10 bits) can be output. In this case, since the signal level is maintained at 8 bits and only the noise level is reduced from 5 bits to 2 bits, the noise can be removed while maintaining the overall SNR.

[0047] In an image sensor, each of the pixel signals output from the pixel array 210 is input to analog logic. The analog signal input to the analog logic is a signal in which the pixel signal is correlated double sampled by the CDS block 230 as described with respect to FIG. 1, but for convenience of explanation, is referred to as an analog signal.

[0048] The analog logic converts the analog signal output from at least one pixel among the pixels into a first digital code by analog-to-digital conversion, and the low-order bit (k) of the first digital code (N bits) of the control signal can be adjusted to be converted into a second digital code (N-k bits). That is, the second digital code can be the remaining high-order bits of the first digital code except for the low-order bit k.

[0049] In some embodiments, for each color of the pixel array, the low-order bit k to be clipped can be set differently. In some embodiments, according to the pixel position in the pixel array, the low-order bit k to be clipped can be set differently.

[0050] Referring to FIG. 5, the analog logic 610 may include an ADC 100, an ADC controller 611, and a bit shifter 612. Although not shown here, according to some embodiments, the analog logic 610 may further include one or more of the peripheral circuits in the image sensor 200 shown in FIG. 1 (for example, the column driver 220, the CDS block 230, the ramp generator 260, the control register block 280, the buffer 290, etc.).

[0051] The ADC 100 receives an analog signal from at least one pixel output line associated with the pixel, amplifies the analog signal according to an analog gain, and then converts the analog signal into a first digital code (for example, including N bits, where "N" is a natural number), and outputs the first digital code. In this case, the analog gain can be adjusted according to the control signal provided by the ADC controller 611.

[0052] Therefore, the ADC controller 611 can generate a control signal by receiving a gain adjustment signal G and a user input signal I.

[0053] The bit shifter 612 receives a first digital code (N bits), shifts k lower-order bits, and outputs the resulting second digital code (N-k bits). In some cases, "k" is an integer greater than or equal to zero that is adjusted by a control signal based on the user input signal I. Here, in some embodiments, the ADC controller 611 may be included in whole or in part within the ADC 100, in some embodiments, the ADC controller 611 may be included in whole or in part within the control register block 280, and / or according to some embodiments, the ADC controller 611 may be included as a separate component within the image sensor 200.

[0054] The digital logic 650 may receive the second digital code (N-k bits), convert the second digital code into another code having N bits, and output this code. Thus, in some embodiments, the digital logic 650 may include a random bit generator (RNG) 651. The random bit generator 651 may be used to generate a number of random bits. Here, the number of random bits may be a number defined by a control signal based on the user input signal I and may be the same as the number of bits ( "k") shifted by the bit shifter 612. That is, the digital logic 650 may output an N-bit "third digital code" by providing and including k random bits that substantially replace the number of lower-order bits (e.g., the second digital code (N-k bits)).

[0055] FIG. 6 is a graph showing the number of bits with reduced signal levels illustrating the operation of the image sensing device according to an embodiment of the present invention concept, and FIG. 7 is a graph showing the number of gain output bits further illustrating the operation of the image sensing device according to an embodiment of the present invention concept.

[0056] Referring to FIG. 6, when the gain level is low, the ADC controller 611 may adjust the number of bits (k) to be clipped according to the signal level. When the signal level of the analog signal is high, since most of the lower-order bits of the converted first digital code become noise, the influence of the noise can be reduced by clipping some of the lower-order bits of the first digital code.

[0057] Referring to FIG. 7, since the noise is amplified together with the signal as the gain level (e.g., G L1 to G Ln) of the ADC 100 increases, the number of bits to be clipped (shift #) in the first digital code (output bits) may also increase.

[0058] That is, on the premise that the SNR remains constant, the number of lower-order bits to be clipped can also vary according to the number of bits of the first digital code. Consistent with Equation 1 above, since the noise level increases as the number of bits of the first digital code increases, the number of bits to be clipped (bit reduction) may also increase. As an example, in the case of code level CL2, two lower-order bits can be clipped, or as another example, in the case of code level CLn, n lower-order bits can be clipped, and thus the noise level can be locally removed while maintaining the overall SNR.

[0059] As shown with respect to FIGS. 6 and / or 7, when the lower-order bits are clipped, the switching power of the transistors can be reduced when the digital logic 650 processes the clipped signal (i.e., the second digital signal).

[0060] FIG. 8 is a block diagram showing a peripheral circuit of an image sensor according to an embodiment of the concept of the present invention in one example.

[0061] Here, the analog logic 620 converts an analog signal output from at least one of the plurality of pixels into a first digital code by analog-to-digital conversion, and the lower-order bits (k) of the first digital code (N bits) of the control signal can be adjusted to be converted into a second digital code (N' bits).

[0062] Referring to FIG. 8, the analog logic 620 may include an ADC 100, an ADC controller 621, and a zero-padding unit 622. Although not shown, in some embodiments, the analog logic 620 may further include one or more of the peripheral circuits of the image sensor 200 shown in FIG. 1 (e.g., a column driver 220, a CDS block 230, a ramp generator 260, a control register block 280, a buffer 290, or the like).

[0063] The ADC 100 can receive an analog signal from an output line of at least one of the pixels, amplify the analog signal according to an analog gain, and then convert the analog signal into a first digital code (N bits) and output the first digital code. In this case, the analog gain can be adjusted according to a control signal of the ADC controller 621.

[0064] The ADC controller 621 can generate a control signal by receiving a gain adjustment signal and a user input signal.

[0065] The zero-padding unit 622 clips k lower-order bits (the second digital code) from the first digital code (N bits), and pads k zeros (0) to replace the clipped lower-order bits. That is, as many zeros as the number of lower-order bits excluded from the first digital code can be padded and output as the third digital code. The analog logic 620 outputs the third digital code (N' bits) processed by the zero-padding unit 622. In this case, k is an integer greater than or equal to zero adjusted by a control signal based on the user input signal I.

[0066] According to some embodiments, the ADC controller 621 may be included in whole or in part in the ADC 100, according to some embodiments, the ADC controller 621 may be included in whole or in part in the control register block 280, and / or according to some embodiments, the ADC controller 621 may be included as a separate component in the image sensor 200.

[0067] Here, the digital logic 650 performs digital processing on the third digital code (N' bits) to output the final digital signal. Contrary to the embodiment shown in FIG. 5, the digital logic 650 may not have a separate configuration for adding lower-order bits.

[0068] FIG. 9 is a perspective view showing an image sensor 700 that can be used as an alternative to the image sensor 200 in the image sensing device 1 shown in FIG. 1.

[0069] Referring to FIG. 9, the image sensor 700 may include a vertical layer stack. Here, the pixel array 210 may be implemented in the first layer "Layer 1", and the remaining part of the image sensor 200 (e.g., the logic circuit "LOGIC") may be implemented in the second layer "Layer 2". The logic circuit "LOGIC" may include all components of the image sensor 200 other than the pixel array 210. That is, assuming that the main surface of the wafer corresponds to the horizontal plane, the pixel array region and the logic circuit region may be vertically stacked at the wafer level.

[0070] The first layer "Layer 1" may include a sensing region SA and a first pad region PA1. The sensing region SA includes pixels PX, and the first pad region PA1 is disposed on the periphery of the sensing region SA. The upper pad PAD1 may be included in the first pad region PA1 and may be connected to the logic circuit "LOGIC" in the second pad region PA2 of the second layer "Layer 2" and the pads PAD21 and PAD22 through vias.

[0071] In some embodiments, pads PAD21 and PAD22 may be input I / F pads, and pad PAD23 may be an output I / F pad.

[0072] FIG. 10 is a block diagram of an electronic device 1000 including a multi-camera module according to an embodiment of the inventive concept, and FIG. 11 is a block diagram further showing the multi-camera module shown in FIG. 10.

[0073] Referring to FIG. 10, the electronic device 1000 may include a camera module group 1100, an application processor 1200, a power management integrated circuit (PMIC) 1300, and an external memory 1400.

[0074] The camera module group 1100 may include, for example, camera modules 1100a, 1100b, and 1100c. Thus, FIG. 10 shows an example in which the camera module group 1100 includes three (3) camera modules, but the scope of the inventive concept is not limited thereto. As another option, the camera module group 1100 may include two (2) camera modules, four (4) camera modules, etc.

[0075] An exemplary structure of the camera module 1100b will be described below with reference to FIG. 11. The following description can be directly applied to other camera modules of the camera module group 1100, namely, camera modules 1100a and 1100c.

[0076] Referring to FIG. 11, the camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140, and a storage unit 1150.

[0077] The prism 1105 may include a reflective surface 1107 made of a light-reflective material and may thus change the path of incident light L received from an external source.

[0078] In some embodiments, the prism 1105 may change the path of the incident light L in relation to the first arbitrary direction X and the second direction Y substantially perpendicular to the first direction X. In addition, the prism 1105 may change the path of the incident light L from the first direction X to the second direction Y by rotating the light L about the central axis 1106 of the reflective surface 1107 in a "A" motion and / or a "B" motion. In such a case, the OPFE 1110 may move in a third direction Z substantially perpendicular to the first direction X and the second direction Y.

[0079] In some embodiments, the maximum rotation angle of the prism 1105 may be 15 degrees or less than 15 degrees in the "+A" motion, and may be 15 degrees or greater than 15 degrees in the "-A" motion, but the scope of the inventive concept is not limited thereto.

[0080] In some embodiments, the prism 1105 may move at an angle of about 20°, about 10° to 20°, or about 15° to about 20° in the "+B" motion or the "-B" motion. The angle by which the prism 1105 moves may be the same in both the "+B" motion and the "-B" motion, or may be almost similar, differing by only about 1°.

[0081] In some embodiments, the prism 1105 may move in a third direction Z parallel to the extension direction of the central axis 1106 of the reflective surface 1107.

[0082] The OPFE 1110 may include, for example, a set of m optical lenses (where "m" is a natural number). Here, the m optical lenses may move in the second direction Y to change the optical zoom ratio of the camera module 1100b. For example, if the preset optical zoom ratio of the camera module 1100b is Z, the optical zoom ratio of the camera module 1100b may be changed to 3Z or 5Z (or greater than 5Z) by moving the m optical lenses of the OPFE 1110.

[0083] The actuator 1130 may move the OPFE 1110 or the optical lens to a specific position. For example, the actuator 1130 may adjust the position of the optical lens so that the image sensor 1142 may be located at the focal length of the optical lens for accurate sensing.

[0084] The image sensing device 1140 may include an image sensor 1142, control logic 1144, and a memory 1146. The image sensor 1142 may sense an image of a target object using incident light L provided via an optical lens. The control logic 1144 may control the general operation of the camera module 1100b. For example, the control logic 1144 may control the operation of the camera module 1100b according to a control signal provided via a control signal line CSLb.

[0085] The memory 1146 may store information necessary for the operation of the camera module 1100b, such as calibration data 1147. The calibration data 1147 may include information necessary for generating image data using the incident light L. The calibration data 1147 may include, for example, rotation degree information, focal length information, and optical axis information. In a case where the camera module 1100b is implemented as a multi-state camera whose focal length (local distance) varies according to the position of the optical lens, the calibration data 1147 may include the focal lengths of different positions (or different states) of the optical lens and autofocus information.

[0086] The storage unit 1150 may store the image data sensed by the image sensor 1142. The storage unit 1150 may be disposed outside the image sensing device 1140 and may be implemented to be stacked on the sensor chip forming the image sensing device 1140. In some embodiments, the storage unit 1150 may be implemented as an electrically erasable programmable read-only memory (EEPROM), but the concept of the present invention is not limited thereto.

[0087] Referring to FIGS. 10 and 11, in some embodiments, the camera modules 1100a, 1100b, and 1100c may include their own actuators 1130. Depending on the operation of their own actuators 1130, the camera modules 1100a, 1100b, and 1100c may include the same calibration data 1147 or different calibration data 1147.

[0088] In some embodiments, one of the camera modules 1100a, 1100b, and 1100c (e.g., the camera module 1100b) may be a folded lens type camera module including a prism 1105 and an OPFE 1110, and the other camera modules (i.e., the camera modules 1100a and 1100c) may be vertical camera modules that do not include the prism 1105 and the OPFE 1110. However, the scope of the concept of the present invention is not limited thereto.

[0089] In some embodiments, the camera module 1100c can be, for example, a depth camera capable of using infrared (IR) light to extract depth information. In such a case, the application processor 1200 can generate a three-dimensional (3D) depth image by combining the image data provided by the camera module 1100c and the image data provided by other camera modules (i.e., the camera modules 1100a and 1100b).

[0090] In some embodiments, at least two of the camera modules 1100a, 1100b, and 1100c can have different fields of view. In such a case, at least two of the camera modules 1100a, 1100b, and 1100c (e.g., the camera modules 1100a and 1100b) can have different optical lenses, but the present inventive concept is not limited thereto.

[0091] In addition, in some embodiments, the camera modules 1100a, 1100b, and 1100c can have different fields of view. In such a case, the camera modules 1100a, 1100b, and 1100c can have different optical lenses, but the present inventive concept is not limited thereto.

[0092] In some embodiments, the camera modules 1100a, 1100b, and 1100c can be physically separated from each other. That is, the camera modules 1100a, 1100b, and 1100c may not share the sensing area of a single image sensor 1142 together, but independent image sensors 1142 can be provided in the camera modules 1100a, 1100b, and 1100c.

[0093] Referring to FIG. 10, the application processor 1200 can include an image processor 1210, a memory controller 1220, and an internal memory 1230. The application processor 1200 can be implemented separately from the camera modules 1100a, 1100b, and 1100c. For example, the application processor 1200 and the camera modules 1100a, 1100b, and 1100c can be implemented as separate semiconductor chips.

[0094] The image processor 1210 can include a plurality of sub-image processors 1212a, 1212b, and 1212c, an image generator 1214, and a camera module controller 1216.

[0095] The image processor 1210 may include as many sub-image processors as there are existing camera modules (e.g., sub-image processors 1212a, 1212b, and 1212c corresponding to camera modules 1100a, 1100b, and 1100c, respectively).

[0096] The image data generated by camera modules 1100a, 1100b, and 1100c may be provided to sub-image processors 1212a, 1212b, and 1212c via separate image signal lines ISLa, ISLb, and ISLc. For example, the image data generated by camera module 1100a may be provided to sub-image processor 1212a via image signal line ISLa, the image data generated by camera module 1100b may be provided to sub-image processor 1212b via image signal line ISLb, and the image data generated by camera module 1100c may be provided to sub-image processor 1212c via image signal line ISLc. The transmission of the image data may be performed via, for example, a camera serial interface (CIS) based on a mobile industry processor interface (MIPI), but the inventive concept is not limited thereto.

[0097] In some embodiments, a single sub-image processor may be provided to correspond to multiple camera modules. For example, sub-image processors 1212a and 1212c may be incorporated into a single integrated sub-image processor, and the image data provided by camera module 1100a or the image data provided by camera module 1100c may be selected by a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor.

[0098] The image data provided to each of sub-image processors 1212a, 1212b, and 1212c may be provided to the image generator 1214. The image generator 1214 may generate an output image using the image data provided to the image generator 1214 from each of sub-image processors 1212a, 1212b, and 1212c according to image generation information “generating information” or a mode signal “Mode Signal”.

[0099] Specifically, the image generator 1214 can generate an output image by combining at least some of the image data provided by the camera modules 1100a, 1100b, and 1100c having different fields of view according to the image generation information "generating information" or the mode signal "Mode Signal". In addition, the image generator 1214 can generate an output image by selecting one of the image data generated by the camera modules 1100a, 1100b, and 1100c having different fields of view according to the image generation information "generating information" or the mode signal "Mode Signal".

[0100] In some embodiments, the image generation information "generating information" may include a zoom signal or a zoom factor. In some embodiments, the mode signal "Mode Signal" may be, for example, a signal based on the mode selected by the user.

[0101] In a case where the image generation information "generating information" is a zoom signal or a zoom factor and the camera modules 1100a, 1100b, and 1100c have different fields of view, the image generator 1214 can perform different operations according to the type of the zoom signal. For example, if the zoom signal is a first signal, the image data provided by the camera module 1100a can be combined with the image data provided by the camera module 1100c, and the combined image data and the image data provided by the camera module 1100b can be used to generate an output image. And if the zoom signal is a second signal different from the first signal, one of the image data provided by the camera module 1100a, the image data provided by the camera module 1100b, and the image data provided by the camera module 1100c can be selected, and the selected image data can be used to generate an output image. However, the scope of the inventive concept is not limited thereto, and the method of processing the image data can vary.

[0102] In some embodiments, the image generator 1214 can receive image data having different exposure times from at least one of the sub-image processors 1212a, 1212b, and 1212c, and can perform high dynamic range processing on the image data, thereby generating combined image data having an enhanced dynamic range.

[0103] The camera module group 1100 may include the image sensor 200 shown in FIG. 1.

[0104] The ISP 151 shown in FIG. 1 can be implemented inside the camera module 1100 shown in FIG. 11, in the sub-image processors 1212a, 1212b, and 1212c shown in FIG. 10, or in the image generator 1214 shown in FIG. 10.

[0105] The camera module controller 1216 can provide control signals to the camera modules 1100a, 1100b, and 1100c. The control signals provided by the camera module controller 1216 can be provided to the camera modules 1100a, 1100b, and 1100c via separate control signal lines CSLa, CSLb, and CSLc.

[0106] One of the camera modules 1100a, 1100b, and 1100c (e.g., the camera module 1100b) can be designated as the master camera according to the mode signal "Mode Signal" or the image generation information "Generating Information" including the zoom signal, and the other camera modules (i.e., the camera modules 1100a and 1100c) can be designated as slave cameras. This type of information can be included in the control signal and then provided to the camera modules 1100a, 1100b, and 1100c via separate control signal lines CSLa, CSLb, and CSLc.

[0107] The master camera module and the slave camera module can be changed according to the zoom factor or the mode signal "Mode Signal". For example, if the camera module 1100a has a wider field of view than the camera module 1100b, but has a zoom ratio with a smaller zoom factor than the camera module 1100b, then the camera module 1100b can operate as the master, and the camera module 1100a can operate as the slave. Conversely, if the camera module 1100a has a zoom ratio with a larger zoom factor than the camera module 1100b, then the camera module 1100a can operate as the master, and the camera module 1100b can operate as the slave.

[0108] In some embodiments, the control signals provided by the camera module controller 1216 to the camera modules 1100a, 1100b, and 1100c may include a synchronization enable signal. For example, if the camera module 1100b is the main camera and the camera modules 1100a and 1100c are slave cameras, the camera module controller 1216 may transmit a synchronization enable signal to the camera module 1100b. The camera module 1100b may receive the synchronization enable signal, generate a synchronization signal based on the synchronization enable signal, and provide the synchronization signal to the camera modules 1100a and 1100c via the synchronization signal line SSL. The camera modules 1100a, 1100b, and 1100c may transmit image data to the application processor 1200 synchronously with the synchronization signal.

[0109] In some embodiments, the control signals provided by the camera module controller 1216 to the camera modules 1100a, 1100b, and 1100c may include mode information corresponding to the mode signal "Mode Signal" provided to the camera modules 1100a, 1100b, and 1100c. The camera modules 1100a, 1100b, and 1100c may operate in one of a first operation mode and a second operation mode associated with a sensing speed according to the mode information.

[0110] In the first operation mode, the camera modules 1100a, 1100b, and 1100c may generate an image signal at a first speed (i.e., generate an image signal at a first frame rate), encode the image signal at a second speed higher than the first speed (i.e., encode the image signal at a second frame rate higher than the first frame rate), and transmit the encoded image signal to the application processor 1200. Here, the second speed may be less than 30 times the first speed.

[0111] The application processor 1200 may store the received image signal (i.e., the encoded image signal) in the internal memory 1230 or the external memory 1400, read and decode the encoded image signal from the internal memory 1230 or the external memory 1400, and display the image data generated based on the decoded image signal. For example, the decoding of the encoded image signal may be performed by the sub-processors 1212a, 1212b, and 1212c of the image processor 1210, and image processing may be performed on the decoded image signal.

[0112] In the second operation mode, the camera modules 1100a, 1100b, and 1100c can generate an image signal at a third speed lower than the first speed (i.e., generate an image signal with a third frame rate lower than the first frame rate), and can transmit the image signal to the application processor 1200. The image signal transmitted to the application processor 1200 can be an unencoded signal. The application processor 1200 can perform image processing on the image signals received from the camera modules 1100a, 1100b, and 1100c, or can store the received image signals in the internal memory 1230 or the external memory 1400.

[0113] The PMIC 1300 can supply power, such as a power supply voltage, to the camera modules 1100a, 1100b, and 1100c. For example, under the control of the application processor 1200, the PMIC 1300 can supply a first power to the camera module 1100a via the power signal line PSLa, a second power to the camera module 1100b via the power signal line PSLb, and a third power to the camera module 1100c via the power signal line PSLc.

[0114] The PMIC 1300 can generate the power corresponding to each of the camera modules 1100a, 1100b, and 1100c, and control the level of the power in response to the power control signal PCON from the application processor 1200. The power control signal PCON can include power control signals for different operation modes of the camera modules 1100a, 1100b, and 1100c. For example, the operation modes of the camera modules 1100a, 1100b, and 1100c can include a low power mode, in which case the power control signal PCON can include information indicating which camera module is to operate in the low power mode and information indicating the power level to be set. The power levels provided can be the same for all the camera modules 1100a, 1100b, and 1100c, or can vary depending on the camera module. In addition, the power levels provided can be changed dynamically.

[0115] Although the inventive concept has been shown and described with reference to several illustrated embodiments, those skilled in the art will understand that various changes can be made in form and detail without departing from the spirit and scope of the inventive concept as defined by the appended claims and their equivalents.

[0116] 1: Image sensing device 100: Analog-to-digital converter (ADC) / ADC block 150: Digital Signal Processor (DSP) 151: Image Signal Processor (ISP) 152: Camera control 153: Interface (I / F) 200, 700, 1142: Image sensor 210: Pixel array 220: Column driver 230: Correlated Double Sampling (CDS) block 260: Ramp generator 270: Timing generator 280: Control register block 290: Buffer 300: Display unit 400: Object 500: Lens 610, 620: Analog logic 611, 621: ADC controller 612: Bit shifter 622: Zero padding unit 650: Digital logic 651: Random Bit Generator (RNG) 1000: Electronic device 1100: Camera module group 1100a, 1100b, 1100c: Camera module 1105: Prism 1106: Central axis 1107: Reflective surface 1110: Optical Path Folding Element (OPFE) 1130: Actuator 1140: Image sensing device 1144: Control logic 1146: Memory 1147: Calibration data 1150: Storage unit 1200: Application processor 1210: Image processor 1212a, 1212b, 1212c: Sub-image processor 1214: Image generator 1216: Camera module controller 1220: Memory controller 1230: Internal Memory 1300: Power Management Integrated Circuit (PMIC) 1400: External Memory A: Motion B: (Blue) Pixel / Motion C L2, C Ln: Code Level CSLa, CSLb, CSLc: Control Signal Lines G: Gain Adjustment Signal G L1~G Ln: Gain Level Gb, Gr: (Green) Pixel Generating Information: Image Generation Information I: User Input Signal I 2C: Inter-Integrated Circuit ISLa, ISLb, ISLc: Image Signal Lines K: Kernel Unit L: Incident Light / Light Layer1: First Layer Layer2: Second Layer LOGIC: Logic Circuit M1_1, M1_2, M1_3, M1_4: Transmission Transistors M2: Reset Transistor / Transistor M3: Source Follower Transistor / Transistor M4: Selection Transistor / Transistor Mode Signal: Mode Signal P1~P4: Pixel PA1: First Pad Area PAD1: Upper Pad PAD21, PAD22, PAD23: Pads PCON: Power Control Signal PD1, PD2, PD3, PD4: Photodiodes PSLa, PSLb, PSLc: Power Signal Lines PX: Pixel R: (Red) Pixel SA: Sensing Area S_RESET, S_SEL, S_TG1, S_TG2, S_TG3, S_TG4: Control Signals SSL: Synchronization Signal Line X: First direction / first arbitrary direction Y: Second direction Z: Third direction

Claims

1. An image sensor, comprising: A pixel array arranged in a Bayer pattern and comprising pixels that generate charges according to incident light; And analog logic, configured to use analog-to-digital conversion to convert an analog signal output from at least one of the pixels into a first digital code, and to generate a second digital code by eliminating one or more low-order bits of the first digital code in response to a control signal, wherein the second digital code is represented using fewer bits than the first digital code.

2. The image sensor as claimed in claim 1, wherein the analog logic comprises: An analog-to-digital converter (ADC) controller is configured to generate the control signal in response to a gain adjustment signal and a user input signal, and an analog-to-digital converter (ADC) is configured to adjust the analog gain in response to the control signal and amplify the analog signal using the analog gain.

3. The image sensor as claimed in claim 2, wherein the analog logic further comprises: A bit shifter is configured to shift out one or more low-order bits according to the control signal and output the remaining number of high-order bits as a second bit code, and bit logic receives the second bit code and includes a random bit generator configured to generate random bits, wherein the first bit code includes the one or more low-order bits and the remaining high-order bits, the bit logic combines the second bit code and the number of random bits to generate a third bit code, and the third bit code is represented using the same number of bits as the first bit code.

4. The image sensor as claimed in claim 2, wherein the analog logic further comprises: A zero-filling unit is configured to remove adjusted low-order bits from the first digital code in response to the control signal to generate the second digital code, and to fill the second digital code with a number of zeros equal to the number of the low-order bits to generate the third digital code.

5. A method of operating an image sensing device, the method comprising: Generates an analog signal corresponding to the charge generated by at least one pixel in response to incident light; The analog signal from the output line of the at least one pixel is converted to generate a first digital code; And in response to a first control signal, one or more low-order bits of the first digital code are eliminated to generate a second digital code, wherein the second digital code is represented using fewer bits than the first digital code.

6. The method of claim 5, wherein the analog signal is amplified with a gain adjusted according to a second control signal.

7. An image sensing device, comprising: A lens for receiving incident light; a pixel array comprising pixels configured to generate charges in response to the incident light and output an analog signal corresponding to the charges; a column decoder configured to enable at least one of the pixels; and first logic configured to receive the analog signal via the output line of the pixel enabled by the column decoder and convert the analog signal into an N-bit first digital code. A second logic is configured to process an N-bit second digital code to output image data; and a control circuit is configured to control the operation of the first logic and the second logic, wherein the second digital code includes the remaining (Nk) high-order bits other than the k low-order bits of the first digital code, N and k are natural numbers, and N is greater than k, wherein the first logic receives a first control signal from the control circuit and amplifies the analog signal with an analog gain corresponding to the first control signal to convert the analog signal into the first digital code, wherein the first logic includes a bit shifter configured to set the value of k according to the second control signal received from the control circuit and output bits other than the k low-order bits from the first digital code.

8. The image sensing apparatus of claim 7, wherein the second logic further includes a random bit generator configured to generate k random bits, and the second logic generates the second bit code, combines the bits output by the bit shifter and outputs them as higher-order bits and outputs the k random bits as lower-order bits.

Images