Imaging device and operating method thereof

Abstract

An imaging device according to the present invention comprises an image sensing unit that receives image signals from pixels, converts the received image signals into digital data, and outputs the converted image data, and a digital processing unit that processes the image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, and when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock.

Description

Technology Field

[0001] The present invention relates to an image device and a method of operating the same. Background Technology

[0002] Generally, image sensors capture two-dimensional or three-dimensional images of an object. Image sensors generate images of an object using photoelectric conversion elements that respond to the intensity of light reflected from the object. Recently, image sensors using CMOS (complementary metal-oxide semiconductor) have become widely used. Conventional image sensors cause pixel noise due to the influence of power fluctuations. The problem to be solved

[0003] The objective of the present invention is to provide an imaging device that is less affected by power fluctuations and a method of operating the same. means of solving the problem

[0004] An imaging device according to an embodiment of the present invention comprises: an image sensing unit that receives image signals from pixels, converts the received image signals into digital data, and outputs the converted image data; and a digital processing unit that processes the image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, and when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock.

[0005] A method of operating an imaging device according to an embodiment of the present invention may include: a step of converting an image signal into image data by performing a count operation until the image signal and the ramp voltage are equal; and a step of reducing digital power consumption by using a clock dispersion method when performing the count operation.

[0006] An image device according to an embodiment of the present invention comprises: a pixel array having a plurality of pixels arranged in a plurality of row lines and a plurality of column lines; a row driver selecting one of the plurality of row lines; an analog-to-digital conversion circuit converting analog signals output from the pixel array into digital data; a digital clock generator generating a transmission clock for transmitting the digital data to an image signal processor; and a timing controller controlling the timing of the pixel array, the row driver, the analog-to-digital conversion circuit, and the digital clock generator, wherein the analog-to-digital conversion circuit comprises a comparison circuit having comparators that compare pixel signals output from the pixel array with a ramp voltage; and a counter circuit having counters that count the outputs of each of the comparators, and wherein the digital clock generator distributes the transmission clock in response to a horizontal synchronization signal or a counter activation signal received from the timing controller. Effects of the invention

[0007] An image sensor according to an embodiment of the present invention, an image device having the same, and a method of operating the same can minimize changes in digital power by dispersing the digital clock when performing analog operations in the image sensor. Brief explanation of the drawing

[0008] The drawings attached below are intended to aid in understanding the embodiments thereof, and embodiments are provided along with a detailed description. FIG. 1 is a drawing illustrating an imaging device for explaining the concept of the present invention. FIG. 2 is a drawing exemplarily showing an image sensor according to an embodiment of the present invention. FIG. 3 is a diagram exemplarily showing the timing of a digital clock of a blocking clock method according to an embodiment of the present invention. FIGS. 4a and 4b are drawings exemplarily showing a digital processing unit implemented with a plurality of serially connected ISPs according to an embodiment of the present invention and the timing of its digital clock. FIG. 5 is a diagram exemplarily showing the timing of a digital clock of a divided clock method according to another embodiment of the present invention. FIGS. 6a, 6b, 6c, and 6d are exemplary drawings showing the timing of a digital clock applying digital clock dispersion to a portion of an ADC according to an embodiment of the present invention. FIG. 7 is a diagram exemplarily showing the digital current consumption of an imaging device according to an embodiment of the present invention. FIG. 8 is a diagram exemplarily showing the digital current consumption of an imaging device according to another embodiment of the present invention. FIG. 9a is a diagram exemplifying an imaging device according to another embodiment of the present invention, and FIG. 9b is a diagram exemplifying a digital processing unit implemented with parallel-connected ISPs according to another embodiment of the present invention. FIG. 10 is a flowchart showing the operation method of an imaging device according to an embodiment of the present invention. FIG. 11 is a drawing showing an exemplary electronic device having a multi-camera module. Figure 12 is a drawing showing the detailed configuration of the camera module of Figure 11. Specific details for implementing the invention

[0009] Below, the contents of the present invention will be described clearly and in detail using the drawings so that a person skilled in the art can easily implement the invention.

[0010] FIG. 1 is a drawing illustrating an imaging device (10) for explaining the concept of the present invention. Referring to FIG. 1, the imaging device (10) may include an image sensing unit (100) and a digital processing unit (200).

[0011] The image sensing unit (100) may be implemented to receive an image from a pixel, convert the received image into a digital value, and output the converted image data (IDATA) to a digital processing unit (200). Additionally, the image sensing unit (100) may include a timing controller (170) that generates timing for controlling internal components of the image sensing unit (100).

[0012] The digital processing unit (200) may be implemented to receive image data (IDATA) output from the image sensing unit (100), process the image data (IDATA) to be suitable for the human eye, and output the processed image data (PDATA) to an external device (e.g., a display device). Additionally, the digital processing unit (200) may include a digital clock generator (201) and at least one image signal processor (ISP(s), 210).

[0013] A digital clock generator (201) can be implemented to generate an optimal digital clock (DCLK) according to the analog operation of the image sensing unit (100) in order to control fluctuations in digital power that affect the analog quality of the image sensing unit (100). For example, the digital clock generator (201) can generate a digital clock (DCLK) in a predetermined clock timing sequence in response to a counter enable signal (CNT_EN) or a horizontal synchronization signal (H_SYN) for performing an Analog to Digital Conversion (ADC) operation of the image sensing unit (100). This predetermined sequence of digital clocks (DCLK) can control digital power fluctuations by bypassing / blocking / dividing the digital clock (DCLK) during the ADC operation interval. By controlling the output of this digital clock (DCLK), pixel noise of the image sensing unit (100) can be reduced. That is, by controlling the digital clock (DCLK) of the digital domain of the imaging device (10), the influence of the analog domain can be minimized.

[0014] Additionally, while the image sensing unit (100) detects and outputs an image, the digital clock generator (201) can generate a digital clock (DCLK) having multiple clock distribution intervals that are gated or divided. The start and end points of each of the multiple clock distribution intervals can be determined by user settings. For example, the digital clock generator (201) can output the digital clock (DCLK) in the reverse direction of the counter enable signal (CNT_EN) used for the ADC operation of the image sensing unit (100). In this case, the clock cut-off interval, clock division interval, or clock division ratio between all start and end points can be determined by user settings.

[0015] In an embodiment, the digital clock generator (201) can output a digital clock (DCLK) based on a reference point of a horizontal interval (such as a reference point for setting the counter activation area of ​​the ADC operation) in response to a horizontal synchronization signal (H_SYN). In this case, there may be multiple clock distribution intervals for the distribution control of the digital clock (DCLK). The positions (cycle counts) of the start and end points of each of the multiple clock distribution intervals can be determined by user settings. For example, each of the multiple clock distribution intervals can start at a time earlier than the counter activation signal (CNT_EN) by referring to the setting area of ​​the counter activation signal (CNT_EN) of the ADC operation, and the end point can also be set before the counter activation signal (CNT_EN) is cleared. In this case, the clock blocking and clock splitting ratios for the respective start and end point areas of the multiple clock distribution intervals can be determined by user settings.

[0016] At least one image signal processor (210) can be implemented to process image data (IDATA) received from an image sensing unit (100) in synchronization with a digital clock (DCLK) and to output processed data (PDATA) to the outside. In an embodiment, the image signal processor (210) can perform depth map generation, 3D modeling, panorama generation, feature point extraction, image synthesis, or image compensation (e.g., noise reduction, resolution adjustment, brightness adjustment, blurring, sharpening, or softing) on ​​image data (IDATA). A typical imaging device causes noise components due to power fluctuations across the entire chip when converting the voltage value of a pixel signal into a digital value in the ADC section. On the other hand, the imaging device (10) of the present invention can minimize the influence of digital power fluctuations by scattering the digital clock (DCLK) of the digital processing unit (200) in the ADC section of the image sensing unit (100) using a gated clock scheme or a divided clock scheme.

[0017] FIG. 2 is a diagram illustrating an image sensing unit (100) according to an embodiment of the present invention. Referring to FIG. 2, the image sensing unit (100) may include a pixel array (110), a row driver (120), an analog-to-digital conversion circuit (130), a ramp voltage generator (160), a timing controller (170), and a buffer (180).

[0018] A pixel array (110) may include a plurality of pixels arranged in a matrix form, each connected to a plurality of row lines and a plurality of column lines (CL). Each of the plurality of pixels may include a light-sensing element. For example, the light-sensing element may include a photodiode, a phototransistor, a photogate, or a pinned photodiode. Each of the plurality of pixels may include at least one light-sensing element. In an embodiment, each of the plurality of pixels may include a plurality of light-sensing elements. Each of the plurality of light-sensing elements may be stacked together.

[0019] Each of the plurality of pixels can detect light using a light-sensing element and convert it into an electrical signal, which is a pixel signal. Each of the plurality of pixels can detect light in a specific spectral region. For example, the plurality of pixels may include a red pixel that converts light in the red spectral region into an electrical signal, a green pixel that converts light in the green spectral region into an electrical signal, and a blue pixel that converts light in the blue spectral region into an electrical signal. A color filter for transmitting light in a specific spectral region may be placed above each of the plurality of pixels.

[0020] The row driver (120) may be implemented to drive the pixel array (110) in rows. The row driver (120) may decode a row control signal (e.g., an address signal) generated by the timing controller (170) and select at least one row line among the row lines constituting the pixel array (110) in response to the decoded row control signal. For example, the row driver (120) may generate a row selection signal. Then, the pixel array (110) outputs a pixel signal from the row selected by the row selection signal provided by the row driver (120). The pixel signal may include a reset signal and an image signal.

[0021] The analog-to-digital conversion circuit (130) may be implemented to convert an analog pixel signal input from a pixel array (110) into digital data in response to an ADC enable signal (ADC_EN). The analog-to-digital conversion circuit (130) may include a comparison circuit (140, CDB) and a counter circuit (150, DBS).

[0022] A comparison circuit (140) may be implemented to compare a pixel signal output from a unit pixel connected to any one of the column lines (CL) constituting the pixel array (110) with a ramp voltage (RAMP). The comparison circuit (140) may include a plurality of comparators (141) provided corresponding to each column, and each comparator (141) may be connected to the pixel array (110) and the ramp voltage generator (160).

[0023] A comparator (141, CMP) can be implemented to receive a pixel signal and a ramp voltage (RAMP) generated from a ramp voltage generator (160) as inputs, compare them, and output a comparison result signal to an output terminal. Additionally, the comparator (141) can generate a comparison result signal to which a correlated double sampling (CDS) technique is applied. Pixel signals output from multiple pixels may have deviations caused by the unique characteristics of each pixel (e.g., fixed pattern noise (FPN), etc.) or by differences in the characteristics of the logic for outputting a pixel signal from a pixel (PX). The correlated double sampling technique is a method that calculates a reset component (or reset signal) and an image component (or image signal) for each of the pixel signals to compensate for these deviations between pixel signals, and extracts the difference as a valid signal component. The comparator (141) can output a comparison result signal to which the correlated double sampling technique is applied.

[0024] Additionally, the comparator (141) may be implemented as a two-stage amplifier. For example, the comparator (141) may include a first amplifier that compares a pixel signal with a ramp voltage and a second amplifier that amplifies and outputs the output of the first amplifier. In an embodiment, the first amplifier may operate based on a smaller amount of bias current than in the comparison operation stage during the auto-zero stage. Accordingly, the input range may be increased while noise is reduced. In an embodiment, the second amplifier adaptively controls the current sources generating the bias current according to the operation stage and may generate a minimum bias current before and after the decision. Accordingly, power fluctuations due to the operation of the second amplifier may be prevented. In an embodiment, the first amplifier may include a limiting circuit connecting the output terminal and the common node. Here, the limiting circuit prevents the voltage level of the common node from falling below the minimum value at which the first amplifier can operate normally and compensates for voltage fluctuations occurring at the output node.

[0025] Additionally, the comparison circuit (140) can be implemented to output a decision signal (e.g., output signal of a comparator) at different points in time depending on the column line group.

[0026] The counter circuit (150) may include a plurality of counters. Each of the plurality of counters (151, CNT) may be connected to the output terminal of the comparator (141) and implemented to count based on the output of each comparator (141). The counter control signal (CTCS) may include a counter enable signal (CNT_EN), a counter clock signal, a counter reset signal that controls the reset operation of the plurality of counters (151), and an inversion signal that inverts the internal bit of each of the plurality of counters. The counter circuit (150) may output digital data by counting a comparison result signal according to the counter clock signal. In an embodiment, each of the plurality of counters may be enabled by the counter enable signal (CNT_EN, see FIG. 1).

[0027] The counter (151, CNT) may include an up / down counter or a bit-wise counter. In this case, the bit-wise counter can perform operations similar to those of an up / down counter. For example, the bit-wise counter can perform the function of performing only an up count and the function of inverting all bits inside the counter to make them one's complement when a specific signal is received. The bit-wise counter can perform a reset count and then invert it to convert it into one's complement, i.e., a negative value.

[0028] A ramp voltage generator (160) may be implemented to generate a ramp voltage (or an ADC reference voltage). The ramp voltage generator (160) may operate based on a ramp control signal (CTRP) provided by a timing controller (170). The ramp control signal (CTRP) may include a ramp enable signal, a mode signal, etc. When the ramp enable signal is activated, the ramp voltage generator (160) may generate a ramp voltage (RAMP) having a slope set based on the mode signal.

[0029] The timing controller (170) can be implemented to control the operation or timing of the low driver (120), the analog-to-digital conversion circuit (130), and the ramp voltage generator (160) by outputting a control signal or a clock signal to each of the low driver (120), the analog-to-digital conversion circuit (130), and the ramp voltage generator (160).

[0030] In an embodiment, the timing controller (170) can transmit information related to the ADC operation interval of the image sensing unit (100) to the digital processing unit (200, see FIG. 1). For example, the timing controller (170) can transmit an ADC enable signal (ADC_EN), a horizontal synchronization signal (H_SYN), or a counter enable signal (CNT_EN) to the digital processing unit (200).

[0031] Additionally, the timing controller (170) can generate switching control signals provided to the comparison circuit (140) to make the decision speeds different depending on the column line group.

[0032] The buffer (180) may be implemented to temporarily store digital data output from the analog-to-digital conversion circuit (130), amplify it, and output it. The buffer (180) may include a column memory block (181, MEM) and a sense amplifier (182, SA).

[0033] The column memory block (181, MEM) may include multiple memories. Each of the multiple memories (183) may temporarily store digital data output from each of the multiple counters (151) and then output it to the sense amplifier (182).

[0034] A sense amplifier (182, SA) can be implemented to detect and amplify digital data output from multiple memories. The sense amplifier (182) can output the amplified digital data as image data (IDATA).

[0035] Below, distributed control of a digital clock (DCLK) according to an embodiment of the present invention will be described in detail.

[0036] FIG. 3 is a diagram exemplarily showing the timing of a digital clock (DCLK) of a blocking clock method according to an embodiment of the present invention. As shown in FIG. 3, an ADC enable signal (ADC_EN) may have a high level in a first ADC interval (A1) and a second ADC interval (A2). Here, the first ADC interval (A1) may be a interval for converting the reset voltage, and the second ADC interval (A2) may be a interval for converting the pixel voltage. In an embodiment, the second ADC interval (A2) may be longer than the first ADC interval (A1).

[0037] The digital clock (DCLK) can be gated before and after each of the first and second ADC intervals (A, B). For example, the digital clock (DCLK) may include a first clock gated interval (G1) and a second clock gated interval (G2). Here, the first clock gated interval (G1) may include a first ADC interval (A1), and the second clock gated interval (G2) may include a second ADC interval (A2). That is, the digital clock (DCLK) can be output normally in intervals excluding the first and second clock gated intervals (G1, G2).

[0038] In an embodiment, the digital clock (DCLK) can be output according to a predetermined timing in response to an ADC enable signal (ADC_EN). For example, the digital clock (DCLK) can set a cutoff interval in response to a counter enable signal for ADC operation.

[0039] As illustrated in FIG. 3, the image data activation signal (IDATA_EN) may have a high level in a predetermined interval to indicate that the image data (IDATA) received from the image signal processor (210) of the digital processing unit (200) is valid. In an embodiment, the image data activation signal (IDATA_EN) may be generated from a timing controller (170, see FIG. 1). In another embodiment, the image data activation signal (IDATA_EN) may be generated internally within the digital processing unit (200, see FIG. 1) in response to an ADC activation signal (ADC_EN). The blank interval illustrated in FIG. 3 is a interval in which no signal reading is performed by the image sensing unit (100, see FIG. 1). The image data activation signal (IDATA_EN) may have a first stall interval (S1) and a second stall interval (S2). Here, the first stall interval (S1) corresponds to the first clock cutoff interval (G1), and the second stall interval (S2) corresponds to the second clock cutoff interval (G2). In the embodiment, in the first and second stall intervals (S1, S2), there is no output of converted image data (IDATA).

[0040] In an embodiment, the image data activation signal (IDATA_EN) is at a high level, and in the interval excluding the first and second stall intervals (S1, S2), the processed data (PDATA, see FIG. 1) can be output externally.

[0041] The control method of the digital clock (DCLK) according to an embodiment of the present invention can minimize digital influence in the ADC operation by blocking the clock in the interval corresponding to the ADC operation.

[0042] Meanwhile, the converted digital data (IDATA) of the image sensing unit (100) can be output to a plurality of serially connected ISPs (Image Signal Processors).

[0043] FIGS. 4a and 4b are exemplary drawings illustrating a digital processing unit (200) implemented with a plurality of serially connected ISPs according to an embodiment of the present invention and the timing of its digital clock (DCLK). Referring to FIG. 4b, image data (IDATA) can be digitally processed by four serially connected ISPs (ISP1 to ISP4; 211 to 124) in synchronization with the digital clock (DCLK).

[0044] In FIG. 4b, the image data activation signal (IDATA_EN) corresponding to each of the ISPs (211 to 214) may have common first and second stall intervals (S1, S2). In an embodiment, each of the ISPs (211 to 214) may process the received data with appropriate latencies (LT1 to LT3) in intervals excluding the first and second stall intervals (S1, S2). For example, the second ISP (212) may receive data processed by the first ISP (211) and perform a predetermined processing operation on the received data. In this case, the image data activation signal (IDATA_EN2) of the second ISP (212) has a high level after the first latency (LT1) of the image data activation signal (IDATA_EN1) of the first ISP (211).

[0045] Meanwhile, when the blank interval in the output section for the processed data (PDATA) is insufficient, a divided clock scheme may be applied.

[0046] FIG. 5 is a diagram exemplarily showing the timing of a digital clock (DCLK) of a divided clock method according to another embodiment of the present invention. Referring to FIG. 5, the timing of the digital clock (DCLK) has a first clock divided section (D1) corresponding to a first ADC section (A1) and a second clock divided section (D2) corresponding to a second ADC section (A2), compared to that shown in FIG. 3. That is, the digital clock (DCLK) can be divided in the first ADC section (A1) and the second ADC section (A2).

[0047] In the embodiment, when the clock division ratio is set, the digital clock generator (201) can pass n-1 clock gates and one clock bypass from the start of the clock division interval. Additionally, the clock can be bypassed from the end point of the divided clock.

[0048] The control method of the digital clock (DCLK) according to an embodiment of the present invention can minimize digital influence in the ADC operation while securing a blank interval by dividing the clock in the interval corresponding to the ADC operation.

[0049] Meanwhile, the blocking clock method and the split clock method according to the embodiment of the present invention may be applied only to a part of the ADC section.

[0050] FIGS. 6a, FIGS. 6b, FIGS. 6c, and FIGS. 6d are exemplary drawings showing the timing of a digital clock (DCLK) that applies digital clock dispersion to a portion of an ADC according to an embodiment of the present invention.

[0051] Referring to FIG. 6a, there may be a third clock cutoff section (G3) corresponding to a part of the first ADC section (A1), and a fourth clock cutoff section (G4) corresponding to a part of the second ADC section (A2). For example, the section corresponding to the first 25% of the first ADC section (A1) may be included in the third clock cutoff section (G3), and the section corresponding to the first 25% of the second ADC section (A2) may be included in the fourth clock cutoff section (G4). Here, the section corresponding to the first 25% may be a dark area sensitive to image quality characteristics.

[0052] Referring to FIG. 6b, there may be a third clock division section (D3) corresponding to a part of the first ADC section (A1), and a fourth clock division section (D4) corresponding to a part of the second ADC section (A2). For example, the section corresponding to the first 25% of the first ADC section (A1) may be included in the third clock division section (D3), and the section corresponding to the first 25% of the second ADC section (A2) may be included in the fourth clock division section (D4).

[0053] Referring to FIG. 6c, there may be a fifth clock cutoff section (G5) and a fifth clock division section (D5) corresponding to the first ADC section (A1), and a sixth clock cutoff section (G6) and a sixth clock division section (D6) corresponding to the second ADC section (A2). For example, the section corresponding to the first 25% of the first ADC section (A1) may be the fifth clock cutoff section (G5), and the section corresponding to the remaining 75% may be the fifth clock division section (D5). Similarly, the section corresponding to the first 25% of the second ADC section (A2) may be the sixth clock cutoff section (G6), and the section corresponding to the remaining 75% may be the sixth clock division section (D6).

[0054] Referring to FIG. 6d, there may be two clock division sections (G7, D7) corresponding to the first ADC section (A1), and two clock division sections (G8, D8) corresponding to the second ADC section (A2). For example, the section corresponding to the first 25% of the first ADC section (A1) may be a clock division section (G7) according to the first clock division ratio, and the section corresponding to the remaining 75% may be a clock division section (D7) according to the second clock division ratio. Here, the second clock division ratio may be greater than the first clock division ratio. However, it should be understood that the present invention is not limited thereto.

[0055] Similarly, the section corresponding to the first 25% of the second ADC section (A1) may be a clock division section (G8) according to the first clock division ratio, and the section corresponding to the remaining 75% may be a clock division section (D8) according to the second clock division ratio.

[0056] Meanwhile, it should be understood that the ratio of the clock blocking interval or clock splitting interval of the present invention is not limited thereto.

[0057] FIG. 7 is a diagram illustrating the digital current consumption of an imaging device (10) according to an embodiment of the present invention. Referring to FIG. 7, during a horizontal interval, an analog counter clock can be activated in a first ramp interval (R1) and a second ramp interval (R2) according to a change in the ADC reference voltage. Here, a first count operation corresponding to a reset voltage is performed in the first ramp interval (R1), and a second count operation corresponding to a pixel voltage is performed in the second ramp interval (R2).

[0058] The digital clock (DCLK) can be output in response to the horizontal synchronization signal (H_SYN). Additionally, the digital clock (DCLK) may not be output in response to the counter clock. For example, the digital clock (DCLK) may be blocked in intervals corresponding to the first and second ramp intervals (R1, R2). In this case, the digital current consumption in the digital processing unit (200) can be significantly reduced in the ramp intervals (R1, R2). Thus, the count operation for ADC operation can be less affected by digital power.

[0059] FIG. 8 is a diagram illustrating the digital current consumption of an imaging device (10) according to another embodiment of the present invention. Referring to FIG. 8, the digital clock (DCLK) may be output in response to a horizontal synchronization signal (H_SYN) and may not be output in response to a counter clock. For example, the digital clock (DCLK) may be divided into intervals corresponding to first and second ramp intervals (R1, R2). At this time, the digital current consumption consumed by the digital processing unit (200) may be reduced in the ramp intervals (R1, R2).

[0060] Meanwhile, in FIGS. 1 to 8, the imaging device (10) illustrates distributed control of a digital clock (DCLK) used in a digital processing unit (200). However, the present invention is not limited thereto. The present invention is applicable to distributed control of a digital clock used internally in an image sensing unit.

[0061] FIG. 9a is a drawing illustrating an imaging device (20) according to another embodiment of the present invention. Referring to FIG. 9, the imaging device (20) may include an image sensing unit (100a) and a digital processing unit (300).

[0062] The image sensing unit (100a) may include a digital clock generator (101) and a timing controller (170). The digital clock generator (101) may be implemented to generate a transmission clock (TCLK). The image sensing unit (100a) may output image data (IDATA) to a digital processing unit (300) in synchronization with the transmission clock (TCLK).

[0063] The digital clock generator (101) can generate a clock-distributed (blocked / divided) forward clock (TCLK) when the image sensor (100a) is in analog operation (e.g., ADC operation). Here, the clock-distributed forward clock (TCLK) may be similar to the blocking clock or divided clock described in FIGS. 1 through 8.

[0064] The digital processing unit (300) may include at least one image signal processor (ISP(s), 310). The image signal processor (310) may perform processing operations on received image data (IDATA) in synchronization with the transmission clock (TCLK). Meanwhile, a plurality of ISPs of the digital processing unit according to an embodiment of the present invention may be connected in parallel.

[0065] FIG. 9b is a diagram illustrating an exemplary digital processing unit (400) according to another embodiment of the present invention. Referring to FIG. 9b, the digital processing unit (400) may include a digital clock generator (401), a line buffer (402), a plurality of ISPs (411 to 414), a merging module (420), and a storage module (430). The digital clock generator (401) may be implemented to distribute a digital clock (DCLK) in the ADC section of an image sensing unit, as described in FIG. 1 to 8.

[0066] A line buffer (402) can be implemented to store image data acquired from an image sensing unit in line units. At least a portion of the data stored in the line buffer (402) may be associated data of a plurality of photodiodes arranged according to a two-dimensional array of the image sensing unit. The plurality of photodiodes may be arranged within the image sensing unit based on pixels arranged two-dimensionally along two axes perpendicular to each other. A line is a group of pixels arranged within the image sensing unit and may correspond to a group of pixels arranged along one of the two axes. In an embodiment, the data stored in the line buffer (401) may be divided into multiple parts. The divided multiple data may be transmitted to a corresponding ISP through the corresponding buffer.

[0067] Each of the plurality of ISPs (411 to 414) may be implemented to perform digital processing operations on data stored in a buffer in synchronization with the digital clock (DCLK). These digital processing operations may include color interpolation (CI), gamma correction, edge enhancement (EE), and noise reduction (NR). A merging module (420) may be implemented to merge some or all of the data output from the plurality of ISPs (411 to 414). A storage module (430) may be implemented to store the merged data from the merging module (420). The stored data may be frame data corresponding to image data acquired from an image sensing unit. An Application Processor (AP) may process this frame data based on the running application. For example, the AP may generate an image file or video file in a specified format (e.g., JPEG format or MPEG format) based on the identified frame data.

[0068] FIG. 10 is a flowchart showing a method of operation of an imaging device according to an embodiment of the present invention. Referring to FIGS. 1 to 10, the method of operation of the imaging device (10) may proceed as follows.

[0069] The image sensing unit (100) can perform an analog-to-digital conversion operation (S110). While performing the analog-to-digital conversion operation, the digital processing unit (200) or the image sensing unit (100) can reduce digital power consumption by using a clock distribution method (S120). Here, the clock distribution method can be implemented as a blocking clock method (clock blocking method) or a divided clock method (clock division method).

[0070] In the embodiment, the clock distribution method may block the clock or split the clock in all or part of the ADC section. For example, the clock may be blocked or a split clock output may be output in a part of the ADC section, for example, about the first 25% section, or in a dark area sensitive to image quality characteristics. Additionally, the clock may be blocked in the first 25% section of the ADC section, and a split clock may be output in the last 75% section.

[0071] In an embodiment, the ADC interval may be set into a plurality of clock distribution intervals. In an embodiment, the set clock distribution intervals may overlap each other. In an embodiment, a priority may be assigned to each of the intervals by the user. When they overlap, a clock corresponding to the priority may be output. In an embodiment, the start point of the later interval may be the same as or precede the end point of the earlier interval in terms of time.

[0072] FIG. 11 is a drawing illustrating an exemplary electronic device having a multi-camera module. Referring to FIG. 11, the electronic device (1000) may include a camera module group (1100), an application processor (1200), a PMIC (1300), and an external memory (1400).

[0073] The camera module group (1100) may include a plurality of camera modules (1100a, 1100b, 1100c). Although an embodiment in which three camera modules (1100a, 1100b, 1100c) are arranged is illustrated in the drawings, the embodiments are not limited thereto. In an embodiment, the camera module group (1100) may be modified to include only two camera modules. Additionally, in an embodiment, the camera module group (1100) may be modified to include n camera modules (n is a natural number greater than or equal to 4).

[0074] FIG. 12 is a drawing showing the detailed configuration of the camera module (1100b) of FIG. 11. The following description may be applied in the same way to other camera modules (1100a, 1100b) according to the embodiment. Referring to FIG. 12, the camera module (1100b) may include a prism (1105), an optical path folding element (OPFE) (1110), an actuator (1130), an imaging device (1140), and a storage unit (1150).

[0075] The prism (1105) can modify the path of light (L) incident from the outside by including a reflective surface (1107) of a light-reflecting material.

[0076] In an embodiment, the prism (1105) can change the path of light (L) incident in a first direction (X) to a second direction (Y) perpendicular to the first direction (X). Additionally, the prism (1105) can change the path of light (L) incident in the first direction (X) to a second direction (Y) perpendicular to the first direction (X) by rotating the reflective surface (1107) of the light-reflecting material in direction A around the central axis (1106) or by rotating the central axis (1106) in direction B. At this time, the OPFE (1110) can also move in a third direction (Z) perpendicular to the first direction (X) and the second direction (Y).

[0077] In the embodiment, as illustrated, the maximum rotation angle of the prism (1105) in the A direction may be 15 degrees or less in the plus (+) A direction and greater than 15 degrees in the minus (-) A direction, but the embodiments are not limited thereto.

[0078] In the embodiment, the prism (1105) can move in the plus (+) or minus (-) B direction by about 20 degrees, or between 10 and 20 degrees, or between 15 and 20 degrees, where the angle of movement can be moved by the same angle in the plus (+) or minus (-) B direction, or by almost similar angles within a range of about 1 degree.

[0079] In an embodiment, the prism (1105) can move the reflective surface (1106) of the light-reflecting material in a third direction (e.g., Z direction) parallel to the extension direction of the central axis (1106).

[0080] OPFE (1110) may include, for example, groups of m (where m is a natural number) optical lenses. The m lenses can be moved in a second direction (Y) to change the optical zoom ratio of the camera module (1100b). For example, when the basic optical zoom ratio of the camera module (1100b) is Z, moving the m optical lenses included in the OPFE (1110) may change the optical zoom ratio of the camera module (1100b) to 3Z, 5Z, or an optical zoom ratio of 5Z or more.

[0081] The actuator (1130) can move the OPFE (1110) or the optical lens (hereinafter referred to as the optical lens) to a specific position. For example, the actuator (1130) can adjust the position of the optical lens so that the image sensor (1142) is positioned at the focal length of the optical lens for accurate sensing.

[0082] The imaging device (1140) may include an image sensor (1142), control logic (1144), and memory (1146). The image sensor (1142) can sense an image of a sensing target using light (L) provided through an optical lens. The control logic (1144) can control the overall operation of the camera module (1100b). For example, the control logic (1144) can control the operation of the camera module (1100b) according to a control signal provided through a control signal line (CSLb).

[0083] The memory (1146) can store information necessary for the operation of the camera module (1100b), such as calibration data (1147). The calibration data (1147) may include information necessary for the camera module (1100b) to generate image data using light (L) provided from the outside. The calibration data (1147) may include, for example, information regarding the degree of rotation described above, information regarding the focal length, information regarding the optical axis, etc. If the camera module (1100b) is implemented in the form of a multi-state camera in which the focal length changes according to the position of the optical lens, the calibration data (1147) may include focal length values ​​for each position (or state) of the optical lens and information related to auto-focusing.

[0084] The storage unit (1150) can store image data sensed through the image sensor (1142). The storage unit (1150) may be placed outside the imaging device (1140) and may be implemented in a stacked form with the sensor chip constituting the imaging device (1140). In an embodiment, the storage unit (1150) may be implemented as an EEPROM (Electrically Erasable Programmable Read-Only Memory), but the embodiments are not limited thereto.

[0085] Referring to FIG. 11 and FIG. 12 together, in an embodiment, each of the plurality of camera modules (1100a, 1100b, 1100c) may include an actuator (1130). Accordingly, each of the plurality of camera modules (1100a, 1100b, 1100c) may include identical or different calibration data (1147) according to the operation of the actuator (1130) included therein.

[0086] In an embodiment, one of the plurality of camera modules (1100a, 1100b, 1100c) (e.g., 1100b) is a camera module in the form of a folded lens including the previously described prism (1105) and OPFE (1110), and the remaining camera modules (e.g., 1100a, 1100b) may be camera modules in the form of a vertical lens that do not include the prism (1105) and OPFE (1110), but the embodiments are not limited thereto.

[0087] In an embodiment, one of the plurality of camera modules (1100a, 1100b, 1100c) (e.g., 1100c) may be a vertical depth camera that extracts depth information using, for example, IR (Infrared Ray). In this case, the application processor (1200) may generate a 3D depth image by merging image data provided from this depth camera with image data provided from another camera module (e.g., 1100a or 1100b).

[0088] In an embodiment, at least two of the plurality of camera modules (1100a, 1100b, 1100c) may have different field of view angles. In this case, for example, the optical lenses of at least two of the plurality of camera modules (1100a, 1100b, 1100c) may be different from each other, but are not limited thereto. Additionally, in an embodiment, the field of view of each of the plurality of camera modules (1100a, 1100b, 1100c) may be different from each other. In this case, the optical lenses included in each of the plurality of camera modules (1100a, 1100b, 1100c) may also be different from each other, but are not limited thereto. In an embodiment, each of the plurality of camera modules (1100a, 1100b, 1100c) may be physically separated from one another. That is, instead of the plurality of camera modules (1100a, 1100b, 1100c) dividing and using the sensing area of ​​a single image sensor (1142), an independent image sensor (1142) may be placed inside each of the plurality of camera modules (1100a, 1100b, 1100c).

[0089] Referring again to FIG. 11, the application processor (1200) may include an image processing device (1210), a memory controller (1220), and an internal memory (1230). The application processor (1200) may be implemented separately from a plurality of camera modules (1100a, 1100b, 1100c). For example, the application processor (1200) and the plurality of camera modules (1100a, 1100b, 1100c) may be implemented separately from each other as separate semiconductor chips.

[0090] The image processing device (1210) may include a plurality of sub-image processors (1212a, 1212b, 1212c), an image generator (1214), and a camera module controller (1216). Additionally, the image processing device (1210) may include a plurality of sub-image processors (1212a, 1212b, 1212c) corresponding to the number of camera modules (1100a, 1100b, 1100c).

[0091] Image data generated from each camera module (1100a, 1100b, 1100c) can be provided to corresponding sub-image processors (1212a, 1212b, 1212c) via separate image signal lines (ISLa, ISLb, ISLc). For example, image data generated from camera module (1100a) can be provided to sub-image processor (1212a) via image signal line (ISLa), image data generated from camera module (1100b) can be provided to sub-image processor (1212b) via image signal line (ISLb), and image data generated from camera module (1100c) can be provided to sub-image processor (1212c) via image signal line (ISLc). Such image data transmission can be performed, for example, using a Camera Serial Interface (CSI) based on MIPI (Mobile Industry Processor Interface), but the embodiments are not limited thereto.

[0092] Meanwhile, in the embodiment, a single sub-image processor may be arranged to correspond to a plurality of camera modules. For example, the sub-image processor (1212a) and the sub-image processor (1212c) may not be implemented separately as illustrated, but rather integrated into a single sub-image processor, and image data provided from the camera module (1100a) and the camera module (1100c) may be selected through a selection element (e.g., a multiplexer) and then provided to the integrated sub-image processor.

[0093] Image data provided to each sub-image processor (1212a, 1212b, 1212c) may be provided to an image generator (1214). The image generator (1214) may generate an output image using image data provided from each sub-image processor (1212a, 1212b, 1212c) according to image generating information or a mode signal. Specifically, the image generator (1214) may generate an output image by merging at least some of the image data generated from camera modules (1100a, 1100b, 1100c) having different viewing angles according to image generating information or a mode signal. Additionally, the image generator (1214) may generate an output image by selecting any one of the image data generated from camera modules (1100a, 1100b, 1100c) having different viewing angles according to image generating information or a mode signal.

[0094] In the embodiment, the image generation information may include a zoom signal (zoom signal or zoom factor). Also, in the embodiment, the mode signal may be a signal based on a mode selected by a user, for example. When the image generation information is a zoom signal (zoom factor) and each camera module (1100a, 1100b, 1100c) has a different viewing angle (angle of view), the image generator (1214) may perform different operations depending on the type of zoom signal. For example, when the zoom signal is a first signal, the image data output from the camera module (1100a) and the image data output from the camera module (1100c) may be merged, and then an output image may be generated using the merged image signal and the image data output from the camera module (1100b) that was not used for merging. If the zoom signal is a second signal different from the first signal, the image generator (1214) may not perform such image data merging and may generate an output image by selecting any one of the image data output from each camera module (1100a, 1100b, 1100c). However, the embodiments are not limited thereto, and the method of processing image data can be modified as needed.

[0095] In an embodiment, the image generator (1214) receives multiple image data with different exposure times from at least one of a plurality of sub-image processors (1212a, 1212b, 1212c) and performs HDR (high dynamic range) processing on the multiple image data to generate merged image data with increased dynamic range. At least one of the plurality of sub-image processors (1212a, 1212b, 1212c) may be implemented to perform digital clock dispersion when performing analog operation in at least one of the camera modules (1100a, 1100b, 1100c), as described in FIGS. 1 to 10.

[0096] A camera module controller (1216) can provide control signals to each camera module (1100a, 1100b, 1100c). Control signals generated from the camera module controller (1216) can be provided to the corresponding camera modules (1100a, 1100b, 1100c) through separate control signal lines (CSLa, CSLb, CSLc). Additionally, one of the plurality of camera modules (1100a, 1100b, 1100c) may be designated as a master camera (e.g., 1100b) according to image generation information including a zoom signal or a mode signal, and the remaining camera modules (e.g., 1100a, 1100c) may be designated as slave cameras. This information may be included in a control signal and provided to the corresponding camera modules (1100a, 1100b, 1100c) through separate control signal lines (CSLa, CSLb, CSLc).

[0097] The camera module operating as a master and slave may be changed according to the zoom factor or operation mode signal. For example, if the field of view of the camera module (1100a) is wider than the field of view of the camera module (1100b) and the zoom factor indicates a low zoom magnification, the camera module (1100b) may operate as a master and the camera module (1100a) may operate as a slave. Conversely, if the zoom factor indicates a high zoom magnification, the camera module (1100a) may operate as a master and the camera module (1100b) may operate as a slave.

[0098] In an embodiment, the control signal provided to each camera module (1100a, 1100b, 1100c) from the camera module controller (1216) may include a sync enable signal. For example, if the camera module (1100b) is a master camera and the camera modules (1100a, 1100c) are slave cameras, the camera module controller (1216) may transmit a sync enable signal to the camera module (1100b). The camera module (1100b) that receives this sync enable signal may generate a sync signal based on the received sync enable signal and provide the generated sync signal to the camera modules (1100a, 1100c) through a sync signal line (SSL). The camera module (1100b) and camera modules (1100a, 1100c) can be synchronized with this sync signal to transmit image data to the application processor (1200).

[0099] In an embodiment, the control signal provided to a plurality of camera modules (1100a, 1100b, 1100c) from the camera module controller (1216) may include mode information according to a mode signal. Based on this mode information, the plurality of camera modules (1100a, 1100b, 1100c) may operate in a first operation mode and a second operation mode with respect to the sensing speed.

[0100] A plurality of camera modules (1100a, 1100b, 1100c) can, in a first operating mode, generate an image signal at a first speed (e.g., generate an image signal at a first frame rate) and encode it at a second speed higher than the first speed (e.g., encode an image signal at a second frame rate higher than the first frame rate), and transmit the encoded image signal to an application processor (1200). At this time, the second speed may be 30 times or less of the first speed.

[0101] The application processor (1200) stores the received image signal, that is, the encoded image signal, in a memory (1230) provided internally or in a storage (1400) outside the application processor (1200), and subsequently decodes the encoded image signal from the memory (1230) or the storage (1400) and displays image data generated based on the decoded image signal. For example, a corresponding sub-processor among a plurality of sub-processors (1212a, 1212b, 1212c) of the image processing device (1210) may perform decoding and may also perform image processing on the decoded image signal.

[0102] A plurality of camera modules (1100a, 1100b, 1100c) can generate an image signal at a third speed lower than the first speed in a second operation mode (e.g., generate an image signal at a third frame rate lower than the first frame rate) and transmit the image signal to an application processor (1200). The image signal provided to the application processor (1200) may be an unencoded signal. The application processor (1200) may perform image processing on the received image signal or store the image signal in memory (1230) or storage (1400).

[0103] The PMIC (1300) can supply power, such as power voltage, to each of the plurality of camera modules (1100a, 1100b, 1100c). For example, the PMIC (1300) can supply first power to the camera module (1100a) through a power signal line (PSLa), supply second power to the camera module (1100b) through a power signal line (PSLb), and supply third power to the camera module (1100c) through a power signal line (PSLc), under the control of the application processor (1200).

[0104] The PMIC (1300) can generate power corresponding to each of the plurality of camera modules (1100a, 1100b, 1100c) and adjust the power level in response to a power control signal (PCON) from the application processor (1200). The power control signal (PCON) may include power adjustment signals for each operating mode of the plurality of camera modules (1100a, 1100b, 1100c). For example, the operating mode may include a low power mode, and in this case, the power control signal (PCON) may include information about the camera module operating in the low power mode and the power level being set. The power levels provided to each of the plurality of camera modules (1100a, 1100b, 1100c) may be the same or different from each other. Additionally, the power level may be changed dynamically.

[0105] Meanwhile, the above-described contents of the present invention are merely specific embodiments for carrying out the invention. The present invention will include not only the concrete and practically usable means themselves, but also technical ideas, which are abstract and conceptual ideas that can be utilized as technology in the future. Explanation of the symbols

[0106] 10, 20: Image device 100, 110a: Image sensing unit 200, 300, 400: Digital processing unit 170: Timing controller 101, 201: Digital clock generator A1, A2: Analog-to-digital conversion section G1, G2: Clock cutoff section D1, D2: Clock division interval S1, S2: Stall section

Claims

Claim 1 An imaging device comprising: an image sensing unit that receives image signals from pixels, converts the received image signals into digital, and outputs the converted image data; and a digital processing unit that processes image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, and when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock, and the image sensing unit includes an analog-to-digital conversion circuit that converts the image signals into image data by comparing the received image signals with a ramp voltage and performing count operations corresponding to the comparison results; and a timing controller that generates a counter activation signal related to the count operations; wherein the digital clock generator distributes the digital clock according to a predetermined timing sequence in response to the counter activation signal. Claim 2 delete Claim 3 delete Claim 4 An imaging device according to claim 1, wherein the digital clock generator distributes the digital clock according to a predetermined timing sequence in response to a horizontal synchronization signal of the timing controller. Claim 5 An imaging device comprising: an image sensing unit that receives image signals from pixels, converts the received image signals into digital, and outputs the converted image data; and a digital processing unit that processes image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, and when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock, the image sensing unit performs a first ADC (Analog-to-Digital Conversion) operation corresponding to a reset voltage and a second ADC operation corresponding to a pixel voltage, and the digital clock generator gates the digital clock in all or part of each of the first ADC section corresponding to the first ADC operation and the second ADC section corresponding to the second ADC operation. Claim 6 An imaging device according to claim 1, wherein the image sensing unit performs a first ADC (Analog-to-Digital Conversion) operation corresponding to a reset voltage and a second ADC operation corresponding to a pixel voltage, and the digital clock generator divides the digital clock in all or part of each of the first ADC section corresponding to the first ADC operation and the second ADC section corresponding to the second ADC operation. Claim 7 An imaging device comprising: an image sensing unit that receives image signals from pixels, converts the received image signals into digital, and outputs converted image data; and a digital processing unit that processes image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, and when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock, the image sensing unit performs a first ADC (Analog-to-Digital Conversion) operation corresponding to a reset voltage and a second ADC operation corresponding to a pixel voltage, and the digital clock generator gates the digital clock in a portion of each of the first ADC section corresponding to the first ADC operation and the second ADC section corresponding to the second ADC operation, and divides the digital clock in the remainder of each of the first ADC section and the second ADC section. Claim 8 An imaging device according to claim 1, wherein the image sensing unit performs a first ADC (Analog-to-Digital Conversion) operation corresponding to a reset voltage and a second ADC operation corresponding to a pixel voltage, and the digital clock generator distributes the digital clock in a first stall section including a first ADC section corresponding to the first ADC operation and a second stall section including a second ADC section corresponding to the second ADC operation. Claim 9 An image sensing unit that receives image signals from pixels, converts the received image signals into digital, and outputs the converted image data; An imaging device comprising a digital processing unit that processes image data in synchronization with a digital clock, wherein the digital processing unit includes a digital clock generator that generates the digital clock, wherein when the image signals are converted into image data in the image sensing unit, the digital clock generator distributes the digital clock, wherein the image sensing unit performs a first ADC (Analog-to-Digital Conversion) operation corresponding to a reset voltage and a second ADC operation corresponding to a pixel voltage, wherein the digital clock generator distributes the digital clock in a first stall section including a first ADC section corresponding to the first ADC operation and a second stall section including a second ADC section corresponding to the second ADC operation, wherein each of the first ADC section and the second ADC section is divided into a plurality of clock distribution sections, and wherein the digital clock is distributed according to different methods in the plurality of clock distribution sections. Claim 10 A method of operating an imaging device comprising: a step of converting an image signal into image data by performing a count operation until the image signal and the ramp voltage are equal; and a step of reducing digital power consumption by using a clock dispersion method when performing the count operation, wherein the clock dispersion method is an operation of dispersing a digital clock according to a predetermined timing sequence in response to a counter activation signal related to the count operation.

Images