Pixel array for reducing loss of image information and image sensor including the same

Abstract

Disclosed are a pixel array and an image sensor including the same that reduce the loss of image information. The pixel array includes: a color filter array (CFA) unit, each CFA unit includes a color filter array block, each color filter array block includes a color pixel, and the color pixel includes a sub-block. The sub-block includes at least one first color pixel that senses a first color, at least one second color pixel that senses a second color different from the first color, and at least one third color pixel that senses a third color different from the first color and the second color.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Korean Patent Application No. 10-2020-0013305, filed on February 4, 2020, and Korean Patent Application No. 10-2020-0056661, filed on May 12, 2020, the entire subject matter of which is incorporated herein by reference in its entirety. Technical Field

[0003] The inventive concept relates to image sensors, and more particularly, to image sensors including a pixel array that reduces loss of image information during signal processing. Background Art

[0004] An image sensor is a device that captures a two-dimensional (2D) or three-dimensional (3D) image of an object. An image sensor generates an image of an object by using a photoelectric conversion device that reacts based on the intensity of light reflected from the object. Recently, with the development of complementary metal oxide semiconductor (CMOS) technology, CMOS image sensors (CIS) using CMOS are being widely used.

[0005] The image sensor may include a pixel array, and as the image sensor is implemented with a high resolution, the pixel array may include a plurality of color pixels. A mosaic process based on interpolation and / or extrapolation may be performed to convert a raw image output from the image sensor into a specific mode such as an RGB image, and when the distance from the adjacent color pixels that can be referred to during the process increases, a problem of loss of image information occurs. Summary of the invention

[0006] The inventive concept provides a pixel array and an image sensor including the same, which prevent an increase in image loss during processing of an image captured by the image sensor.

[0007] According to one aspect of the inventive concept, a pixel array is provided, which includes: a color filter array (CFA) unit, each CFA unit includes a CFA block, each CFA block includes a color pixel, and the color pixel includes a sub-block, wherein the sub-block includes at least one first color pixel for sensing a first color, at least one second color pixel for sensing a second color different from the first color, and at least one third color pixel for sensing a third color different from the first color and the second color.

[0008] According to one aspect of the inventive concept, an image sensor is provided, the image sensor comprising: a pixel array including a color filter array (CFA) unit, each CFA unit including a CFA block, each CFA block including color pixels, the color pixels of each CFA block including sub-blocks, wherein the sub-blocks include at least one first color pixel for sensing a first color, at least one second color pixel for sensing a second color different from the first color, and at least one third color pixel for sensing a third color different from the first color and the second color; a row driver configured to generate a control signal, the control signal controlling the generation of pixel signals by the color pixels of the pixel array; a read circuit configured to generate pixel data according to the pixel signals; and a controller configured to control the row driver and the read circuit.

[0009] According to one aspect of the inventive concept, there is provided an image sensor pixel array, comprising: a plurality of color filter array (CFA) units, each CFA unit comprising a 2×2 arranged CFA block, wherein each CFA block comprises color pixels of the same color, and the color pixels of the same color are arranged in a boundary area of ​​a surrounding sub-block of the CFA block, wherein the sub-block comprises at least one first color pixel for sensing a first color, at least one second color pixel for sensing a second color different from the first color, and at least one third color pixel for sensing a third color different from the first color and the second color. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Embodiments of the inventive concept will be more clearly understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0011] Figure 1 is a block diagram illustrating an image sensor pixel array according to an embodiment of the inventive concept;

[0012] Figure 2 and Figure 3 Examples of pixel arrays according to embodiments of the inventive concept are respectively shown;

[0013] Figure 4 is a conceptual diagram illustrating a re-stitching operation performed on a color mode according to an embodiment of the inventive concept;

[0014] Figure 5 and Figure 6 are diagrams showing examples of an interpolation value distance determination method, respectively;

[0015] Figure 7 is based on Figure 4 Example of interpolated distance calculated for the GR-GB-GR pattern pair;

[0016] Figure 8 , Fig. 9 , Fig.10 and Fig.11 are diagrams showing various examples of pixel arrays according to embodiments of the inventive concept;

[0017] Fig.12 is a block diagram showing an example of an image processing apparatus including an image sensor according to an embodiment;

[0018] Fig.13 is a diagram showing an implementation example of a color pixel according to an embodiment;

[0019] Fig.14 is a block diagram of an electronic device including a multi-camera module to which an image sensor according to an embodiment is applied; and

[0020] Fig.15 yes Fig.14 Block diagram of the camera module. DETAILED DESCRIPTION

[0021] Hereinafter, embodiments of the inventive concept will be described in some additional detail with reference to the accompanying drawings.

[0022] Figure 1 1 is a block diagram illustrating an image sensor 100 according to an embodiment of the inventive concept. In a particular embodiment, the image sensor 100 may include a complementary metal oxide semiconductor (CMOS) image sensor (CIS). The image sensor 100 may include a pixel array 110, a row driver 120, a read circuit 130, and a controller 140, wherein the controller 140 may be used to control the row driver 120 and the read circuit 130.

[0023] The pixel array 110 may include pixels, wherein each pixel may include at least one photo-sensing device. Here, the photo-sensing device may detect (or sense) electromagnetic energy in a defined frequency band (hereinafter, "incident light"). Therefore, each pixel of the pixel array 110 may generate an electrical signal corresponding to the intensity of the incident light. For example, the photo-sensing device may include a photodiode, a photogate, and / or a phototransistor. In certain embodiments, the pixels of the pixel array 110 may be color pixels arranged in various patterns. Therefore, each color pixel may generate a corresponding electrical signal (hereinafter, "pixel signal") associated with one or more colors.

[0024] Various signal processing operations may be performed on the pixel signals provided by the pixel array 110. For example, a re-mosaic operation may be performed on the pixel signals, and based on the re-mosaic operation, the pattern of the pixel array 110 may be converted to a desired pattern (eg, a Bayer pattern).

[0025] In this regard, the pixel array 110 may output pixel signals respectively corresponding to incident light absorbed by the photosensitive devices to the read circuit 130. The row driver 120 may output a control signal for controlling each color pixel included in the pixel array 110. For example, the row driver 120 may output a control signal that may reset the photosensitive devices or may cause the photosensitive devices to output pixel signals in response to accumulated charges associated with the incident light.

[0026] Read circuit 130 may receive pixel signals output by pixel array 110 and may generate pixel values ​​(or pixel data). For example, read circuit 130 may include at least one analog-to-digital converter (ADC) configured to generate digital signals corresponding to one or more analog pixel signals as pixel data.

[0027] In this way, Figure 1 Image sensor 100 may provide pixel data to an image processing device (not shown) capable of performing one or more digital signal processing operations (e.g., a re-stitching operation). Such digital signal processing may be performed "internal" to image sensor 100, as opposed to external signal processing that may be performed by a processing device (rather than image sensor 100).

[0028] In the following, reference will be made to Figure 1 , Figure 2 and Figure 3 Specific implementation examples for the pixel array 110 and constituent color pixels of the pixel array 110 according to an embodiment of the inventive concept are described.

[0029] Reference Figure 1 and Figure 2 , the pixel array 110 may include one or more color filter arrays (CFAs). In various arrangements, these CFAs may be referred to as CFA units. Here, each CFA may be configured to sense a specific color. Hereinafter, the terms "color filter", "color pixel", "filter array", and "pixel array" may be used variously in the description of exemplary CFAs.

[0030] For example, a CFA may be defined as a separate (or identifiable) element disposed in the pixel array 110 and including at least one photosensitive device. Alternatively, a CFA may be defined as a portion of the overall pixel array 110 (e.g., a color pixel including a corresponding color filter). In this regard, a CFA unit, a CFA block, and / or a CFA sub-block may be understood to include at least one color pixel.

[0031] In certain embodiments, the pixel array 110 may include a plurality of CFA units 111 defined by an arrangement of constituent units. For example, the pixel array 110 may include a plurality of CFA units 111 disposed (or arranged) in a lateral direction and a longitudinal direction. Each of the CFA units 111 may include a specific number of color pixels (i.e., have a defined "size"). The color pixels forming the CFA units may be arranged differently (e.g., in a defined matrix having a plurality of rows and a plurality of columns).

[0032] In certain embodiments, each of CFA units 111 may include a plurality of CFA blocks, wherein each of the CFA blocks has a defined structure (e.g., a structure of a minimum size). Thus, the plurality of CFA blocks may be repeatedly arranged to form a CFA unit 111 of a desired size, shape, and / or structure.

[0033] Figure 2 One possible CFA 111 is shown including M CFA blocks arranged in rows in the longitudinal direction and N CFA blocks arranged in columns in the transverse direction, where 'M' and 'N' are both integers greater than 1. Figure 2 The CFA 111 is described as including M times N (M×N) CFA blocks.

[0034] Here, it should be noted that emerging, high-quality image sensors (eg, CIS) may include huge arrays and a large number of color pixels. Therefore, the size of the CFA unit and the number and size of the blocks that make up the CFA may increase.

[0035] Thus, when a particular CFA block of CFA unit 111 is associated with a single color (e.g., when the CFA block includes pixels that sense a single color), the distance between adjacent color pixels may become an increasingly important design factor. For example, during certain signal processing operations (such as interpolation, extrapolation, etc.), it may be necessary to reference adjacent color pixels.

[0036] Therefore, given increasing CFA unit and CFA block sizes (e.g., as image size increases), the process of converting pixel data provided by image sensor 100 into, for example, an RGB image may result in a loss of image information. That is, as the size of the CFA block increases, the distance between adjacent color pixels may increase, thereby increasing the loss rate.

[0037] However, a pixel array according to an embodiment of the present inventive concept (e.g., pixel array 110 of image sensor 100) may include a plurality of CFA units 111, wherein each CFA unit 111 includes a plurality of CFA blocks, and yet color pixels (or color filters) associated with each CFA unit 111 and / or CFA block may have a pattern that effectively reduces a loss rate during image processing operations.

[0038] An implementation example according to an embodiment of the inventive concept will be described, assuming that each CFA unit 111 includes four (4) equally sized CFA blocks arranged horizontally and vertically in a 2×2 configuration. It is further assumed that the heterogeneous CFA blocks can be configured to sense at least two colors, not just one color. In one example, the CFA blocks according to an embodiment of the inventive concept can sense all colors applied to the pixel array 110. In various methods, a CFA pattern including multiple CFA blocks can be designed to effectively reduce the possible loss of image information.

[0039] Figure 3 One possible configuration for a CFA block according to an embodiment of the inventive concept is shown, the CFA block including color pixels arranged in a c by d (c×d) matrix of color pixels, where 'c' and 'd' are both integers greater than 1. Here, for example, each of the color pixels may be a red pixel, a blue pixel, or a green pixel, and the CFA block may include at least one red pixel, at least one blue pixel, and / or at least one green pixel.

[0040] exist Figure 3 In the illustrated embodiment of , it is assumed that the CFA block includes one or more sub-blocks, wherein each sub-block includes, for example, an a by b (a×b) matrix of color pixels, wherein 'a' and 'b' are both integers greater than 1.

[0041] In certain embodiments, at least two of the first color pixel, the second color pixel, and / or the third color pixel (e.g., color pixels sensing the first color, the second color, and the third color, respectively) of the CFA block may be respectively disposed in certain regions (e.g., a boundary region, an inner region, a left region, a right region, an upper region, a lower region, etc.) of the CFA block. Figure 3 In the illustrated example of , each of the red pixel R, the blue pixel B, and the green pixel G is included in a sub-block and may be respectively disposed in a boundary region of the CFA block. In this regard, certain embodiments of the inventive concept may define a "boundary region" to include at least some of the peripheral region. In addition, in this regard, certain embodiments of the inventive concept may include a boundary region that surrounds (or at least partially surrounds) a sub-block disposed in the middle.

[0042] In certain embodiments, based on the number of various color pixel types included in the CFA block, the CFA block may be considered a first color (e.g., red) CFA block, a second color (e.g., blue) CFA block, or a third color (e.g., green) CFA block. For example, when the number of red pixels is the largest (or dominant) among the color pixels of the CFA block, the CFA block may be considered a red CFA block. Alternatively, if the red pixels are dominantly disposed in (arranged around or along) a boundary region of the CFA block, the CFA block may be considered a red CFA block.

[0043] In the above-described embodiment, the red pixel, the blue pixel, and the green pixel all applied to the pixel array 110 are merely examples, and the embodiment is not limited thereto. For example, color pixels based on various filters such as a cyan filter and an RGBW filter may be applied to the embodiment, and the embodiment is not limited to a mode of sensing a specific color.

[0044] Figure 4 (include Figure 4 (a), (b), (c), (d), (e) and (f)) in the figures are conceptual diagrams further illustrating the re-stitching process of the color pattern for the pixel array according to an embodiment of the present invention. Here, for simplicity, a simple 8×8 CFA unit is assumed, which includes four (4) 4×4 CFA blocks arranged in 2×2 (e.g., 2 rows and 2 columns), but it should be clear to those skilled in the art that any reasonable number of color units and / or CFA blocks may be used in other embodiments of the present invention. Here, the CFA unit pattern includes all of the 8×8 color units. Therefore, it is assumed that Figure 4 The CFA unit in (e) comprises four (4) heterogeneous 4×4 CFA blocks. It is also assumed that Figure 4 The CFA unit of (e) includes red pixels, green pixels, and blue pixels, and each of the CFA blocks includes at least one red pixel, at least one green pixel, and at least one blue pixel.

[0045] Figure 4 The re-stitching operation is shown. Figure 4 The CFA unit pattern of the 8×8 CFA unit in (e) is converted to Figure 4 Therefore, the CFA unit pattern and the converted Bayer pattern may be referred to as a pattern CFA pair or a CFA pattern pair.

[0046] exist Figure 4 In the illustrated embodiment, Figure 4 The first CFA block of (a) in FIG. 1 predominantly includes red pixels and is referred to as a red CFA block. Similarly, Figure 4The second CFA block of (b) in FIG. 1 predominantly includes green pixels and is referred to as a green CFA block, and Figure 4 The third CFA block of (c) in FIG. 1 predominantly includes blue pixels and is referred to as a blue CFA block. In this context, it should be noted that the term "predominantly includes" refers to arranging color pixels of a specific type around the boundary of the corresponding CFA block. However, this is not always the case, and different embodiments of the inventive concept may variously and differently define the type or name of a CFA block having a specific color according to various definitions of the term "predominantly includes", such as (for example) relative numerical counts of different color pixels in a CFA block, relative positional settings of various color pixels in a CFA block, etc.

[0047] like Figure 4 As shown in (e) of FIG. , an exemplary 8×8 CFA unit includes a red CFA block (upper right), two (2) green CFA blocks (upper left and lower right), and a blue CFA block (lower left).

[0048] Each of the green CFA block, the red CFA block, and the blue CFA block includes a sub-block having a defined size and / or a defined composition of color pixels. For example, the sub-blocks may be a 2×2 arrangement of color pixels. In a particular embodiment, each sub-block will include at least one red pixel, at least one green pixel, and at least one blue pixel. In a particular embodiment, the corresponding color pixels of the sub-blocks will be arranged in the same manner (e.g., in Figure 4 In the example shown in (d), from top to bottom and from left to right, it is green-blue-red-green).

[0049] exist Figure 4 In the example of (a) in FIG. 5 , a 4×4 red CFA block may include thirteen (13) red pixels, two (2) green pixels, and one (1) blue pixel. Figure 4 In the example of (b) in FIG. 1 , a 4×4 green CFA block may include one (1) red pixel, fourteen (14) green pixels, and one (1) blue pixel. Figure 4 In the example of (c) in FIG. 1 , a 4×4 blue CFA block may include one (1) red pixel, two (2) green pixels, and thirteen (13) blue pixels. More notably, Figure 4 The red CFA block, green CFA block, and blue CFA block in (a), (b), and (c) all include Figure 4 Here, for the convenience of description, Figure 4 The 8×8 CFA unit of (e) in FIG. 5 may be referred to as a GR-GB mode unit based on two letters of each of the GBRG pixels of the GRBG CFA block and the sub-block.

[0050] Considering the above exemplary embodiments, a re-stitching operation can be used to convert a CFA unit into one of various color modes. That is, Figure 4 The CFA unit in (e) can be converted to Figure 4 The Bayer pattern of (f) in Figure 4 As shown in (f) of , the Bayer pattern obtained by conversion using a re-stitching operation may have a structure in which a plurality of 2×2 pixels are repeatedly arranged. For example, the 2×2 pixels may include green-red-blue-green (GRBG) pixels arranged in the order of the upper left, the upper right, the lower left, and the lower right. For ease of description, a pair of a GR-GB pattern unit and a GRBG after a re-stitching operation for obtaining a Bayer pattern may be referred to as a GR-GB-GR pattern pair.

[0051] In order to apply Figure 4 For each of the pattern pairs in (e) and (f) in FIG. 1 , converting the 8×8 CFA unit to the Bayer pattern using a re-stitching operation may require interpolation of red, green, and blue, respectively. Assuming that interpolation is performed on green, edge information of the image may be included more in green than in red and blue. Therefore, neighboring green pixels required for interpolation in each of four (4) directions (e.g., a forward diagonal direction, a reverse diagonal direction, a horizontal direction, and a vertical direction) may be selected, and the average value of the distances between n (where 'n' is an integer greater than 1) neighboring green pixels and a target pixel (e.g., a center pixel) for which interpolation is calculated may be calculated as the interpolation distance of green. The selection of neighboring green pixels is performed in an order of proximity based on a direction relative to the target pixel (e.g., the center pixel) for which interpolation is calculated. This method may be used to calculate the interpolation distance relative to the target pixel as one of many possibilities. Alternatively, in the case of performing interpolation values ​​on blue and / or red, one or more neighboring pixels may be selected regardless of the direction relative to a target pixel for which the interpolation value is calculated.

[0052] Figure 5 and Figure 6 is a conceptual diagram further illustrating an example of determining an interpolation distance. Figure 5 In , the interpolation distance is determined for the interpolated value of green, and in Figure 6 , the interpolation distance is determined for the interpolated values ​​of red and blue. Figure 5 and Figure 6 An example is shown in which two (2) adjacent color pixels are selected in each direction during interpolation, however, the inventive concept is not limited thereto. In other embodiments, more than two (2) adjacent color pixels may be selected and interpolation may be performed.

[0053] Reference Figure 5 (include Figure 5 In (a), (b), (c), (d) and (e)), multiple groups of two (2) adjacent green pixels that are closest to each other in each specific direction are selected for the interpolated value of green. Therefore, adjacent green pixels that are closest to each other in the forward diagonal direction, the reverse diagonal direction, the horizontal direction and the vertical direction relative to the target pixel (e.g., the central red pixel) are selected. Figure 5 In the case of, when the number n of neighboring pixels is 1, the first interpolation distance may be calculated as 1.21. In addition, when the number n of neighboring pixels is 2, the second interpolation distance may be calculated as 2.13, and when the number "n" of neighboring pixels is 3 or more, the third interpolation distance and the fourth interpolation distance may be calculated similarly.

[0054] Figure 6 (include Figure 6 (a), (b), (c) and (d)) show examples of interpolation value distances determined for the interpolation value of red relative to a target pixel (e.g., a central green pixel) (see Figure 6 (a) and (b) in FIG. 1 ), and / or an example of an interpolation distance determined relative to the interpolation value of the target pixel for blue (see FIG. 1 ). Figure 6 (c) and (d) in the figure.

[0055] In the interpolation values ​​of red and blue, regardless of the direction, one or more neighboring pixels closest to each other relative to the center pixel can be selected, and the average value of the distances between the selected neighboring pixels and the target pixel for which the interpolation values ​​are calculated can be calculated as the interpolation value distance of red / blue. Figure 6 An example is shown in which two (2) neighboring pixels closest to each other are selected in the interpolated values ​​of red and blue, and as shown in FIG. Figure 6 As shown in (b) of FIG. 1 , two (2) adjacent red pixels that are closest to each other in the vertical direction may be selected. Figure 6 As shown in (d) in FIG. 1 , two adjacent blue pixels that are closest to each other in the vertical direction may be selected. In this case, when the number of adjacent pixels "n" is 1 and 2, respectively, relative to Figure 6 The interpolation distance of the target pixel of (a) and (c) in FIG. 1 can be calculated as 2.0. In addition, even when the number "n" of neighboring pixels is 3 or more, the value of the interpolation distance can be similarly calculated.

[0056] In certain embodiments, as described above, the CFA unit may include CFA blocks (e.g., 2×2 CFA blocks), and each of the CFA blocks may include sub-blocks (e.g., 2×2 color pixels) having a specific pattern, thereby reducing the interpolation value distance calculated relative to the target pixel. Figure 4As shown, each CFA block may include color pixels associated with at least two (2) different colors (rather than a single color). In addition, each CFA block may include at least one red pixel, at least one green pixel, and at least one blue pixel. Thus, adjacent color pixels that are in relatively close locations relative to a target pixel may be selected.

[0057] In addition, in certain embodiments, reference Figure 4 For the pattern pair in the embodiment of the present invention, the color pixels of the sub-block in the CFA unit (e.g., 2×2 color pixels indicated by 'SB') may have the same color pattern as the color pattern of the pixels at the same position in the converted Bayer pattern, and thus, the interpolation distance of each of the color pixels in the sub-block in the CFA unit may be calculated as 0. Therefore, according to the embodiment, the finally calculated average interpolation distance for the CFA unit may be reduced, and thus, for example, the image loss rate may be reduced in the interpolation processing operation and / or the extrapolation processing operation using the re-stitching operation.

[0058] Figure 7 Yes Figure 7 A set of graphs (a) and (b) in further illustrate the Figure 4 The GR-GB-GR pattern pairs in (e) and (f) determine (or calculate) the interpolation distance.

[0059] Here, the corresponding interpolation distance (or value) can be calculated by applying the above calculation method to all the re-adjusted color pixels. Figure 7 As shown, the interpolation value distance of each of the color pixels in the sub-blocks included in each of the CFA blocks of the CFA unit can be calculated as 0. In addition, in other color pixels, the interpolation value distance of each of the color pixels in the sub-blocks included in each of the CFA blocks of the CFA unit can be calculated as 0. Figure 4 The interpolation value distance of each of the adjacent color pixels referenced by the interpolation value is reduced by using each of the pattern pairs in (e) and (f). Therefore, the total average of the interpolation value distances calculated in the CFA unit can be minimized, thereby minimizing the loss of image information.

[0060] Hereinafter, examples of various color filter arrays that can be implemented according to embodiments of the present inventive concept will be described. However, the scope of the present inventive concept is not limited to the examples shown. With respect to the detailed examples of the color filter arrays described below, the detailed pixel arrangement of the color filter arrays can be modified under the conditions that the specific benefits of the present inventive concept can be achieved.

[0061] Figure 8 (include Figure 8 (a), (b), (c) and (d)), Fig. 9 (include Fig. 9(a), (b), (c), (d), (e), (f), (g), (h) and (i)), Fig.10 (include Fig.10 (a), (b), (c), (d), (e), (f), (g), (h) and (i)) and Fig.11 (include Fig.11 (a), (b), (c), (d), (e), (f), (g), (h) and (i)) (collectively referred to as “ Figures 8 to 11 ”) is a diagram illustrating additional examples of pixel arrays according to embodiments of the inventive concept.

[0062] Here again, a CFA unit may include color pixels arranged in various patterns. Figure 4 The pattern is generated by modifying various methods such as flipping, mirroring, transposing and / or cropping. Figures 8 to 11 Each of the pattern pairs shown in .

[0063] For example, Figure 8 The pattern pairs shown can be combined with Figure 4 The pattern pair of corresponds to the pattern pair obtained by applying the cropping modification. In addition, Fig. 9 The pattern pairs shown can be combined with Figure 4 The mode applies the transpose or Figure 4 corresponds to the pattern pair obtained by applying the transpose and crop modifications, and Fig.10 The pattern pairs shown can be combined with Figure 4 The mode is for application mirroring or Figure 4 The pattern pair of corresponds to the pattern pair obtained by applying the mirroring and cropping modifications. In addition, Fig.11 The pattern pairs shown can be combined with Figure 4 The mode of the application is flipped or Figure 4 The pattern pair corresponds to the pattern pair obtained by applying the flip and crop modifications.

[0064] In the above description, the pixel array according to the embodiment may include Figure 4 The CFA unit of the mode shown may also include a Figures 8 to 11 In addition, in addition to the modes described herein, the pixel array according to the embodiment may also have various modes that can be modified differently. In addition, the above-mentioned terms "flip", "mirror", "transpose" and "crop" may be terms used to describe the concepts of various modes applied to the CFA unit according to the embodiment, and the pixel array according to the embodiment of the present invention may optionally include the various CFA units described herein.

[0065] Reference Figure 8 ,according to Figure 4 The CFA units of the embodiment of the present invention may be repeatedly arranged in the horizontal direction and the vertical direction, and each of the CFA units may include a green CFA block disposed at the upper left, a red CFA block disposed at the upper right, a blue CFA block disposed at the lower left, and a green CFA block disposed at the lower right. In addition, each of the green CFA block, the red CFA block, and the blue CFA block may include a Figure 4 2 shows sub-blocks of the same specific size (eg, a size of 2×2), and the sub-blocks may include a green pixel, a blue pixel, a red pixel, and a green pixel in the order of upper left, upper right, lower left, and lower right.

[0066] exist Figure 8 In the figure, they are shown respectively Figure 4 The CFA unit with an 8×8 size shown in (e) has color pixels repeated twice in the horizontal and vertical directions. In this case, the coordinates of the color pixels set in the leftmost and topmost parts can be defined as (1, 1), and the coordinates of the color pixels set in the rightmost and bottommost parts can be defined as (16, 16).

[0067] According to certain embodiments, CFA units having various modes may be defined, for example, with respect to Figure 8 By performing cropping on the region PR1 from (5, 5) to (12, 12), the (x, y) coordinates defined in (a) are Figure 8 (b) in FIG. 4 shows a GB-GB CFA pattern unit, which includes green-blue-red-green (GBRG) CFA blocks arranged in the order of upper left, upper right, lower left, and lower right. In addition, by performing cropping on the region PR2 from (5, 9) to (12, 16), Figure 8 (c) in FIG. 4 shows an RG-GB CFA pattern unit, which includes red-green-green-blue (RGGB) CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and by performing cropping on a region PR3 from (9, 5) to (16, 12), Figure 8 (d) in FIG. 4 shows a BG-GB CFA pattern unit, which includes a blue-green-green-red (BGGR) CFA block arranged in the order of upper left, upper right, lower left, and lower right. In addition, Figure 8 Each of the GB-GB CFA mode unit, the RG-GB CFA mode unit, and the BG-GB CFA mode unit shown in FIG. Figure 4 The same sub-block as shown in (d) in FIG. 1 , in addition, Figure 8 The CFA model unit shown and Figure 4The GRBG Bayer pattern shown in (f) can form a pair.

[0068] That is to say, Figure 4 Compared with the CFA unit shown in (e) in the embodiment, Figure 8 In the CFA unit of the CFA unit shown in FIG. 1 , the relative positions of the CFA blocks may be changed, but the color pixels of the sub-blocks included in each of the CFA blocks may have the same Figure 4 In addition, in certain embodiments, the color pixels of the sub-block in the CFA unit may have the same pattern as the pattern of pixels at the same position in the converted Bayer pattern.

[0069] Fig. 9 shows a CFA unit and a Bayer pattern that can be implemented according to an embodiment, for example, Fig. 9 The pattern pairs shown can be used with Figure 4 Each of the patterns shown in (e) and (f) above is transposed or Figure 4 Each of the patterns shown in (e) and (f) corresponds to the case where transposition and cropping are applied.

[0070] The CFA unit according to an embodiment may have Fig. 9 In addition, the Bayer pattern obtained by the conversion based on the re-stitching operation performed on the color pixels of the CFA unit may have Fig. 9 The pattern shown in (f) in .

[0071] like Fig. 9 As shown in (e), (g), (h) and (i) in FIG. 1 , the CFA unit may include four (4) heterogeneous 4×4 red / green / blue CFA blocks, and a sub-block including 2×2 color pixels may be set in each CFA block. In addition, Fig. 9 The sub-blocks of the CFA blocks of (e), (g), (h), and (i) in FIG. 1 may be implemented to be identical to each other, and as shown in FIG. Fig. 9 As shown in (d) in Fig. 9 The sub-blocks of the CFA blocks of (e), (g), (h) and (i) may each include green-red-blue-green color pixels arranged in the order of upper left, upper right, lower left and lower right.

[0072] According to various embodiments, Fig. 9 As shown in (e) in FIG. 8 , the CFA unit may include green-blue-red-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GB-GR mode unit. Fig. 9The GB-GR pattern unit of (e) may have a structure symmetrical to the GR-GB pattern unit according to the above-described embodiment in the reverse oblique direction (or diagonal line).

[0073] like Fig. 9 As shown in (g) in FIG. 1 , the CFA unit may include green-red-blue-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GR-GR mode unit. In addition, Fig. 9 The CFA unit of (h) may include a blue-green-green-red CFA block arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a BG-GR mode unit. In addition, Fig. 9 The CFA unit of (i) may include red-green-green-blue CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as an RG-GR mode unit. Fig. 9 Each of the CFA units shown in (e), (g), (h) and (i) and Fig. 9 The GBRG Bayer pattern in (f) can constitute a pattern pair.

[0074] Fig.10 shows a CFA unit and a Bayer pattern that may be implemented according to various other embodiments, and for example, Fig.10 The pattern pairs shown can be used with Figure 4 Each of the patterns shown in (e) and (f) applies a mirror image or a Figure 4 Each of the modes shown in (e) and (f) in FIG. 1 corresponds to the case where mirroring and cropping are applied. The CFA unit according to the embodiment may have Fig.10 In addition, the Bayer pattern obtained by the conversion based on the re-stitching operation performed on the color pixels of the CFA unit may have Fig.10 The pattern shown in (f) in .

[0075] like Fig.10 As shown in (e), (g), (h) and (i) in FIG. 1 , the CFA unit may include four (4) heterogeneous 4×4 red / green / blue CFA blocks, and a sub-block including 2×2 color pixels may be set in each CFA block. In addition, Fig.10 The sub-blocks of the CFA blocks of (e), (g), (h), and (i) in FIG. 1 may be implemented to be identical to each other, and as shown in FIG. Fig.10 As shown in (d) in Fig.10 The sub-blocks of the CFA blocks of (e), (g), (h) and (i) may each include blue-green-green-red color pixels arranged in the order of upper left, upper right, lower left and lower right.

[0076] According to various embodiments, Fig.10 As shown in (e) of FIG. 1 , the CFA unit may include red-green-green-blue CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as an RG-BG pattern unit. In addition, by applying mirroring to the GR-GB pattern unit according to the above-described embodiment, Fig.10 The RG-BG mode unit of (e) may have a structure symmetrical to the GR-GB mode unit with respect to the longitudinal axis.

[0077] like Fig.10 As shown in (g) in FIG. 1 , the CFA unit may include a blue-green-green-red CFA block arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a BG-BG mode unit. In addition, Fig.10 The CFA unit of (h) may include green-red-blue-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GR-BG mode unit. In addition, Fig.10 The CFA unit of (i) may include green-blue-red-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GB-BG mode unit. Fig.10 Each of the CFA units shown in (e), (g), (h) and (i) and Fig.10 The RGGB Bayer pattern in (f) can constitute a pattern pair.

[0078] Fig.11 shows a CFA unit and a Bayer pattern that may be implemented according to other embodiments, for example, Fig.11 The pattern pairs shown can be used with Figure 4 Each of the patterns shown in (e) and (f) above should be flipped or Figure 4 Each of the modes shown in (e) and (f) in FIG. 1 corresponds to the case where flipping and cropping are applied. The CFA unit according to the embodiment may have the following Fig.11 In addition, the Bayer pattern obtained by the conversion based on the re-stitching operation performed on the color pixels of the CFA unit may have Fig.11 The pattern shown in (f) in .

[0079] like Fig.11 As shown in (e), (g), (h) and (i) in FIG. 1 , the CFA unit may include four (4) heterogeneous 4×4 red / green / blue CFA blocks, and a sub-block including 2×2 color pixels may be set in each CFA block. In addition, Fig.11The sub-blocks of the CFA blocks of (e), (g), (h), and (i) in FIG. 1 may be implemented to be identical to each other, and as shown in FIG. Fig.11 As shown in (d) in Fig.11 The sub-blocks of the CFA blocks of (e), (g), (h) and (i) in FIG. 1 may each include red-green-green-blue color pixels arranged in the order of upper left, upper right, lower left and lower right.

[0080] According to various embodiments, Fig.11 As shown in (e) of FIG. 1 , the CFA unit may include a blue-green-green-red CFA block arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a BG-RG pattern unit. In addition, by applying flipping to the GR-GB pattern unit according to the above-described embodiment, Fig.11 The BG-RG mode unit of (e) may have a structure symmetrical to the GR-GB mode unit with respect to the transverse axis.

[0081] like Fig.11 As shown in (g) in FIG. 1 , the CFA unit may include red-green-green-blue CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as an RG-RG mode unit. In addition, Fig.11 The CFA unit of (h) may include green-blue-red-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GB-RG pattern unit. In addition, Fig.11 The CFA unit of (i) may include green-red-blue-green CFA blocks arranged in the order of upper left, upper right, lower left, and lower right, and this may be referred to as a GR-RG pattern unit. Fig.11 Each of the CFA units shown in (e), (g), (h) and (i) and Fig.11 The BGGR Bayer pattern in (f) can constitute a pattern pair.

[0082] According to the above-mentioned embodiments, the color mode of the pixel array may be configured differently, and for example, a plurality of colors (e.g., red (R), green (G), and blue (B)) may be sensed together in a CFA block implemented to have a specific size, and thus, the distance to the adjacent color pixels used in the interpolation and / or extrapolation processing may be reduced, thereby reducing image loss. In the above-mentioned embodiments, it has been described that the color pixels of the outer area in one CFA block sense the same color, but the pixel array may be implemented so that at least one color pixel of the outer area senses a different color. In addition, it has been described that two green pixels are provided in each sub-block, but in an embodiment, some color pixels of each sub-block may be modified.

[0083] Fig.12 is a block diagram illustrating an image processing apparatus 200 including an image sensor according to an embodiment of the inventive concept.

[0084] Reference Fig.12 The image processing apparatus 200 may include a pixel array 210 having a CFA, and may include an image processor 220 that performs image processing by using pixel values ​​from the pixel array 210. According to an embodiment, the above-mentioned image sensor may include the pixel array 210, and may include at least some elements included in the image processor 220.

[0085] The pixel array 210 may include a plurality of CFA units 211 having various modes according to the above-described embodiments, and may provide pixel values ​​to the image processor 220 based on a plurality of color pixels arranged in the pixel array 210. In addition, the image processor 220 may include various logic devices for processing pixel values ​​to generate image data, for example, the image processor 220 may include a binning processor 221 and a re-stitching processor 222. Based on the control of the binning processor 221, when a high image quality mode is selected, image processing may be performed by using pixel values ​​having a relatively high resolution, or when a low image quality mode is selected, image processing may be performed by using pixel values ​​having a relatively low resolution. In addition, the re-stitching processor 222 may perform an interpolation operation on each pixel in association with the re-stitching process, and according to the above-described embodiments, a pixel value corresponding to the Bayer pattern may be generated by using the pixel values ​​of the CFA units.

[0086] Fig.13 is a diagram illustrating one possible implementation example of a color pixel according to an embodiment of the inventive concept.

[0087] According to the above embodiments, each color pixel may include a color filter corresponding thereto, and may also include a photoelectric sensing device (eg, a photodiode) for sensing light (or color).

[0088] According to an embodiment, a color pixel may include a plurality of sub-pixels. Fig.13 In the example, each color pixel includes 2×2 sub-pixels, but each color pixel may include more sub-pixels. In addition, the photoelectric sensing device can be set by sub-pixel unit, so multiple photoelectric sensing devices can be set based on one color pixel. In addition, although Fig.13 Although not shown in the figure, the image sensor according to the embodiment may include a plurality of micro lenses (not shown), and the micro lenses may be provided by pixel units, or may be provided by sub-pixel units.

[0089] When calculating the pixel signal corresponding to each color pixel, at least some of the signals generated by the plurality of sub-pixels may be used. Fig.13 The upper left color pixel is a red pixel and the plurality of sub-pixels similarly sense red, and a pixel signal provided from a corresponding red pixel may be generated by processing based on at least some of the signals generated by the plurality of sub-pixels.

[0090] The embodiments may be implemented differently, and for example, a signal generated by sensing two or more colors may be provided from one color pixel. For example, a plurality of color filters for sensing different colors may be provided in a plurality of sub-pixels included in one color pixel, and according to the above-described embodiments, each color pixel may be implemented differently within a range for reducing an interpolation distance.

[0091] In the process of describing the embodiment, for the convenience of description, various mode pairs based on cropping, transposition, mirroring and flipping or various mode pairs based on their combination have been described, but the embodiment is not limited thereto. Figure 4 The illustrated CFA cell applies other concepts such as rotation and translation, etc., and thus, CFA cells and pattern pairs according to embodiments may be implemented.

[0092] In addition, in the embodiment, an 8×8 CFA unit with 1×1 pixel as the minimum unit has been described above, but in the embodiment, the minimum unit is not limited to 1×1 pixel, and a color mode can be implemented based on the condition that a pixel group with a number of q×r (where 'q' is a positive integer and 'r' is a positive integer) is the minimum unit.

[0093] Fig.14 is a block diagram of an electronic device 1000 including a multi-camera module to which an image sensor according to an embodiment is applied. Fig.15 yes Fig.14 Detailed block diagram of the camera module.

[0094] Reference Fig.14 , the electronic device 1000 may include a camera module group 1100 , an application processor 1200 , a power management integrated circuit (PMIC) 1300 , and an external memory 1400 .

[0095] The camera module group 1100 may include one or more camera modules 1100a, 1100b, and 1100c. Although the drawings show an embodiment in which three camera modules 1100a, 1100b, and 1100c are provided, the embodiments are not limited thereto. In some embodiments, the camera module group 1100 may be modified to include only two camera modules. In addition, in some embodiments, the camera module group 1100 may be modified to include n (where n is a natural number of 4 or greater) camera modules.

[0096] In the following, reference will be made to Fig.15 A detailed configuration of the camera module 1100 b is described, but the following description is equally applicable to other camera modules 1100 a and 1100 c according to embodiments.

[0097] Reference Fig.15 , the camera module 1100b may include a prism 1105, an optical path folding element (OPFE) 1110, an actuator 1130, an image sensing device 1140 and a storage 1150.

[0098] The prism 1105 may include a reflective surface 1107 including a light reflecting material, and may change a path of a light ray L incident from the outside.

[0099] In some embodiments, the prism 1105 may change the path of the light L incident in the first direction X to a second direction Y perpendicular to the first direction X. In addition, the prism 1105 may rotate the reflective surface 1107 including the light reflective material in the A direction relative to the central axis 1106, or may rotate the central axis 1106 in the B direction to change the path of the light L incident in the first direction X to a second direction Y perpendicular to the first direction X. In this case, the OPFE 1110 may move in a third direction Z perpendicular to the first direction X and the second direction Y.

[0100] In some embodiments, as shown in the figure, in the A direction, the maximum rotation angle of the prism 1105 may be 15 degrees or less in the +A direction and may be greater than 15 degrees in the -A direction, but the embodiment is not limited thereto.

[0101] In some embodiments, the prism 1105 may be moved by about 20 degrees, 10 to 20 degrees, or 15 to 20 degrees in the +B or -B direction. Here, the movement angle in the +B or -B direction may be the same, or the prism 1105 may be moved up to an almost similar angle within a range of about 1 degree.

[0102] In some embodiments, the prism 1105 may move the reflective surface 1107 including the light reflective material in a third direction (eg, Z direction) parallel to the extension direction of the central axis 1106 .

[0103] For example, the OPFE 1110 may include m optical lenses grouped into m (where m is a natural number) groups. The m optical lenses may move in the second direction Y to change the optical zoom ratio of the camera module 1100b. For example, in a case where the basic optical zoom ratio of the camera module 1100b is Z, when the m optical lenses included in the OPFE 1110 move, the optical zoom ratio of the camera module 1100b may be changed to 3Z, 5Z, or an optical zoom ratio greater than 5Z.

[0104] The actuator 1130 may move the OPFE 1110 or the optical lens to a specific position. For example, the actuator 1130 may adjust the position of the optical lens so that the image sensor 1142 is placed at the focal length of the optical lens for accurate sensing.

[0105] The image sensing device 1140 may include an image sensor 1142, a control logic 1144, and a memory 1146. The image sensor 1142 may sense an image of a sensed object by using a light L provided through an optical lens. According to the above-described embodiment, the image sensor 1142 may include a pixel array, and a color pattern of each of a plurality of color pixels included in the pixel array may be based on a pattern of each of the CFA unit, the CFA block, and the sub-block in the above-described embodiment.

[0106] The control logic 1144 may control all operations of the camera module 1100b. For example, the control logic 1144 may control the operations of the camera module 1100b based on a control signal provided through the control signal line CSLb.

[0107] The memory 1146 may store information required for the operation of the camera module 1100b, such as calibration data 1147. The calibration data 1147 may include information required for the camera module 1100b to generate image data by using the externally provided light L. For example, the calibration data 1147 may include information about the above-mentioned rotation angle, information about the focal length, and information about the optical axis. When the camera module 1100b is implemented as a multi-state camera (the focal length of the multi-state camera varies based on the position of the optical lens of the multi-state camera), the calibration data 1147 may include a focal length value of the optical lens based on the position and information related to autofocus.

[0108] The storage 1150 may store image data sensed by the image sensor 1142. The storage 1150 may be provided outside the image sensing device 1140, and may be implemented as a type stacked above or below a sensor chip configuring the image sensing device 1140. In some embodiments, the storage 1150 may be implemented in an electrically erasable programmable read-only memory (EEPROM), but the embodiment is not limited thereto.

[0109] Reference Fig.14 and Fig.15 In some embodiments, each of the plurality of camera modules 1100a, 1100b, and 1100c may include an actuator 1130. Therefore, each of the plurality of camera modules 1100a, 1100b, and 1100c may include the same or different calibration data 1147 based on the operation of the actuator 1130 included therein.

[0110] In some embodiments, one of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., camera module 1100b) may include a folded lens type camera module including the above-described prism 1105 and OPFE 1110, and the other camera modules (e.g., camera modules 1100a and 1100c) may include a vertical camera module that does not include the prism 1105 and OPFE 1110. However, embodiments are not limited thereto.

[0111] In some embodiments, one of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., camera module 1100c) may include a vertical depth camera that extracts depth information by using infrared (IR). In this case, the application processor 1200 may combine image data provided from the depth camera with image data provided from another camera module (e.g., camera module 1100a or 1100b) to generate a three-dimensional (3D) depth image.

[0112] In some embodiments, at least two of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., camera modules 1100a and 1100b) may have different fields of view. In this case, for example, optical lenses of the at least two of the plurality of camera modules 1100a, 1100b, and 1100c (e.g., camera modules 1100a and 1100b) may be different, but the embodiment is not limited thereto.

[0113] In addition, in some embodiments, the fields of view of the plurality of camera modules 1100a, 1100b, and 1100c may be different. In this case, the optical lenses included in the plurality of camera modules 1100a, 1100b, and 1100c may be different, but the embodiment is not limited thereto.

[0114] In some embodiments, the plurality of camera modules 1100a, 1100b, and 1100c may be arranged physically separated from each other. That is, the sensing area of ​​one image sensor 1142 may not be divided and may not be used by the plurality of camera modules 1100a, 1100b, and 1100c, and an independent image sensor 1142 may be provided in each of the plurality of camera modules 1100a, 1100b, and 1100c.

[0115] Refer to Fig.14, the application processor 1200 may include an image processing device 1210, a memory controller 1220, and an internal memory 1230. The application processor 1200 may be implemented independently with respect to the plurality of camera modules 1100a, 1100b, and 1100c. For example, the application processor 1200 and the plurality of camera modules 1100a, 1100b, and 1100c may be implemented as separate semiconductor chips separated from each other.

[0116] The image processing device 1210 may include a plurality of sub-processors 1212 a , 1212 b , and 1212 c , an image generator 1214 , and a camera module controller 1216 .

[0117] The image processing device 1210 may include a plurality of sub-image processors 1212a, 1212b, and 1212c corresponding to the number of the camera modules 1100a, 1100b, and 1100c.

[0118] A plurality of pieces of image data generated from the camera modules 1100a, 1100b, and 1100c may be provided to the sub-image processors 1212a, 1212b, and 1212c, respectively, through a plurality of image signal lines ISLa, ISLb, and ISLc separated from each other. For example, the image data generated from the camera module 1100a may be provided to the sub-image processor 1212a through the image signal line ISLa, the image data generated from the camera module 1100b may be provided to the sub-image processor 1212b through the image signal line ISLb, and the image data generated from the camera module 1100c may be provided to the sub-image processor 1212c through the image signal line ISLc. Such image data transmission may be performed by using, for example, a camera serial interface (CSI) based on a mobile industry processor interface (MIPI), but the embodiment is not limited thereto.

[0119] In some embodiments, one sub-image processor may be provided to correspond to a plurality of camera modules. For example, the sub-image processor 1212a and the sub-image processor 1212c may not be implemented separately from each other as shown in the figure, but may be integrated into one sub-image processor, and image data provided from each of the camera modules 1100a and 1100c may be selected by a selection element (e.g., a multiplexer), and the image data provided from each of the camera modules 1100a and 1100c may be provided to the integrated sub-image processor.

[0120] The image data provided to each of the sub image processors 1212a, 1212b, and 1212c may be provided to the image generator 1214. The image generator 1214 may generate an output image from the plurality of pieces of image data provided from the sub image processors 1212a, 1212b, and 1212c based on a mode signal or generation information.

[0121] In detail, the image generator 1214 may combine at least some of the multiple pieces of image data generated from the camera modules 1100a, 1100b, and 1100c having different fields of view based on the mode signal or the generation information to generate an output image. In addition, the image generator 1214 may select one piece of image data from the multiple pieces of image data generated from the camera modules 1100a, 1100b, and 1100c having different fields of view based on the mode signal or the generation information to generate an output image.

[0122] In some embodiments, the generated information may include a zoom signal or a zoom factor. Additionally, in some embodiments, the mode signal may be a signal based on a mode selected by a user, for example.

[0123] When the generation information includes a zoom signal (zoom factor) and the camera modules 1100a, 1100b, and 1100c have different fields of view, the image generator 1214 may perform different operations based on the type of the zoom signal. For example, when the zoom signal is a first signal, the image generator 1214 may merge the image data output from the camera module 1100a and the image data output from the camera module 1100c, and then, an output image may be generated by using the merged image signal and the image data that is not used for merging and is output from the camera module 1100b. For example, when the zoom signal is a second signal different from the first signal, the image generator 1214 may not perform merging of multiple image data, and may select one image data from the multiple image data outputs provided by the camera modules 1100a, 1100b, and 1100c to generate an output image. However, the embodiment is not limited thereto, and according to the situation, the method of processing image data may be modified differently.

[0124] In some embodiments, the image generator 1214 may receive multiple image data with different exposure times from at least one of the multiple sub-image processors 1212a, 1212b and 1212c, and may perform high dynamic range (HDR) processing on the multiple image data to generate merged image data with increased dynamic range.

[0125] The camera module controller 1216 may provide a control signal to each of the camera modules 1100a, 1100b, and 1100c. The control signal generated by the camera module controller 1216 may be provided to the camera modules 1100a, 1100b, and 1100c through control signal lines CSLa, CSLb, and CSLc separated from each other.

[0126] One of the plurality of camera modules 1100a, 1100b, and 1100c may be designated as a master camera (e.g., camera module 1100b) based on a mode signal or generation information including a zoom signal, and the other camera modules (e.g., camera modules 1100a and 1100c) may be designated as slave cameras. Such information may be included in a control signal and may be provided to the camera modules 1100a, 1100b, and 1100c through control signal lines CSLa, CSLb, and CSLc separated from each other.

[0127] Whether the camera module operates as a master camera or a slave camera may be changed based on a zoom factor or an operation mode signal. For example, when the field of view of the camera module 1100a is wider than that of the camera module 1100b and the zoom factor indicates a low zoom ratio, the camera module 1100b may operate as a master camera, and the camera module 1100a may operate as a slave camera. On the other hand, when the zoom factor indicates a high zoom ratio, the camera module 1100a may operate as a master camera, and the camera module 1100b may operate as a slave camera.

[0128] In some embodiments, the control signal provided from the camera module controller 1216 to each of the camera modules 1100a, 1100b, and 1100c may include a synchronization enable signal. For example, when the camera module 1100b is a master camera and the camera modules 1100a and 1100c are slave cameras, the camera module controller 1216 may send a synchronization enable signal to the camera module 1100b. The camera module 1100b provided with the synchronization enable signal may generate a synchronization signal based on the synchronization enable signal, and may provide the generated synchronization signal to the camera modules 1100a and 1100c through the synchronization signal line SSL. The camera module 1100b and the camera modules 1100a and 1100c may send image data to the application processor 1200 in synchronization with the synchronization signal.

[0129] In some embodiments, the control signal provided from the camera module controller 1216 to the plurality of camera modules 1100a, 1100b, and 1100c may include mode information based on the mode signal. Based on the mode information, the plurality of camera modules 1100a, 1100b, and 1100c may operate as a first operation mode and a second operation mode associated with sensing speed.

[0130] In the first operation mode, the plurality of camera modules 1100a, 1100b, and 1100c may generate image signals at a first speed (e.g., generate image signals having a first frame rate), encode the image signals at a second speed higher than the first speed (e.g., encode the image signals to have a second frame rate higher than the first frame rate), and transmit the encoded image signals to the application processor 1200. In this case, the second speed may be equal to or less than 30 times the first speed.

[0131] The application processor 1200 may store the transmitted image signal (i.e., the encoded image signal) in the internal memory 1230 included therein or in the external memory 1400 provided outside the application processor 1200, and then may read the encoded image signal from the internal memory 1230 or the external memory 1400 and decode the encoded image signal, and may display image data generated based on the decoded image signal. For example, a corresponding sub-processor among the plurality of sub-processors 1212a, 1212b, and 1212c included in the image processing device 1210 may perform decoding, and may perform image processing on the decoded image signal.

[0132] In the second operation mode, the plurality of camera modules 1100a, 1100b, and 1100c may generate image signals at a third speed lower than the first speed (e.g., generate image signals having a third frame rate lower than the first frame rate) and may transmit the generated image signals to the application processor 1200. The image signals provided to the application processor 1200 may be unencoded signals. The application processor 1200 may perform image processing on the transmitted image signals, or may store the image-processed image signals in the internal memory 1230 or the external memory 1400.

[0133] The PMIC 1300 may supply power (e.g., source voltage) to each of the plurality of camera modules 1100a, 1100b, and 1100c. For example, based on the control of the application processor 1200, the PMIC 1300 may supply a first power to the camera module 1100a through a power signal line PSLa, a second power to the camera module 1100b through a power signal line PSLb, and a third power to the camera module 1100c through a power signal line PSLc.

[0134] In response to the power control signal PCON from the application processor 1200, the PMIC 1300 may generate power corresponding to each of the plurality of camera modules 1100a, 1100b, and 1100c, and may adjust the level of the power. The power control signal PCON may include a power adjustment signal for the operating mode of each of the plurality of camera modules 1100a, 1100b, and 1100c. For example, the operating mode may include a low power mode, in which case the power control signal PCON may include information about the camera module operating based on the low power mode and information about setting the power level. The levels of the plurality of powers provided to the plurality of camera modules 1100a, 1100b, and 1100c, respectively, may be the same or different. In addition, the levels of the plurality of powers may change dynamically.

[0135] While the inventive concept has been particularly shown and described with reference to certain illustrative embodiments, it will be understood that various changes in form and details may be made in the inventive concept without departing from the spirit and scope of the appended claims.

Claims

1. A pixel array, comprising: a color filter array unit, each color filter array unit including a color filter array block, each color filter array block including a sub-block and color pixels of the same color arranged in a boundary area of ​​the color filter array block to surround the sub-block and contact the sub-block, The sub-blocks include color pixels arranged in 2×2, the 2×2 color pixels include a first color pixel sensing a first color, a second color pixel sensing a second color different from the first color, and two third color pixels sensing a third color different from the first color and the second color, and the sub-blocks in each of the color filter array blocks have the same color pattern.

2. The pixel array according to claim 1, wherein: The first color is red, the second color is blue, and the third color is green.

3. The pixel array according to claim 2, wherein: The sub-block includes a green pixel disposed at an upper left position, a blue pixel disposed at an upper right position, a red pixel disposed at a lower left position, and a green pixel disposed at a lower right position.

4. The pixel array according to claim 2, wherein: The sub-block includes a green pixel disposed at an upper left position, a red pixel disposed at an upper right position, a blue pixel disposed at a lower left position, and a green pixel disposed at a lower right position.

5. The pixel array according to claim 2, wherein: The sub-block includes a blue pixel disposed at an upper left position, a green pixel disposed at an upper right position, a green pixel disposed at a lower left position, and a red pixel disposed at a lower right position.

6. The pixel array according to claim 2, wherein: The sub-block includes a red pixel disposed at an upper left position, a green pixel disposed at an upper right position, a green pixel disposed at a lower left position, and a blue pixel disposed at a lower right position.

7. The pixel array according to claim 1, wherein: The color filter array block includes: a first color filter array block and a second color filter array block arranged in a first row direction, a third color filter array block and a fourth color filter array block arranged in a second row direction, the first color filter array block and the third color filter array block are arranged in a first column direction, and the second color filter array block and the fourth color filter array block are arranged in a second column direction.

8. The pixel array according to claim 7, wherein: The first color is red, the second color is blue, and the third color is green.

9. The pixel array according to claim 8, wherein: One of the color filter array blocks is a red color filter array block including red pixels, the number of which is dominant in the red color filter array block, another of the color filter array blocks is a blue color filter array block including blue pixels, the number of which is dominant in the blue color filter array block, and the other two of the color filter array blocks are green color filter array blocks including green pixels, the number of which is dominant in each green color filter array block.

10. The pixel array according to claim 9, wherein: The red filter array block includes red pixels in a boundary area of ​​the red filter array block, the number of the red pixels dominates in the boundary area of ​​the red filter array block, the blue filter array block includes blue pixels in a boundary area of ​​the blue filter array block, the number of the blue pixels dominates in the boundary area of ​​the blue filter array block, and the green filter array blocks include green pixels in the boundary areas of the green filter array blocks, respectively, the number of the green pixels dominates in the boundary area of ​​the green filter array block.

11. An image sensor, comprising: a pixel array comprising a color filter array unit, each color filter array unit comprising a color filter array block, each color filter array block comprising a sub-block and color pixels of the same color arranged in a boundary area of ​​the color filter array block to surround the sub-block and contact the sub-block, wherein the sub-block comprises color pixels arranged in 2×2, the 2×2 arranged color pixels comprising one first color pixel sensing a first color, one second color pixel sensing a second color different from the first color, and two third color pixels sensing a third color different from the first color and the second color, and the sub-blocks in each of the color filter array blocks have the same color pattern; a row driver configured to generate control signals that control generation of pixel signals for color pixels of the pixel array; a read circuit configured to generate pixel data according to the pixel signal; and A controller is configured to control the row driver and the read circuit.

12. The image sensor according to claim 11, further comprising: A re-stitching processor is configured to perform a re-stitching operation on the pixel data to convert a color pattern of at least one of the color filter array elements into a Bayer pattern.

13. The image sensor according to claim 12, wherein: The color pattern of the sub-block is the same as the color pattern of at least a portion of the Bayer pattern.

14. The image sensor according to claim 11, wherein: The color filter array block includes: a first color filter array block and a second color filter array block arranged in a first row direction, a third color filter array block and a fourth color filter array block arranged in a second row direction, the first color filter array block and the third color filter array block are arranged in a first column direction, and the second color filter array block and the fourth color filter array block are arranged in a second column direction.

15. The image sensor according to claim 14, wherein: One of the color filter array blocks is a red color filter array block including red pixels, the number of which is dominant in the red color filter array block, another of the color filter array blocks is a blue color filter array block including blue pixels, the number of which is dominant in the blue color filter array block, and the other two of the color filter array blocks are green color filter array blocks including green pixels, the number of which is dominant in each green color filter array block.

16. An image sensor pixel array, comprising: a plurality of color filter array units, each color filter array unit comprising a 2×2 arrangement of color filter array blocks, Wherein, each color filter array block includes color pixels with the same color, and the color pixels with the same color are arranged in a boundary area of ​​the color filter array block that surrounds a sub-block and contacts the sub-block, wherein the sub-block includes color pixels arranged in 2×2, and the 2×2 arranged color pixels include a first color pixel that senses a first color, a second color pixel that senses a second color different from the first color, and two third color pixels that sense a third color different from the first color and the second color, and the sub-blocks included in each of the color filter array blocks are the same.

17. The image sensor pixel array of claim 16, wherein: The 2×2 arranged color filter array blocks include a green color filter array block, a red color filter array block, and a blue color filter array block, the green color filter array block includes twelve green pixels surrounding the sub-block, the red color filter array block includes twelve red pixels surrounding the sub-block, and the blue color filter array block includes twelve blue pixels surrounding the sub-block.

Images