Image demosaicing method

Abstract

The image demosaicing method comprises the following steps: determining a red image pixel value, a green image pixel value and a blue image pixel value for a red pixel of a Bayer unit, such that each of the first weighted sum, the second weighted sum, and the third weighted sum of the red image pixel value, the green image pixel value, and the blue image pixel value is equal to the respective sensor pixel value. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to an average value of green sensor pixel values output by green pixels of the Bayer unit. The third weighted sum is equal to a blue sensor pixel value output by a blue pixel of the Bayer cell. The method further includes outputting a demosaiced image having an RGB triple at a position in the demosaiced image corresponding to a position of the red pixel, the RGB triple including a red image pixel value, a green image pixel value, and a blue image pixel value.

Description

Technical Field

[0001] This disclosure relates to interpolation, and more particularly to crosstalk-based interpolation. Background Technology

[0002] Color image sensors are ubiquitous. Color image sensors (including CMOS and CCD image sensors) consist of pixel arrays with multiple color pixels. In many cases, color pixels include R (red) color pixels, G (green) color pixels, and B (blue) color pixels. It is generally assumed that R color pixels detect only R color signals, G color pixels detect only G color signals, and B color pixels detect only B color signals. B color signals can be light integrated into the B band, for example, at wavelengths of 400 nanometers (nm) to 500 nm. G color signals can be light integrated into the G band, for example, at wavelengths of 500 nm to 600 nm. R color signals can be light integrated into the R band, for example, at wavelengths of 600 nm to 700 nm.

[0003] To detect individual color signals, color pixels are covered by color filters. An R color filter covers R color pixels. The R color filter blocks all light except for light in the R band (e.g., 600nm to 700nm wavelength). A G color filter covers G color pixels. The G color filter blocks all light except for light in the G band (e.g., 500nm to 600nm wavelength). A B color filter covers B color pixels. The B color filter blocks all light except for light in the B band (e.g., 400nm to 500nm wavelength).

[0004] A Bayer pattern unit consists of 2×2 color pixels: one R pixel, one B pixel, and two G pixels diagonally. For simplicity, an R color pixel can be represented as an R pixel, an R color filter as an R filter, and an R color signal as an R signal. A G color pixel can be represented as a G pixel, a G color filter as a G filter, and a G color signal as a G signal. A B color pixel can be represented as a B pixel, a B color filter as a B filter, and a B color signal as a B signal. A Bayer pattern unit can be simply represented as a Bayer unit. In a Bayer unit, the R pixel can be assumed to detect only the R signal and not the G or B signals. The G pixel in a Bayer unit can be assumed to detect only the G signal and not the B or R signals. The B pixel in a Bayer unit can be assumed to detect only the B signal and not the R or G signals.

[0005] When extracting an R image, the R image contains missing signals because it can be assumed that there are no R signals at the locations of G and B pixels. The missing color signals can be recovered using interpolation. For example, the missing signals can be approximated by averaging the signals of adjacent signals of the same color. For instance, the missing G signal at an R or B pixel can be interpolated by calculating the average of the G signals from the upper, lower, left, and right G pixels.

[0006] However, in practice, an R filter may also allow light in the G and B bands to pass through, a G filter may also allow light in the B and R bands to pass through, and a B filter may also allow light in the G and R bands to pass through, resulting in color crosstalk in the pixel. Therefore, in practice, a crosstalk-based method is needed to estimate the amount of light in the R, G, and B bands in a color pixel. Related methods for interpolation are also needed. Summary of the Invention

[0007] In a first aspect, a method for demosaicing an image captured by an image sensor is disclosed, the image sensor having a pixel array including a first Bayer unit. The method includes: for a red pixel of the first Bayer unit, determining a first red image pixel value, a first green image pixel value, and a first blue image pixel value via circuitry communicatively coupled to the pixel array, such that each weighted sum of a first weighted sum, a second weighted sum, and a third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the corresponding sensor pixel value output by the pixel of the first Bayer unit. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to the average of the first green sensor pixel value output by the first green pixel of the first Bayer unit and the second green sensor pixel value output by the second green pixel of the first Bayer unit. The third weighted sum is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The method further includes: outputting a demosaic image via a circuit, wherein the demosaic image has a first RGB triplet at a position corresponding to the position of a red pixel in the pixel array, the first RGB triplet including a first red image pixel value, a first green image pixel value, and a first blue image pixel value.

[0008] In a second aspect, a method for demosaicing an image captured by an image sensor is disclosed, the image sensor having a pixel array including a first Bayer unit. The method includes: for a red pixel of the first Bayer unit, determining a first red image pixel value, a first green image pixel value, a first blue image pixel value, and a first infrared (IR) image pixel value via circuitry communicatively coupled to the pixel array, such that each weighted sum of a first weighted sum, a second weighted sum, a third weighted sum, and a fourth weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the corresponding sensor pixel value output by the pixel of the first Bayer unit. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to the green sensor pixel value output by the green pixel of the first Bayer unit. The third weighted sum is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The fourth weighted sum is equal to the IR sensor pixel value output by the IR pixel of the first Bayer unit. The method further includes: outputting a demosaic image via a circuit, wherein the demosaic image has a first RGB quadruple at a position corresponding to the position of a red pixel in the pixel array, the first RGB quadruple including a first red image pixel value, a first green image pixel value, a first blue image pixel value and a first IR image pixel value.

[0009] In a third aspect, a method for demosaicing an image captured by an image sensor is disclosed. The image sensor has a pixel array comprising a plurality of pixels, each pixel group comprising a plurality of pixels of number P. The method includes: for a first pixel of the pixel group, determining P image pixel values ​​via a circuit communicatively coupled to the pixel array, such that for each of the plurality of pixels, a weighted sum of the corresponding P image pixel values ​​equals a sensor pixel value output by the pixel. The method further includes: outputting a demosaic image via a circuit, the demosaic image having an image pixel value array at a position corresponding to the position of the first pixel in the pixel array, the image pixel value array comprising each of the P image pixel values. Attached Figure Description

[0010] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following drawings, wherein the same reference numerals denote the same parts in the various views, unless otherwise specified.

[0011] Figure 1A The assumed spectral transmittance of the R filter is shown.

[0012] Figure 1B The assumed spectral transmittance of the G filter is shown.

[0013] Figure 1C The assumed spectral transmittance of filter B is shown.

[0014] Figure 2 The typical spectral transmittance of a color filter is shown.

[0015] Figure 3 A color pixel array of an image sensor with a Bayer array according to an embodiment of the present invention is shown.

[0016] Figure 4 An example of interpolation based on solving three linear equations according to an embodiment of the present invention is shown.

[0017] Figure 5 Typical spectral transmittance of color filters covering the B band (400 nm to 500 nm), G band (500 nm to 600 nm), R band (600 nm to 700 nm), and IR band (700 nm to 1100 nm) is shown.

[0018] Figure 6 A color pixel array comprising R, G, B and IR pixels is shown in an image sensor according to an embodiment of the present invention.

[0019] Figure 7 A color pixel array of an image sensor according to an embodiment of the present invention is shown.

[0020] Figure 8 This is a functional block diagram of an image sensor, and embodiments of this image sensor include... Figure 6 or Figure 7 A color pixel array.

[0021] Figures 9 to 12 These are respectively shown for use with image sensors (such as...) Figure 8 A flowchart illustrating an embodiment of a method for demosaicing images captured by an image sensor.

[0022] The corresponding reference characters indicate the components in each view. Those skilled in the art will understand that the schematic diagrams of the elements in the figures are drawn for simplicity and clarity and are not strictly to scale. For example, the dimensions of some elements in the figures may be exaggerated relative to other elements to aid in a better understanding of the various embodiments of the invention. Detailed Implementation

[0023] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, it will be apparent to those skilled in the art that these specific details are unnecessary for practicing the invention. In other instances, well-known materials or methods have not been described in detail to avoid obscuring the invention.

[0024] When this specification refers to "an embodiment," it means that a specific feature, structure, or characteristic associated with that embodiment is included in at least one embodiment of the invention. Therefore, phrases such as "in one embodiment" appearing in different locations within this specification do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics can be combined in any suitable combination and / or sub-combination in one or more embodiments.

[0025] We can assume that the spectral transmittance of R filter 102 is as follows: Figure 1A As shown, the spectral transmittance of filter G 104 is as follows: Figure 1B As described above, and the spectral transmittance of filter B 106 as follows Figure 1C The spectral transmittance of R filter 102 includes a rectangular window from λ = 600 nm to λ = 700 nm, which has 100% spectral transmittance. The spectral transmittance of G filter 104 includes a rectangular window from λ = 500 nm to λ = 600 nm, which has 100% spectral transmittance. The spectral transmittance of B filter 106 includes a rectangular window from λ = 400 nm to λ = 500 nm, which has 100% spectral transmittance.

[0026] Therefore, the spectral transmittance of R filter 102, G filter 104, and B filter 106 do not overlap. Therefore, it can be assumed that the R pixel detects only R signals, the G pixel detects only G signals, and the B pixel detects only B signals.

[0027] Since spectral transmittance depends on the color filter material, such as Figures 1A to 1C The spectral transmittance of R filter 102, G filter 104 and B filter 106 shown may not be realistic. Figure 2 Examples of typical spectral transmittance curves 202, 204, and 206 are shown, corresponding to filters 102, 104, and 106, respectively. Curve 202 shows the spectral transmittance of filter R. Curve 204 shows the spectral transmittance of filter G. Curve 206 shows the spectral transmittance of filter B.

[0028] Although the B-band filter is designed to filter light in the B-band (400nm to 500nm), light in the G-band (500nm to 600nm) and R-band (600nm to 700nm) also passes through it. This B-band filter has a spectral transmittance curve 206 and is collected by the B-pixels. That is, curve 206 has non-zero spectral transmittance outside the B-band. Similarly, curves 204 and 202 have non-zero spectral transmittance outside the G-band and R-band, respectively.

[0029] The precise amounts of light detected in the B-band (400nm to 500nm), G-band (500nm to 600nm), and R-band (600nm to 700nm) light within the B-pixels covered by the B-filter can be calculated based on the spectral transmittance of the B-filter (curve 206). For example, the amount of detected light is equivalent to the area of ​​the spectral transmittance curve within the target color band. The spectral transmittance curve of the color filter is typically provided by the color filter manufacturer, or it can be measured.

[0030] Alternatively, the light in the B-band detected in the B-pixel can be approximated by point 211, which is about 470 nm below the peak of curve 206; the light in the G-band detected in the B-pixel can be approximated by point 212, which is about 525 nm below the peak of curve 204; and the light in the R-band detected in the B-pixel can be approximated by point 213, which is about 630 nm below the peak of curve 202.

[0031] It is also possible that point 211 is located at 450nm (center of B-band), point 212 at 550nm (center of G-band), and point 213 at 650nm (center of R-band). Other considerations and choices are also possible.

[0032] In an embodiment, for example, curve 206 shows that approximately 48% of the light in the B-band is detected in the B pixel, approximately 10% of the light in the G-band is detected in the B pixel, and approximately 3% of the light in the R-band is detected in the B pixel. The B pixel is located below the B filter.

[0033] Using this approximation, the light intensity detected at the B pixel covered by the B filter, IBpixel, can be determined and expressed as:

[0034] IBpixel=48% B+10% G+3% R. (1)

[0035] Where B is the B signal, which is the light in the B band before passing through the B filter; G is the G signal, which is the light in the G band before passing through the B filter; and R is the R signal, which is the light in the R band before passing through the B filter.

[0036] Equation (1) can be generalized and expressed as:

[0037] IBpixel=a31 R+a32 G+a33 B. (2)

[0038] In the same way, curve 202 will provide

[0039] IRpixel=a11 R+a12 G+a13 B, (3)

[0040] And curve 204 will provide

[0041] IGpixel=a21 R+a22 G+a23 B. (4)

[0042] Where IRpixel is the light intensity detected at the R pixel covered by the R filter, and IGpixel is the light intensity detected at the G pixel covered by the G filter.

[0043] Therefore, we will obtain a set of three linear equations:

[0044] IRpixel=a11 R+a12 G+a13 B, (5)

[0045] IGpixel=a21 R+a22 G+a23 B, (6)

[0046] IBpixel=a31 R+a32 G+a33 B. (7)

[0047] Among them, IRpixel, IGpixel, and IBpixel are known from the light detection at three pixels (R pixel, G pixel, B pixel), while a11, a12, a13, a21, a22, a23, a31, a32, and a33 are from curves 202, 204, and 206. Note that equation (5) is the same as equation (3), equation (6) is the same as equation (4), and equation (7) is the same as equation (2).

[0048] In the set of three linear equations (5) to (7), R, G and B are variables, a11, a12, a13, a21, a22, a23, a31, a32 and a33 are coefficients, and IRpixel, IGpixel and IBpixel are constants.

[0049] Assuming that the R signals (R) in equations (5) to (7) are the same, the G signals (G) in equations (5) to (7) are the same, and the B signals (B) in equations (5) to (7) are the same, the values ​​of R, G, and B (variables R, G, and B) can be determined by solving equations (5) to (7). To assume or ensure that the R, G, and B signals in equations (5) to (7) are the same, according to an embodiment of the invention, adjacent R, G, and B pixels associated with equations (5) to (7) can be selected, such as... Figure 3 As shown.

[0050] The R signal, G signal, and B signal at at least one of the R pixel, G pixel, or B pixel can be determined by solving equations (5) to (7) associated with that R pixel, G pixel, or B pixel. The R pixel, G pixel, and B pixel are adjacent pixels.

[0051] Figure 3 A color pixel array 300 of an image sensor (not shown) with a Bayer array according to an embodiment of the present invention is illustrated. The color pixel array 300 includes a plurality of R pixels, a plurality of G pixels, and a plurality of B pixels. A group of four color pixels forms a Bayer unit 302. The Bayer unit 302 includes an R pixel 304, a G pixel 306, a B pixel 308, and a G pixel 310. It should be understood that the color pixel array is not limited to a Bayer array. Other configurations are possible as long as the four color pixels are adjacent or closely grouped together.

[0052] In this embodiment, R pixel 304, G pixel 306, B pixel 308, and G pixel 310 form a square, namely the Bayer unit 302. R pixel 304 is located at the upper left corner, G pixel 306 at the upper right corner, B pixel 308 at the lower right corner, and G pixel 310 at the lower left corner.

[0053] The light detected at pixel R 304 can be represented by equation (5), the light detected at pixel G 306 by equation (6), and the light detected at pixel B 308 by equation (7). The values ​​of R, G, and B can be obtained by solving the three linear equations (5) to (7). In this embodiment, all pixels R 304, G 306, and B 308 can have the same values ​​of R, G, and B.

[0054] In the embodiment, the light detected at G pixel 306 can be represented by equation (8), and the light detected at G pixel 310 can be represented by equation (9).

[0055] IGpixel1=a21 R+a22 G+a23 B, (8)

[0056] IGpixel2=a21 R+a22 G+a23 B, (9)

[0057] Where IGpixel1 is the light intensity detected at G pixel 306 covered by the G filter, and IGpixel2 is the intensity detected at G pixel 310 covered by the G filter, and a21, a22 and a23 in equations (8) and (9) are the same, because the G filters at G pixels 306 and 310 are the same.

[0058] To solve for three unknowns or variables, a set of three linear equations (no more, no less) with three unknowns or variables may be needed. Therefore, the average of equations (8) and (9) is taken.

[0059] Equation (10) is the average of equations (8) and (9):

[0060] (IGpixel1+IGpixel2) / 2=a21 R+a22 G+a23 B. (10)

[0061] Therefore, a new set of three linear equations can be obtained.

[0062] IRpixel=a11 R+a12 G+a13 B, (11)

[0063] (IGpixel1+IGpixel2) / 2=a21 R+a22 G+a23 B, (12)

[0064] IBpixel=a31 R+a32 G+a33 B, (13)

[0065] Equation (11) is the same as equation (6), equation (12) is the same as equation (10), and equation (13) is the same as equation (7). Similarly, a11, a12, a13, a21, a22, a23, a31, a32, and a33 are known from curves 202, 204, and 206.

[0066] In the set of three linear equations (11) to (13), R, G and B are variables, a11, a12, a13, a21, a22, a23, a31, a32 and a33 are coefficients, and IRpixel, (IGpixel1+IGpixel2) / 2 and IBpixel are constants.

[0067] In this embodiment, all R pixels 304, G pixels 306 and 310, and B pixels 308 can have the same values ​​for R, G, and B. In other words, the Bayer unit has R, G, and B values.

[0068] Furthermore, the R, G, and B signals at at least one of the R, G, or B pixels in a Bayer cell can be determined by solving equations (11) to (13) associated with that R, G, or B pixel. The R, G, and B pixels are adjacent pixels, even if they are not in the same Bayer cell.

[0069] To improve the resolution of the R, G, and B signals, an interpolation-like process can be performed. For simplicity, this interpolation-like process can be considered as interpolation. Without interpolation, each Bayer unit has R, G, and B values. In other words, R, G, and B pixels within the same Bayer unit have the same R, G, and B values. With interpolation, R, G, and B pixels within the same Bayer unit have different R, G, and B values, thereby improving the resolution of the R, G, and B signals.

[0070] Figure 4 An interpolation based on solving three linear equations (11) to (13) according to an embodiment of the invention is shown. Figure 4 A color pixel array 400 of an image sensor (not shown) is illustrated, comprising a plurality of R pixels, a plurality of G pixels, and a plurality of B pixels. The color pixel array 400 includes: R pixels 402, G pixels 404, R pixels 406, G pixels 408… in the first row 432 of the pixel array 400; G pixels 412, B pixels 414, G pixels 416, B pixels 418… in the second row 434 of the pixel array 400; R pixels 422, G pixels 424, R pixels 426, G pixels 428… in the third row 436 of the pixel array 400; and so on.

[0071] In one embodiment, R pixel 402, G pixel 404, B pixel 414, and G pixel 412 form a square, which is a Bayer unit or group 442. R pixel 402 is located at the top left corner, G pixel 404 at the top right corner, B pixel 414 at the bottom right corner, and G pixel 412 at the bottom left corner. G pixel 404, R pixel 406, G pixel 416, and B pixel 414 form a square, which is a Bayer unit or group 444. G pixel 404 is located at the top left corner, R pixel 406 at the top right corner, G pixel 416 at the bottom right corner, and R pixel 414 at the bottom left corner.

[0072] G pixel 412, B pixel 414, G pixel 424, and R pixel 422 form a square, which is a Bayer unit or group 446. G pixel 412 is located in the upper left corner, B pixel 414 in the upper right corner, G pixel 424 in the lower right corner, and R pixel 422 in the lower left corner. B pixel 414, G pixel 416, R pixel 426, and G pixel 424 form a square, which is a Bayer unit or group 448. B pixel 414 is located in the upper left corner, G pixel 416 in the upper right corner, R pixel 426 in the lower right corner, and G pixel 424 in the lower left corner.

[0073] In the first step of line 432, the R, G, and B values ​​or signals at R pixel 402 are determined by solving three linear equations (11) to (13) for the pixels associated with R pixel 402 (these pixels are R pixel 402, G pixel 404, G pixel 412, and B pixel 414 in Bayer units or groups 442). IGpixel1 and IGpixel2 in equation (12) are the light intensities detected at G pixel 404 and G pixel 412, respectively. Equation (11) is associated with R pixel 402. Equation (12) is associated with G pixel 404 and G pixel 412. Equation (13) is associated with B pixel 414.

[0074] In the second step of line 432, by replacing IGpixel2 with IGpixel3, equation (12) is changed to equation (12a), where IGpixel3 is the light intensity detected at G pixel 416.

[0075] (IGpixel1+IGpixel3) / 2=a21 R+a22 G+a23 B. (12a)

[0076] Furthermore, in the second step of line 432, the R, G, and B values ​​or signals at G pixel 404 are determined by solving three linear equations (11), (12a), and (13) associated with the pixels in the Bayer unit or group 444. This process continues. Equation (11) is associated with R pixel 406. Equation (12a) is associated with G pixels 404 and G pixels 416, where IGpixel1 is the light intensity detected at G pixel 404 and IGpixel3 is the light intensity detected at G pixel 416. Equation (13) is associated with B pixel 414.

[0077] In the first step of line 434, the R, G and B values ​​or signals at G pixel 412 are determined by solving three linear equations (11) to (13) for the pixels associated with G pixel 412 (these pixels are G pixel 412, B pixel 414, R pixel 422 and G pixel 424 in Bayer unit or group 446).

[0078] In the second step of line 434, the R, G, and B values ​​or signals at pixel B 414 are determined by solving three linear equations (11) to (13) for the pixels associated with pixel B 414 (these pixels are B pixel 414, G pixel 416, G pixel 424, and R pixel 426 in the Bayer unit or group 448). And so on.

[0079] Therefore, group 442 (similar to) Figure 3In the Bayer unit 302), each of the R pixel 402, G pixel 404, G pixel 412, and B pixel 414 has a different R, G, and B value or signal. In contrast, Figure 3 The R pixel 304, G pixel 306, G pixel 310 and B pixel 308 in the middle Bayer unit 302 have the same R, G and B values ​​or signals.

[0080] In an embodiment, Figure 3 In the middle Bayer unit 302, the G pixel 310 is replaced by an IR pixel. An IR pixel is a pixel covered by an IR filter and not covered by an R, G, or B filter.

[0081] Figure 5 Examples of typical spectral transmittance for a color filter 500 covering the B-band (400 nm to 500 nm), G-band (500 nm to 600 nm), R-band (600 nm to 700 nm), and IR-band (700 nm to 1100 nm) are shown. Curve 502 shows the spectral transmittance of the R-band filter. Curve 504 shows the spectral transmittance of the G-band filter. Curve 506 shows the spectral transmittance of the B-band filter. Curve 508 shows the spectral transmittance of the IR-band filter.

[0082] Although the B filter is designed to filter light in the B band (400nm to 500nm) only, light in the G band (500nm to 600nm), R band (600nm to 700nm), and IR band (700nm to 1100nm) also passes through the B filter, which has a spectral transmittance curve 506 and is collected by B pixels.

[0083] Figure 6 A color pixel array 600 of an image sensor (not shown) according to an embodiment of the present invention is illustrated. The color pixel array includes a plurality of R pixels, a plurality of G pixels, a plurality of B pixels, and a plurality of IR pixels. The color pixel array 600 includes: R pixels 602, G pixels 604, R pixels 606, G pixels 608… in a first row 632 of the pixel array 600; IR pixels 612, B pixels 614, IR pixels 616, B pixels 618… in a second row 634 of the pixel array 600; R pixels 622, G pixels 624, R pixels 626, G pixels 628… in a third row 636 of the pixel array 600; and so on.

[0084] Equations (5) to (7) need to be modified to include IR light detected in R, G, and B pixels. Equations (5) to (7) become equations (14) to (16).

[0085] IRpixel=a11 R+a12 G+a13 B+a14 IR, (14)

[0086] IGpixel=a21 R+a22 G+a23 B+a24 IR, (15)

[0087] IBpixel=a31 R+a32 G+a33 B+a34 IR, (16)

[0088] Among them, a14 IR, a24 IR and a34 IR are the IR light detected in R pixel, G pixel and B pixel, respectively.

[0089] In the same way, the light detected at the IR pixel is represented as:

[0090] IIRpixel=a41 R+a42 G+a43 B+a44 IR, (17)

[0091] Where IIRpixel is the light intensity detected at IR pixel 612 covered by the IR filter. Therefore, a new set of four linear equations (14) to (17) is obtained to determine the R, G, B and IR signals at the pixel (R, G, B or IR pixel).

[0092] In the set of four linear equations (14) to (17), R, G, B and IR are variables, a11, a12, a13, a14, a21, a22, a23, a24, a31, a32, a33, a34, a41, a42, a43 and a44 are coefficients, and IRpixel, IGpixel, IBpixel and IIRpixel are constants.

[0093] In the first step of line 632, the R, G, B, and IR values ​​or signals at pixel R 602 are determined by solving four linear equations (14) to (17) for the associated pixels (R pixel 602, G pixel 604, IR pixel 612, and B pixel 614 in group 642). Equation (14) is associated with R pixel 602. Equation (15) is associated with G pixel 604. Equation (16) is associated with B pixel 614. Equation (17) is associated with IR pixel 612.

[0094] In the second step of line 632, the R, G, B, and IR values ​​or signals at pixel G 604 are determined by solving four linear equations (14) to (17) for the associated pixels in pixel group 644. Equation (14) is associated with pixel R 606. Equation (15) is associated with pixel G 604. Equation (16) is associated with pixel B 614. Equation (17) is associated with pixel IR 616.

[0095] In the first step of line 634, the R, G, B and IR values ​​or signals at IR pixel 612 are determined by solving four linear equations (14) to (17) for the associated pixels (IR pixel 612, B pixel 614, R pixel 622 and G pixel 624 in group 646).

[0096] In the second step of line 634, the R, G, B, and IR values ​​or signals at pixel B 614 are determined by solving four linear equations (14) to (17) for the associated pixels (these pixels are B pixel 614, IR pixel 616, G pixel 624, and R pixel 626 in group 648). And so on.

[0097] Therefore, each of the R pixel 602, G pixel 604, IR pixel 612, and B pixel 614 in the Bayer unit or group 642 has different R, G, B, and IR values ​​or signals. In contrast, Figure 3 The R pixel 304, G pixel 306, G pixel 310 and B pixel 308 in the Bayer unit 302 have the same R, G and B values ​​or signals.

[0098] Furthermore, R, G, and B signals or R, G, B, and IR signals can be provided by a color filter array. The incident light is filtered by the color filters. In an embodiment, the R, G, and B signals or R, G, B, and IR signals can be provided by a metasurface-based color router. The color router changes the direction of the light according to the wavelength.

[0099] In one embodiment, four neighboring pixels have four or three different linear equations, each containing four or three unknowns or variables. These pixels are combined to solve for the four or three unknowns. The four unknowns can be R, G, B, and IR signals. The three unknowns can be R, G, and B signals.

[0100] Figure 7 An image sensor according to an embodiment of the present invention is shown, the image sensor having a color pixel array 700 comprising a plurality of color pixels 702. P neighboring color pixels among the plurality of color pixels form a color pixel group. P is an integer. The color pixel group can be arranged in a 1D (one-dimensional) array 704. The color pixel group can also be arranged in a 2D (two-dimensional) array 706. Furthermore, the color pixel group can be arranged in any shape with adjacent pixels. Each color pixel in the color pixel group is associated with a different linear equation comprising P unknowns or variables. The P neighboring pixels are associated with P different linear equations. The P different linear equations can be combined to solve for the P unknowns. The unknowns can be values ​​or signals in a specific color band.

[0101] Figure 8 This is a functional block diagram of the image sensor 800. Figure 8 The cross-section shown is parallel to the plane formed by orthogonal axes A1 and A2, each of which is orthogonal to axis A3. Image sensor 800 includes a two-dimensional pixel array 820A composed of pixel arrays 820. Examples of pixel arrays 820A include those corresponding to... Figure 4 , Figure 6 and Figure 7 The pixel arrays are 400, 600, and 700.

[0102] The pixel array 820A has M pixel rows 807 (1–M) and N pixel columns 808 (1–N), such as Figure 8 As described above, they are respectively labeled as pixel rows R1, R2, ..., R M And pixel columns C1, C2, ..., C N In some embodiments, the pixel array 820A may be wider along axis A1 than along axis A2, possibly because N is greater than M. The pixel array 820A may be wider along axis A2 than along axis A1, possibly because M is greater than N. The image sensor 800 may include a semiconductor wafer that includes the pixel array 820A.

[0103] The image sensor 800 may also include at least one of a readout circuit 841, a functional logic 842, and a control circuit 843. Figure 8 Circuitry 844 is also depicted, which may include a processor and / or memory. The processor itself may include memory that can store machine-readable instructions, such as software or firmware. The processor may be or include an image signal processor (ISP). Circuitry 844 may be part of image sensor 800 (e.g., as part of functional logic 842) or communicatively coupled to image sensor 800. Circuitry 844 may output a de-mosaic image 890.

[0104] After each pixel 820 acquires its image charge, the image charge is read out by the readout circuit 841 via the column bit line and transmitted to the functional logic 842. The image sensor 800 may also include a control circuit 843 coupled to the pixel array 820A for generating various signals to control the operation of each pixel 820.

[0105] Each pixel is represented as p (820 pixels). mn Where the indices m and n of the pixel coordinates (m, n) represent the row and column of the pixel in the pixel array 820A, respectively. For example, Figure 8 This represents the selected pixel 820 in pixel rows R1-R5 and pixel columns C1-C5. Each 2×2 pixel p mnThe array is composed of individual Bayer units. For example, pixel p 11 p 12 p 21 and p 22 The first Bayer unit is constituted, examples of which include Bayer units 442 and 642. Pixel p 12 p 13 p 22 and p 23 This constitutes the second Bayer unit, examples of which include Bayer units 444 and 644. Pixel p 21 p 22 p 31 and p 32 Examples of third Bayer units include Bayer units 446 and 646.

[0106] Figure 9 This is a flowchart illustrating a method 900 for demosaicing an image captured by an image sensor (800). The image sensor has a pixel array (820A) including a first Bayer unit (442). Descriptions of the methods herein include numbers in parentheses following terms described by the method. These methods include methods 900, 1000, 1100, and 1200. The numbers in parentheses indicate that the element associated with that number is an example of the term. For example, the above "image sensor (800)" means that image sensor 800 is an example of the image sensor of method 900. Method 900 includes at least one of steps 910, 920, and 930.

[0107] Step 910 includes determining the image pixel values ​​for the pixels of the first Bayer unit (442) of the pixel array. In step 910, the values ​​R, G, and B of equations (8) to (13) are examples of the first red image pixel value, the first green image pixel value, and the first blue image pixel value, respectively. In step 910, red pixels can be generalized to "non-green pixels," such as blue pixels. Therefore, red pixels can be replaced with blue pixels, such that the examples of the first Bayer unit are... Figure 4 Bayer Unit 448.

[0108] Step 910 may include at least one of steps 912 and 914. Step 912 includes: for the red pixel (402) of the first Bayer unit (442), determining a first red image pixel value, a first green image pixel value, and a first blue image pixel value via a circuit (844) communicatively coupled to the pixel array.

[0109] The result of step 912 is that each weighted sum of the first weighted sum, the second weighted sum, and the third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the corresponding sensor pixel value of the pixel output of the first Bayer unit. The first weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the red sensor pixel value output by the red pixel. The second weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the average of the first green sensor pixel value output by the first green pixel of the first Bayer unit and the second green sensor pixel value output by the second green pixel of the first Bayer unit. The third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The right sides of equations (11), (12), and (13) are examples of the first weighted sum, the second weighted sum, and the third weighted sum, respectively.

[0110] In this embodiment, the red pixel is located at the top left corner of the first Bayer unit; the first green pixel is located at the top right corner of the first Bayer unit; the second green pixel is located at the bottom left corner of the first Bayer unit; and the blue pixel is located at the bottom right corner of the first Bayer unit. Step 914 introduces the second Bayer unit. In this embodiment, the first green pixel is located at the top left corner of the second Bayer unit; the second red pixel is located at the top right corner of the second Bayer unit; the blue pixel is located at the bottom left corner of the second Bayer unit; and the third green pixel is located at the bottom right corner of the second Bayer unit.

[0111] The image sensor of method 900 may include a pixel array. In this embodiment, red pixels are aligned below a red color filter of the color filter array, each of the first and second green pixels is aligned below a green color filter of the color filter array, and blue pixels are aligned below a blue color filter of the color filter array.

[0112] In the first weighted sum in step 912, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by their respective weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively. The coefficients a11, a12, and a13 in equations (11) to (13) are the weights w. 11 w 12 and w 13 Examples.

[0113] In the second weighted sum in step 912, the pixel values ​​of the first red image, the first green image, and the first blue image can be determined by their respective weights w. 21 w 22 and w 23 The weights are derived from the transmission spectra of the green color filter at red, green, and blue wavelengths, respectively. The coefficients a21, a22, and a23 in equations (11) to (13) are the weights w. 21 w 22 and w 23 Examples.

[0114] In the third weighted sum in step 912, the pixel values ​​of the first red image, the first green image, and the first blue image can be determined by their respective weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at red, green, and blue wavelengths, respectively. The coefficients a31, a32, and a33 in equations (11) to (13) are the weights w. 31 w 32 and w 33 Examples.

[0115] The pixel array of method 900 may include a second Bayer unit (444) that partially overlaps with the first Bayer unit and includes a first green pixel, a blue pixel, a second red pixel (406), and a third green pixel (416). In this embodiment, step 910 may include step 914. Step 914 includes: for the first green pixel (404) of the first Bayer unit, determining a second red image pixel value, a second green image pixel value, and a second blue image pixel value via circuitry.

[0116] The result of step 914 is that each of the following weighted sums—the fourth, fifth, and sixth weighted sums of the second red image pixel value, the second green image pixel value, and the second blue image pixel value—is equal to the corresponding sensor pixel value of the pixel output of the second Bayer unit. The fourth weighted sum is equal to the second red sensor pixel value output by the second red pixel. The fifth weighted sum is equal to the average of the first green sensor pixel value and the third green sensor pixel value output by the third green pixel. The sixth weighted sum is equal to the blue sensor pixel value.

[0117] In this embodiment, the second red pixel is aligned below the red color filter of the color filter array, and the second green pixel is aligned below the green color filter of the color filter array. In the fourth weighted sum in step 914, the second red image pixel value, the first green image pixel value, and the first blue image pixel value can be determined by the aforementioned corresponding weights w.111 w 12 and w 13 Weighting is performed. In the fifth weighted sum in step 914, the pixel values ​​of the second red image, the first green image, and the first blue image can be weighted by the aforementioned corresponding weights w. 211 w 22 and w 23 Weighting is performed. In the sixth weighted sum in step 914, the pixel values ​​of the second red image, the first green image, and the first blue image can be weighted by the aforementioned corresponding weights w. 31 w 32 and w 33 Weighting is applied.

[0118] Step 920 includes outputting a demosaic image (890) via circuitry. Step 920 may include step 922, such that the demosaic image has a first RGB triplet at a position corresponding to the position of the red pixel in the pixel array, the first RGB triplet including a first red image pixel value, a first green image pixel value, and a first blue image pixel value. When step 910 includes step 914, step 920 may include step 924, such that the demosaic image has a second RGB triplet at a position corresponding to the position of the first green pixel in the pixel array, the second RGB triplet including a second red image pixel value, a second green image pixel value, and a second blue image pixel value.

[0119] Step 930 includes: using the readout circuit (841) of the image sensor to read each of the red sensor pixel value, the first green sensor pixel value, the second green sensor pixel value, and the blue sensor pixel value from the pixel array.

[0120] Figure 10 This is a flowchart illustrating a method 1000 for demosaicing an image captured by an image sensor (800). The image sensor has a pixel array (820A) including a first Bayer unit (642). Method 1000 is similar to method 900. In method 900, the color filter array is an RGB color filter array such as in pixel array 400. In method 900, the color filter array is an RGB-IR color filter array such as in pixel array 600. Method 1000 includes at least one of steps 1010, 1020, and 1030, which correspond to steps 910, 920, and 930, respectively.

[0121] Step 1010 includes determining the image pixel values ​​of the pixels of the first Bayer unit (642) of the pixel array. In step 1010, the values ​​R, G, B, and IR of equations (14) to (17) are examples of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value, respectively. In step 1010, red pixels can be generalized to "non-green pixels," such as blue pixels. Therefore, red pixels can be replaced with blue pixels, such that the examples of the first Bayer unit are... Figure 6 Bayer Unit 648.

[0122] Step 1010 may include at least one of steps 1012 and 1014. Step 1012 includes: for the red pixel of the first Bayer unit (642), determining a first red image pixel value, a first green image pixel value, a first blue image pixel value, and a first infrared (IR) image pixel value via a circuit (844) communicatively coupled to the pixel array.

[0123] The result of step 1012 is that each weighted sum of the first weighted sum, second weighted sum, third weighted sum, and fourth weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the corresponding sensor pixel value output by the red pixel. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to the green sensor pixel value output by the green pixel of the first Bayer unit. The third weighted sum is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The fourth weighted sum is equal to the IR sensor pixel value output by the IR pixel of the first Bayer unit. The right sides of equations (14), (15), (16), and (17) are examples of the first weighted sum, the second weighted sum, the third weighted sum, and the fourth weighted sum, respectively.

[0124] In this embodiment, the red pixel is located at the upper left corner of the first Bayer unit; the first green pixel is located at the upper right corner of the first Bayer unit; the IR pixel is located at the lower left corner of the first Bayer unit; and the blue pixel is located at the lower right corner of the first Bayer unit. Step 1014 introduces a second Bayer unit (644). The green pixel is located at the upper left corner of the second Bayer unit; the second red pixel (606) is located at the upper right corner of the second Bayer unit; the blue pixel (614) is located at the lower left corner of the second Bayer unit; and the second IR pixel (616) is located at the lower right corner of the second Bayer unit.

[0125] The image sensor of method 1000 may include a pixel array. In this embodiment, red pixels are aligned below the red color filter of the color filter array, green pixels are aligned below the green color filter of the color filter array, blue pixels are aligned below the blue color filter of the color filter array, and IR pixels are aligned below the IR color filter of the color filter array.

[0126] In the first weighted sum in step 1012, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by their respective weights w. 11 w 12 w 13 and w 14 Weights are applied, with the corresponding weights derived from the transmission spectra of the red color filter at red, green, blue, and IR wavelengths, respectively. The coefficients a11, a12, a13, and a14 in equations (14) to (17) are the weights w. 11 w 12 w 13 and w 14 Examples.

[0127] In the second weighted sum in step 1012, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by their respective weights w. 21 w 22 w 23 and w 24 Weights are applied, with the corresponding weights derived from the transmission spectra of the green color filter at red, green, blue, and IR wavelengths, respectively. The coefficients a21, a22, a23, and a24 in equations (14) to (17) are the weights w... 21 w 22 w B3 and w 24 Examples.

[0128] In the third weighted sum in step 1012, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by their respective weights w. 31 w 32 w 33 and w 34 Weighting is applied, with the corresponding weights derived from the transmission spectra of the blue color filter at red, green, blue, and IR wavelengths, respectively. The coefficients a31, a32, a33, and a34 in equations (14) to (17) are the weights w... 31 w 32 w 33 and w 34 Examples.

[0129] In the fourth weighted sum in step 1012, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by their respective weights w. 41 w 42 w 43 and w 44 Weights are applied, with the corresponding weights derived from the transmission spectra of the IR color filter at red, green, blue, and IR wavelengths, respectively. The coefficients a41, a42, a43, and a44 in equations (14) to (17) are the weights w... 41 w 42 w 43 and w 44 Examples.

[0130] The pixel array of method 1000 may include a second Bayer unit (644) that partially overlaps with the first Bayer unit and includes a first green pixel (604), a blue pixel (614), a second red pixel (606), and a second IR pixel (616). In these embodiments, step 1010 may include step 1014. Step 1014 includes: for the first green pixel of the first Bayer unit (604), determining, via circuitry, a second red image pixel value, a second green image pixel value, and a second blue image pixel value.

[0131] The execution result of step 1014 is as follows: each weighted sum of the fifth, sixth, seventh, and eighth weighted sums of the second red image pixel values, the second green image pixel values, the second blue image pixel values, and the second IR image pixel values ​​is equal to the corresponding sensor pixel value of the pixel output of the second Bayer unit. The fifth weighted sum is equal to the green sensor pixel value. The sixth weighted sum is equal to the second red sensor pixel value output by the second red pixel. The seventh weighted sum is equal to the second IR sensor pixel value output by the second IR pixel. The eighth weighted sum is equal to the blue sensor pixel value.

[0132] In this embodiment, the second red pixel is aligned below the red color filter of the color filter array, and the second IR pixel is aligned below the IR color filter of the color filter array. In the fifth weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by the aforementioned corresponding weights w. 11 w 12 and w 13 and w 14 Weighting is applied. In the sixth weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are weighted by the aforementioned corresponding weights w. 21 w22 w 23 and w 24 Weighting is applied. In the seventh weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are weighted by the aforementioned corresponding weights w. 31 w 32 w 33 and w 34 Weighting is applied. In the eighth weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are weighted by the aforementioned corresponding weights w. 41 w 42 w 43 and w 44 Weighting is applied.

[0133] Step 1020 includes outputting a demosaic image (890) via a circuit. Step 1020 may include step 1022, such that the demosaic image has a first RGB quadruple at the position corresponding to the position of the red pixel in the pixel array, the first RGB quadruple including a first red image pixel value, a first green image pixel value, a first blue image pixel value, and a first IR image pixel value. When step 1010 includes step 1014, step 1020 may include step 1024, such that the demosaic image has a second RGB quadruple at the position corresponding to the position of the green pixel in the pixel array, the second RGB quadruple including a second red image pixel value, a second green image pixel value, a second blue image pixel value, and a second IR image pixel value.

[0134] Step 1030 includes reading each of the red sensor pixel value, green sensor pixel value, blue sensor pixel value and IR sensor pixel value from the pixel array via the readout circuit (841) of the image sensor.

[0135] Figure 11 This is a flowchart illustrating a method 1100 for demosaicing an image captured by an image sensor (800). The image sensor has a pixel array (820A) including a first Bayer unit (442). Method 1100 includes at least one of steps 1110, 920, and 930, the latter two of which are used as method 900. Figure 9 A portion of the ) is introduced and described. Step 1110 may include at least one of steps 1112 and 1114. Steps 1110, 1112 and 1114 are equivalent to steps 910, 912 and 914 of method 900, respectively, wherein equations (5) to (7) replace equations (11) to (13) used in method 900.

[0136] Figure 12 This is a flowchart illustrating a method 1200 for demosaicing an image captured by an image sensor (800). The image sensor has a pixel array (700, 820A) comprising pixel groups comprising a plurality of pixels of number P. A first example of a pixel group is a Bayer unit as described herein, where P = 4. A second example of a pixel group is any array of Bayer units as described herein. Pixel arrays 704 and 706 ( Figure 7 This is also an example of a pixel group in method 1200. (e.g.) Figure 7 As shown, the pixel groups can be arranged in any shape. Method 1200 includes at least one of steps 1210, 1220 and step 930 introduced in the description of method 900.

[0137] Step 1210 includes: for the first pixel of the pixel group, determining P image pixel values ​​via a circuit (844) communicatively coupled to the pixel array, such that for each pixel in the plurality of pixels, the weighted sum of its corresponding P image pixel values ​​is equal to the sensor pixel value output by that pixel. Examples of the weighted sum include the weighted sums shown in equations (5) to (7), (10) to (13), and (15) to (17).

[0138] Step 1220 includes outputting a demosaic image (890) via circuitry, the demosaic image having an image pixel value array at a position corresponding to the position of the first pixel in the pixel array, the image pixel value array including each of P image pixel values.

[0139] Method 1200 may further include: repeating the determination step 1210 and the output step 1220 for each pixel other than the first pixel among a plurality of pixels.

[0140] Feature combination

[0141] The features described above, as well as those protected by the following claims, can be combined in various ways without departing from their scope. The examples listed below illustrate some possible, non-limiting combinations.

[0142] Example 1 is a method for demosaicing an image captured by an image sensor having a pixel array including a first Bayer unit. The method includes: for a red pixel of the first Bayer unit, determining a first red image pixel value, a first green image pixel value, and a first blue image pixel value via circuitry communicatively coupled to the pixel array, such that each weighted sum of a first weighted sum, a second weighted sum, and a third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the corresponding sensor pixel value output by the pixel of the first Bayer unit. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to the average of the first green sensor pixel value output by the first green pixel of the first Bayer unit and the second green sensor pixel value output by the second green pixel of the first Bayer unit. The third weighted sum is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The method further includes: outputting a demosaic image via a circuit, wherein the demosaic image has a first RGB triplet at a position corresponding to the position of a red pixel in the pixel array, the first RGB triplet including a first red image pixel value, a first green image pixel value, and a first blue image pixel value.

[0143] Example 2. According to the method described in Example 1, the image sensor includes a color filter array, where red pixels are aligned below the red color filter of the color filter array; each green pixel in the first green pixel and the second green pixel is aligned below the green color filter of the color filter array; and blue pixels are aligned below the blue color filter of the color filter array; wherein: in the first weighted sum, the first red image pixel value, the first green image pixel value, and the first blue image pixel value are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively; in the second weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are determined by their respective weights w. 21 w 22 and w 23 The weights are derived from the transmission spectra of the green color filter at red, green, and blue wavelengths, respectively; and in the third weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by their respective weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at red, green, and blue wavelengths, respectively.

[0144] Example 3. The method according to any one of Examples 1 or 2, further comprising: reading each of the red sensor pixel value, the first green sensor pixel value, the second green sensor pixel value, and the blue sensor pixel value from the pixel array via the readout circuit of the image sensor.

[0145] Example 4. According to any one of Examples 1 to 3, the circuit is an image signal processor, which is part of or communicatively coupled to an image sensor.

[0146] Example 5. According to any one of Examples 1 to 4, the red pixel is located at the upper left corner of the first Bayer unit; the first green pixel is located at the upper right corner of the first Bayer unit; the second green pixel is located at the lower left corner of the first Bayer unit; and the blue pixel is located at the lower right corner of the first Bayer unit.

[0147] Example 6. According to the method described in Example 5, the pixel array includes a second Bayer unit that partially overlaps with a first Bayer unit and includes a first green pixel, a blue pixel, a second red pixel, and a third green pixel. The method further includes: for the first green pixel of the first Bayer unit, determining a second red image pixel value, a second green image pixel value, and a second blue image pixel value by a circuit, such that: a fourth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the second red sensor pixel value output by the second red pixel; a fifth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the average value of the first green sensor pixel value and the third green sensor pixel value output by the third green pixel; and a sixth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the blue sensor pixel value; wherein, in the output, the demosaic image has a second RGB triplet at a position corresponding to the position of the first green pixel in the pixel array, the second RGB triplet including the second red image pixel value, the second green image pixel value, and the second blue image pixel value.

[0148] Example 7. According to the method described in Example 6, the first green pixel is located at the upper left corner of the second Bayer unit; the second red pixel is located at the upper right corner of the second Bayer unit; the blue pixel is located at the lower left corner of the second Bayer unit; and the third green pixel is located at the lower right corner of the second Bayer unit.

[0149] Example 8. According to the method of any one of Examples 6 or 7, the image sensor includes a color filter array, a second red pixel aligned below a red color filter of the color filter array; a second green pixel aligned below a green color filter of the color filter array; and wherein: in the fourth weighted sum, the second red image pixel value, the first green image pixel value, and the first blue image pixel value are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively; in the fifth weighted sum, the pixel values ​​of the second red image, the first green image, and the first blue image are determined by their respective weights w. 21 w 22 and w 23 The weights are derived from the transmission spectra of the green color filter at red, green, and blue wavelengths, respectively; and in the sixth weighted sum, the pixel values ​​of the second red image, the first green image, and the first blue image are determined by their respective weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at red, green, and blue wavelengths, respectively.

[0150] Example 9 is a method for demosaicing an image captured by an image sensor having a pixel array including a first Bayer unit. The method includes: for a red pixel of the first Bayer unit, determining a first red image pixel value, a first green image pixel value, a first blue image pixel value, and a first infrared (IR) image pixel value via circuitry communicatively coupled to the pixel array, such that each weighted sum of a first weighted sum, a second weighted sum, a third weighted sum, and a fourth weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the corresponding sensor pixel value output by the pixel of the first Bayer unit. The first weighted sum is equal to the red sensor pixel value output by the red pixel. The second weighted sum is equal to the green sensor pixel value output by the green pixel of the first Bayer unit. The third weighted sum is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit. The fourth weighted sum is equal to the IR sensor pixel value output by the IR pixel of the first Bayer unit. The method further includes: outputting a demosaic image via a circuit, wherein the demosaic image has a first RGB quadruple at a position corresponding to the position of a red pixel in the pixel array, the first RGB quadruple including a first red image pixel value, a first green image pixel value, a first blue image pixel value and a first IR image pixel value.

[0151] Example 10. According to the method described in Example 9, the red pixel is located at the upper left corner of the first Bayer unit; the green pixel is located at the upper right corner of the first Bayer unit; the IR pixel is located at the lower left corner of the first Bayer unit; and the blue pixel is located at the lower right corner of the first Bayer unit.

[0152] Example 11. According to the method of Example 10, the pixel array includes a second Bayer unit that partially overlaps with a first Bayer unit and includes green pixels, blue pixels, a second red pixel, and a second IR pixel. The method further includes: for the green pixels of the first Bayer unit, determining the values ​​of a second red image pixel, a second green image pixel, a second blue image pixel, and a second IR image pixel via a circuit, such that: a fifth weighted sum of the second red image pixel, the second green image pixel, the second blue image pixel, and the second IR image pixel is equal to the green sensor pixel value; a sixth weighted sum of the second red image pixel, the second green image pixel, the second blue image pixel, and the second IR image pixel is equal to... The second red sensor pixel value output from the second red pixel; the seventh weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the second IR sensor pixel value output from the second IR pixel; and the eighth weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the blue sensor pixel value; wherein, in the output, the demosaic image has a second RGB quadruple at the position corresponding to the position of the green pixel in the pixel array, the second RGB quadruple including the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value.

[0153] Example 12. According to the method described in Example 11, the green pixel is located at the upper left corner of the second Bayer unit; the second red pixel is located at the upper right corner of the second Bayer unit; the blue pixel is located at the lower left corner of the second Bayer unit; and the second IR pixel is located at the lower right corner of the second Bayer unit.

[0154] Example 13. According to any one of Examples 9 to 12, the image sensor includes a color filter array, where red pixels are aligned below the red color filter of the color filter array; green pixels are aligned below the green color filter of the color filter array; blue pixels are aligned below the blue color filter of the color filter array; and IR pixels are aligned below the IR color filter of the color filter array, wherein: in the first weighted sum, the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value are weighted by corresponding weights w. 11 w 12 w13 and w 14 The weights are derived from the transmission spectra of the red color filter at red, green, blue, and IR wavelengths, respectively; in the second weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are determined by their respective weights w. 21 w 22 w 23 and w 24 The weights are derived from the transmission spectra of the green color filter at red, green, blue, and IR wavelengths, respectively; in the third weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by their respective weights w. 31 w 32 w 33 and w 34 The weights are derived from the transmission spectra of the blue color filter at red, green, blue, and IR wavelengths, respectively; and in the fourth weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are determined by their respective weights w. 41 w 4′ w 43 and w 44 The weights are derived from the transmission spectra of the IR color filter at red, green, blue, and IR wavelengths, respectively.

[0155] Example 14. The method according to any one of Examples 9 to 13, further comprising: reading each of the red sensor pixel value, the green sensor pixel value, the blue sensor pixel value, and the IR sensor pixel value from the pixel array via the readout circuit of the image sensor.

[0156] Example 15. According to any one of Examples 9 to 14, the circuit is an image signal processor, which is part of or communicatively coupled to an image sensor.

[0157] Example 16. According to any one of Examples 11 to 15, the image sensor includes a color filter array, green pixels aligned below a green color filter of the color filter array; blue pixels aligned below a blue color filter of the color filter array; a second red pixel aligned below a red color filter of the color filter array; and a second IR pixel aligned below an IR color filter of the color filter array, wherein: in the fifth weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by corresponding weights w. 11w 12 w 13 and w 14 The weights are derived from the transmission spectra of the red color filter at red, green, blue, and IR wavelengths, respectively; in the sixth weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are determined by their respective weights w. 21 w 22 w 23 and w 24 The weights are derived from the transmission spectra of the green color filter at red, green, blue, and IR wavelengths, respectively; in the seventh weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are determined by their respective weights w. 31 w 32 w 33 and w 34 The weights are derived from the transmission spectra of the blue color filter at red, green, blue, and IR wavelengths, respectively; and in the eighth weighted sum, the pixel values ​​of the second red image, the second green image, the second blue image, and the second IR image are determined by their respective weights w. 41 w 42 w 43 and w 44 The weights are derived from the transmission spectra of the IR color filter at red, green, blue, and IR wavelengths, respectively.

[0158] Example 17. A method for demosaicing an image captured by an image sensor having a pixel array including Bayer units, the method comprising: for a red pixel of a Bayer unit, determining a first red image pixel value, a first green image pixel value, and a first blue image pixel value via a circuit communicatively coupled to the pixel array, such that: a first weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to a red sensor pixel value output by the red pixel; and a second weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to... The green sensor pixel value output by the green pixel adjacent to the red pixel of the Bayer unit; and the third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the blue sensor pixel value output by the blue pixel adjacent to at least one of the red and blue pixels of the Bayer unit; and the demosaic image is output through the circuit, which has a first RGB triplet at a position corresponding to the position of the red pixel in the pixel array, the first RGB triplet including the first red image pixel value, the first green image pixel value, and the first blue image pixel value.

[0159] Example 18. According to the method described in Example 17, the image sensor includes a color filter array, where red pixels, green pixels, and blue pixels are aligned below the red color filter, green color filter, and blue color filter of the color filter array, respectively; wherein: in the first weighted sum, the first red image pixel value, the first green image pixel value, and the first blue image pixel value are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively; in the second weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are determined by their respective weights w. 21 w 22 and w 23 The weights are derived from the transmission spectra of the green color filter at red, green, and blue wavelengths, respectively; and in the third weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by their respective weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at red, green, and blue wavelengths, respectively.

[0160] Example 19. A method for demosaicing an image captured by an image sensor having a pixel array comprising a group of pixels, the pixel group comprising a plurality of pixels, the method comprising: for a first pixel of the pixel group, determining an image pixel value by means of a circuit communicatively coupled to the pixel array such that for each of the plurality of pixels, a weighted sum of the image pixel values ​​is equal to a sensor pixel value output by the pixel; and outputting a demosaic image by means of the circuit, the demosaic image having an image pixel value array at a position corresponding to the position of the first pixel in the pixel array, the image pixel value array comprising each image pixel value in the image pixel values.

[0161] Example 20. According to the method of Example 19, the method further includes: repeating the determination and the output for each pixel other than the first pixel among a plurality of pixels.

[0162] Modifications may be made to the methods and systems described above without departing from the scope of the present embodiments. Therefore, it should be noted that the subjects included in the above description or shown in the accompanying drawings should be interpreted as exemplary rather than limiting. In this document, unless otherwise indicated, the phrase "in embodiments" is equivalent to the phrase "in some embodiments," and not to all embodiments.

[0163] As used in this specification, any of its appendices, and the appended claims, the singular forms “a” and “the” include a plural referent unless the context clearly indicates otherwise. It should also be noted that the term “or” generally includes the meaning of “and / or” unless the context clearly indicates otherwise. Examples of the terms “and / or” and “at least one of…” include, for example, in the case of “A and / or B”, phrases such as “at least one of A and B” and “at least one of A or B” cover the following choices: (i) only A, or (ii) only B, or (iii) both A and B. In the case of “A, B, and / or C”, phrases such as “at least one of A, B, and C” and “at least one of A, B, or C” cover the following choices: (i) only A, or (ii) only B, or (iii) only C, or (iv) only A and B, or (v) only A and C, or (vi) only B and C, or (vii) each of A, B, and C. This can be extended to list as many items as possible.

[0164] The following claims are intended to cover all general and specific features described herein, as well as all statements within the scope of this method and system, and in terms of language, all that can be considered to fall between them are included.

Claims

1. A method for demosaicing an image captured by an image sensor, the image sensor having a pixel array including first Bayer units, the method comprising: For the red pixel of the first Bayer unit, the first red image pixel value, the first green image pixel value, and the first blue image pixel value are determined by a circuit communicatively coupled to the pixel array, such that: The first weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the red sensor pixel value output by the red pixel; The second weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the average of the first green sensor pixel value output by the first green pixel of the first Bayer unit and the second green sensor pixel value output by the second green pixel of the first Bayer unit; and The third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit; as well as The circuit outputs a de-mosaic image, which has a first RGB triplet at a position corresponding to the position of the red pixel in the pixel array. The first RGB triplet includes the first red image pixel value, the first green image pixel value, and the first blue image pixel value.

2. The method according to claim 1, wherein the image sensor comprises a color filter array, The red pixel is aligned below the red color filter of the color filter array; Each of the first green pixel and the second green pixel is aligned below the green color filter of the color filter array; and The blue pixels are aligned below the blue color filter in the color filter array; in: In the first weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively. In the second weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 21 w 22 and w 23 The weighting, wherein the corresponding weights are derived from the transmission spectra of the green color filter at red, green, and blue wavelengths, respectively; and In the third weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at red, green, and blue wavelengths, respectively.

3. The method according to claim 1, further comprising: The image sensor readout circuit reads each of the red sensor pixel value, the first green sensor pixel value, the second green sensor pixel value, and the blue sensor pixel value from the pixel array.

4. The method according to claim 1, wherein the circuit is an image signal processor, the image signal processor being part of the image sensor or communicatively coupled to the image sensor.

5. The method according to claim 1, The red pixel is located at the upper left corner of the first Bayer unit; The first green pixel is located at the upper right corner of the first Bayer unit; The second green pixel is located at the lower left corner of the first Bayer unit; and The blue pixel is located at the lower right corner of the first Bayer unit.

6. The method of claim 5, wherein the pixel array comprises a second Bayer unit, the second Bayer unit partially overlapping the first Bayer unit and comprising the first green pixel, the blue pixel, the second red pixel, and the third green pixel, the method further comprising: For the first green pixel of the first Bayer unit, the circuit determines the second red image pixel value, the second green image pixel value, and the second blue image pixel value, such that: The fourth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the second red sensor pixel value output by the second red pixel; The fifth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the average of the first green sensor pixel value and the third green sensor pixel value output by the third green pixel; and The sixth weighted sum of the second red image pixel value, the second green image pixel value, and the second blue image pixel value is equal to the blue sensor pixel value; In the output, the demosaic image has a second RGB triplet at a position corresponding to the position of the first green pixel in the pixel array. The second RGB triplet includes the second red image pixel value, the second green image pixel value, and the second blue image pixel value.

7. The method according to claim 6, The first green pixel is located at the upper left corner of the second Bayer unit; The second red pixel is located at the upper right corner of the second Bayer unit; The blue pixel is located at the lower left corner of the second Bayer unit; and The third green pixel is located at the lower right corner of the second Bayer unit.

8. The method according to claim 6, wherein the image sensor comprises a color filter array. The second red pixel is aligned below the red color filter of the color filter array; The second green pixel is aligned below the green color filter in the color filter array; and in: In the fourth weighted sum, the pixel values ​​of the second red image, the first green image, and the first blue image are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively. In the fifth weighted sum, the second red image pixel value, the first green image pixel value, and the first blue image pixel value are weighted by corresponding weights w. 21 w 22 and w 23 Weighted, the corresponding weights are derived from the transmission spectra of the green color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively; and In the sixth weighted sum, the second red image pixel value, the first green image pixel value, and the first blue image pixel value are weighted by corresponding weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at the red, green, and blue wavelengths, respectively.

9. A method for demosaicing an image captured by an image sensor, the image sensor having a pixel array including first Bayer units, the method comprising: For the red pixel of the first Bayer unit, the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first infrared (IR) image pixel value are determined by a circuit communicatively coupled to the pixel array, such that: The first weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the red sensor pixel value output by the red pixel; The second weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the green sensor pixel value output by the green pixel of the first Bayer unit; The third weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the blue sensor pixel value output by the blue pixel of the first Bayer unit; and The fourth weighted sum of the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value is equal to the IR sensor pixel value output by the IR pixel of the first Bayer unit; as well as The circuit outputs a de-mosaic image, which has a first RGB quadruple at the position corresponding to the position of the red pixel in the pixel array. The first RGB quadruple includes the first red image pixel value, the first green image pixel value, the first blue image pixel value, and the first IR image pixel value.

10. The method according to claim 9, The red pixel is located at the upper left corner of the first Bayer unit; The green pixel is located at the upper right corner of the first Bayer unit; The IR pixel is located at the lower left corner of the first Bayer unit; and The blue pixel is located at the lower right corner of the first Bayer unit.

11. The method of claim 10, wherein the pixel array includes a second Bayer unit, the second Bayer unit partially overlapping the first Bayer unit and including the green pixel, the blue pixel, the second red pixel, and the second IR pixel, the method further comprising: For the green pixel of the first Bayer unit, the circuit determines the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value, such that: The fifth weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the green sensor pixel value; The sixth weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the second red sensor pixel value output by the second red pixel; The seventh weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the second IR sensor pixel value output by the second IR pixel; as well as The eighth weighted sum of the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value is equal to the blue sensor pixel value; In the output, the demosaic image has a second RGB quadruple at the position corresponding to the position of the green pixel in the pixel array. The second RGB quadruple includes the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value.

12. The method according to claim 11, The green pixel is located at the upper left corner of the second Bayer unit; The second red pixel is located at the upper right corner of the second Bayer unit; The blue pixel is located at the lower left corner of the second Bayer unit; and The second IR pixel is located at the lower right corner of the second Bayer unit.

13. The method according to claim 9, wherein the image sensor comprises a color filter array, The red pixel is aligned below the red color filter of the color filter array; The green pixels are aligned below the green color filter in the color filter array; The blue pixels are aligned below the blue color filter in the color filter array; as well as The IR pixels are aligned below the IR color filters of the color filter array; in: In the first weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by corresponding weights w. 11 w 12 w 13 and w 14 The weights are derived from the transmission spectra of the red color filter at red wavelength, green wavelength, blue wavelength, and IR wavelength, respectively. In the second weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by corresponding weights w. 21 w 22 w 23 and w 24 The weights are derived from the transmission spectra of the green color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively. In the third weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by corresponding weights w. 31 w 32 w 33 and w 34 The weighting, wherein the corresponding weights are derived from the transmission spectra of the blue color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively; and In the fourth weighted sum, the pixel values ​​of the first red image, the first green image, the first blue image, and the first IR image are weighted by corresponding weights w. 41 w 42 w 43 and w 44 The weights are derived from the transmission spectra of the IR color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively.

14. The method according to claim 9, further comprising: The readout circuit of the image sensor reads each of the red sensor pixel value, the green sensor pixel value, the blue sensor pixel value, and the IR sensor pixel value from the pixel array.

15. The method of claim 9, wherein the circuit is an image signal processor, the image signal processor being part of the image sensor or communicatively coupled to the image sensor.

16. The method of claim 11, wherein the image sensor comprises a color filter array, The green pixels are aligned below the green color filter in the color filter array; The blue pixels are aligned below the blue color filter in the color filter array; The second red pixel is aligned below the red color filter of the color filter array; and The second IR pixel is aligned below the IR color filter of the color filter array. in: In the fifth weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by corresponding weights w. 11 w 12 w 13 and w 14 The weights are derived from the transmission spectra of the red color filter at red wavelength, green wavelength, blue wavelength, and IR wavelength, respectively. In the sixth weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by corresponding weights w. 21 w 22 w 23 and w 24 The weights are derived from the transmission spectra of the green color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively. In the seventh weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by corresponding weights w. 31 w 32 w 33 and w 34 The weighting, wherein the corresponding weights are derived from the transmission spectra of the blue color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively; and In the eighth weighted sum, the second red image pixel value, the second green image pixel value, the second blue image pixel value, and the second IR image pixel value are weighted by corresponding weights w. 41 w 42 w 43 and w 44 The weights are derived from the transmission spectra of the IR color filter at the red wavelength, the green wavelength, the blue wavelength, and the IR wavelength, respectively.

17. A method for demosaicing an image captured by an image sensor, the image sensor having a pixel array including Bayer units, the method comprising: For the red pixel of the Bayer unit, the first red image pixel value, the first green image pixel value, and the first blue image pixel value are determined by a circuit communicatively coupled to the pixel array, such that: The first weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the red sensor pixel value output by the red pixel; The second weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the green sensor pixel value output by the green pixel adjacent to the red pixel in the Bayer unit; as well as The third weighted sum of the first red image pixel value, the first green image pixel value, and the first blue image pixel value is equal to the blue sensor pixel value output by the Bayer unit from at least one of the red and blue pixels; as well as The circuit outputs a mosaic image, and the de-mosaic image has a first RGB triplet at the position corresponding to the position of the red pixel in the pixel array. The first RGB triplet includes the first red image pixel value, the first green image pixel value, and the first blue image pixel value.

18. The method of claim 17, wherein the image sensor comprises a color filter array, The red pixel, the green pixel, and the blue pixel are aligned below the red color filter, the green color filter, and the blue color filter of the color filter array, respectively. in: In the first weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 11 w 12 and w 13 The weights are derived from the transmission spectra of the red color filter at red, green, and blue wavelengths, respectively. In the second weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 21 w 22 and w 23 Weighted, the corresponding weights are derived from the transmission spectra of the green color filter at the red wavelength, the green wavelength, and the blue wavelength, respectively; and In the third weighted sum, the pixel values ​​of the first red image, the first green image, and the first blue image are weighted by corresponding weights w. 31 w 32 and w 33 The weights are derived from the transmission spectra of the blue color filter at the red, green, and blue wavelengths, respectively.

19. A method for demosaicing an image captured by an image sensor, the image sensor having a pixel array comprising groups of pixels, the pixel groups comprising a plurality of pixels of number P, the method comprising: For the first pixel of the pixel group, P image pixel values ​​are determined by a circuit that is communicatively coupled to the pixel array, such that for each of the plurality of pixels, the weighted sum of the P image pixel values ​​is equal to the sensor pixel value output by the pixel. as well as The circuit outputs a de-mosaic image, which has an image pixel value array at a position corresponding to the position of the first pixel in the pixel array. The image pixel value array includes each of the P image pixel values.

20. The method according to claim 19, further comprising: For each of the plurality of pixels other than the first pixel, the determination and the output are repeated.

Images