Sparse color image sensor system

Abstract

Systems and techniques are described for generating a color image by computationally combining color-undersampled and shifted information present in multiple component image frames captured by a sparse color filter array including a small number of wavelength-filtered pixels and a majority of remaining panchromatic pixels. A burst capture of multiple image frames is initiated by a color filter array including multiple sub-units, each sub-unit including a small number of one or more wavelength-filtered adjacent pixels and a majority of remaining panchromatic pixels. Each of the multiple image frames is processed to generate a resultant color image.

Description

[Background technology]

[0001] background To capture color information and enable the creation of color images, electronic image sensors in mobile and other imaging systems typically include color filter arrays (CFAs), such as Bayer CFAs, which contain separate wavelength-specific filters arranged or formed in a repeating pattern on individual pixels of the electronic image sensor. However, capturing such color information comes at a significant cost in optical efficiency because the amount of light received through such wavelength-specific filters is significantly reduced. Such disadvantages are exacerbated in low-light conditions, potentially resulting in increased noise, increased aliasing, reduced spatial resolution, and other degradations of image sharpness and quality, including due to post-processing techniques that may be used to compensate for optical inefficiencies. Summary of the Invention [Means for solving the problem]

[0002] Quick Overview In one example, a method may include initiating burst capture of a plurality of image frames by a color filter array (CFA) including a plurality of subunits, each subunit including a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels, and the method may further include receiving the plurality of image frames in response to the initiating step, processing each of the plurality of image frames, and generating an image based at least in part on the processing step.

[0003] Thus, the proposed method may include the use of a CFA having subunits, each subunit including a majority of panchromatic pixels and one or more adjacent pixels for sampling light of a defined range of wavelengths, with the majority of pixels in each subunit being panchromatic. The panchromatic pixels provide luminance information for the captured image frame, while wavelength-filtered pixels are provided for sampling light of a defined wavelength range, e.g., sampling red, green, or blue light. Thus, in an exemplary embodiment, the majority of panchromatic pixels in the CFA provide luminance data but not color information, while the few wavelength-filtered pixels provided may be filtered according to wavelength to undersample the color data to be processed. Thus, color data from the few wavelength-filtered pixels and luminance data from the majority of panchromatic pixels for each (component) image frame captured in a burst capture may be processed to generate a single (complete) color image.

[0004] In this context, one or more wavelength-filtered pixels contiguous, comprising a minority of the pixels of a subunit, includes, inter alia, two or more wavelength-filtered pixels spatially arranged such that no pixel of the subgroup that is not wavelength-filtered is located between two or more wavelength-filtered pixels of the subgroup. Reference herein to a subunit having a single "adjacent" pixel means that the referenced subunit contains only one wavelength-filtered pixel, and the remaining pixels of the subunit are panchromatic.

[0005] The one or more wavelength-filtered adjacent pixels of each sub-unit may include a cluster of one red wavelength-filtered pixel, one blue wavelength-filtered pixel, and one or more green wavelength-filtered pixels.

[0006] The one or more wavelength-filtered adjacent pixels of each sub-unit may consist of a single wavelength-filtered pixel.

[0007] The one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits may be separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

[0008] Processing each of the plurality of image frames may include aligning corresponding subunits of the plurality of image frames to compensate for one or more spatial offsets between respective image frames of the plurality of image frames, which may be due to, for example, involuntary movement of a device including an image sensor and a CFA coupled to the image sensor, causing at least one object depicted in successive image frames captured in a burst to be spatially offset.

[0009] Each subunit of the plurality of subunits may have a ratio of wavelength-filtered pixels to total pixels of less than 1:5.

[0010] Each subunit of the plurality of subunits may be associated with a color sampling rate of less than 20%.

[0011] In another example, an imaging system may include an image sensor and a CFA, wherein the CFA includes a plurality of subunits, each subunit including a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels, and the imaging system may further include an image sensor coupled to the CFA and a processor coupled to the image sensor, wherein the processor may initiate burst capture of a plurality of image frames by the CFA and the image sensor and generate a color image by processing each of the plurality of image frames.

[0012] The one or more wavelength-filtered adjacent pixels of each subunit may include a cluster of at least one first pixel filtered according to a first wavelength and at least one second pixel filtered according to another second wavelength. For example, the one or more wavelength-filtered adjacent pixels of each subunit may include a cluster of at least one first pixel filtered according to a first red wavelength and at least one second pixel filtered according to another second blue or green wavelength. In an exemplary embodiment, each subunit includes a cluster of one red-wavelength-filtered pixel, one blue-wavelength-filtered pixel, and one or more green-wavelength-filtered pixels.

[0013] The one or more wavelength-filtered adjacent pixels of each sub-unit may consist of a single wavelength-filtered pixel.

[0014] The one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits may be separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

[0015] Processing each of the plurality of image frames may include compensating for one or more spatial offsets between respective image frames of the plurality of image frames by aligning corresponding sub-units of the plurality of image frames.

[0016] Each subunit of the plurality of subunits may have a ratio of wavelength-filtered pixels to total pixels of less than 1:5.

[0017] Each subunit of the plurality of subunits may be associated with a color sampling rate of less than 20%.

[0018] In another example, a computing device may include an image sensor and a CFA, wherein the CFA includes multiple subunits, each subunit including a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels; the computing device may further include an image sensor coupled to the CFA, one or more lenses for directing light through the CFA onto the image sensor, and one or more processors coupled to the image sensor; the one or more processors may initiate burst capture of multiple image frames by the CFA and the image sensor and generate a color image by processing each of the multiple image frames.

[0019] The one or more wavelength-filtered adjacent pixels of each sub-unit may include a cluster of one red wavelength-filtered pixel, one blue wavelength-filtered pixel, and one or more green wavelength-filtered pixels.

[0020] The one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits may be separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

[0021] Generating the color image may include compensating for one or more spatial offsets between respective image frames of the plurality of image frames by spatially aligning corresponding subunits of the plurality of image frames.

[0022] Each subunit of the plurality of subunits may provide a color sampling rate of less than 20%.

[0023] The computing device may further include a head-wearable display (HWD). [Effects of the Invention]

[0024] Non-stationary imaging systems, such as handheld cameras or cameras worn on a user's head, typically have "free motion" between the capture of successive images. The techniques described herein can generate a complete RGB image by utilizing bursts of such successive images captured by a ("sparse") color filter array (CFA) by undersampling color information across unaligned image frames with small spatial offsets, such as those resulting from involuntary movements of a user who may be holding or wearing the image capture device. In certain embodiments, after capturing such bursts of image frames with sparsely sampled color information, the captured image frames are then used in "temporal registration." Each image frame is computationally processed to align and merge the image frames to form a single image in which every pixel is assigned a color value (e.g., a red value, a green value, and a blue value). In other embodiments, this temporal registration of image frames may be omitted, such as by utilizing partial (sparse) color data in a mostly monochrome image.

[0025] Systems and techniques are described for generating a color image by computationally combining color-undersampled and shifted information present in multiple component image frames captured by a sparse color filter array containing a small number of wavelength-filtered pixels and a remaining majority of panchromatic pixels. A burst capture of multiple image frames can be initiated by a color filter array containing multiple subunits, each subunit containing a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels. Each of the multiple image frames can then be processed to generate a resultant color image.

[0026] The present disclosure may be better understood, and its numerous features and advantages made apparent to those skilled in the art by referencing the accompanying drawings, in which the same reference numbers used in different drawings indicate similar or identical items. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a block diagram of an example imaging system according to one or more embodiments. [Figure 2] FIG. 1 illustrates a computational process for captured bursts of image data associated with multiple individual image frames, according to one or more embodiments. [Figure 3] FIG. 1 illustrates a portion of an example point-sampling color filter array (CFA) according to one or more embodiments. [Figure 4] FIG. 1 illustrates a portion of an example of a cluster sampling CFA according to one or more embodiments. [Figure 5] FIG. 1 illustrates a portion of an alternative example of a cluster sampling CFA according to one or more embodiments. [Figure 6] FIG. 10 illustrates a portion of another alternative cluster sampling CFA according to one or more embodiments. [Figure 7] FIG. 1 is a block diagram illustrating an overview of the operation of an example imaging system, according to one or more embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0028] Detailed Description Image capture mechanisms on mobile device platforms, such as smartphones or head-mounted displays, typically have small sensors that limit the spatial resolution of such mechanisms, small apertures that limit light gathering capabilities, and small pixels that reduce signal-to-noise ratios. Solutions that involve using color filter arrays (CFAs) with device sensors to provide color information typically involve optical inefficiencies resulting from significant blocking of incident light by a large number of wavelength-filtered picture elements (pixels), which can further reduce resolution. Each wavelength-filtered pixel in such a filter array (e.g., a pixel filtered for one of the red, green, or blue wavelengths) significantly reduces the optical efficiency of the pixel by blocking a portion of the light that passes through that pixel. Additionally, measures are often necessary to reduce crosstalk between pixels with incorrectly assigned color values.

[0029] Embodiments of the technology described herein utilize bursts of images captured by a “sparse CFA” to generate a complete RGB image, such as by sampling color data across unaligned images with small spatial offsets, including spatial offsets due to involuntary movements of a user who may be holding or wearing the image capture device. References to a sparse CFA herein mean that only a relatively small number of pixels in the CFA are filtered according to wavelength to undersample the color data, with the majority of the remaining pixels in the CFA comprising panchromatic (sometimes referred to as “white”) pixels that provide luminance data but no color information. In certain embodiments, after capturing bursts of image frames with such offsets and sparsely sampled color information, the captured image frames are then aligned and merged through a computational process (“temporal registration”) to form a single image in which every pixel is assigned a color value (e.g., a red value, a green value, and a blue value). In certain embodiments, such techniques provide higher effective resolution and greater detail than any of the component image frames, and the sparse CFA blocks significantly less light than a conventional non-sparse CFA configuration (e.g., a Bayer CFA configuration). Thus, in such embodiments, the described techniques computationally combine the color-undersampled and shifted information present in multiple component image frames to generate a full color image.

[0030] In certain embodiments, this temporal overlap can be avoided, such as by utilizing partial color data in a mostly monochrome image. In such embodiments, a majority of (monochrome) pixels maximizes light sensitivity, while a sparse subset of pixels bearing color information provides additional information to, for example, improve the results of various machine learning techniques. In particular, certain phases of machine learning (e.g., image classification and / or object detection) can generally use low-resolution (subsampled) images, and while the availability of color information from the sparse color CFA described herein can be improved by pixel classification (“binning”), the resolution is reduced, resulting in typically fewer pixels with classified color information being included in each “bin.” In contrast, subsequent phases of machine learning (e.g., object recognition) can generally utilize high-resolution images but are less dependent on the presence of detailed color data.

[0031] Typically, an image signal processor (ISP) can perform calculations to estimate the intensity of each of the three primary colors for each pixel in a captured image, even though each pixel is associated with data representing the intensity of only one primary color. However, as described elsewhere herein, such primary color filters can substantially reduce the amount of available light by filtering each pixel according to specific wavelengths, thereby increasing image noise. In contrast, the techniques presented herein can significantly improve the signal-to-noise ratio by utilizing a sparse CFA as a mostly monochrome sensor, and can be performed with less data processing and correspondingly reduced computational and power requirements. Furthermore, by scaling the image to a lower resolution (e.g., for machine learning purposes), high-quality intensity data can be combined with available sparse color data to provide both improved intensity and improved color data, again with less data processing.

[0032] Thus, while the examples provided herein may discuss embodiments that utilize temporal overlap to form a single image in which every pixel is assigned a color value, it will be understood that in various embodiments, the images and information captured by a sparse color CFA can be used in many applications without such temporal overlap. It will be further understood that while this disclosure describes various techniques in light of the red-green-blue (RGB) color model, such techniques may be utilized with a variety of additional color models (e.g., cyan-magenta-yellow (CMY)) without departing from the described embodiments.

[0033] FIG. 1 is a block diagram of an imaging system 100 according to one or more embodiments. In certain embodiments, the imaging system 100 may comprise part of a mobile device such as a smartphone, a head-mounted display (HWD), or other device. The imaging system includes a lens 105, a color filter array (CFA) 110, an image sensor 115, a controller 125, a processor 130 coupled to a system memory 135, and a display 140. In the illustrated embodiment, the imaging system 100 may be utilized to capture and process a burst of multiple image frames 120 for purposes of generating a complete image in accordance with the techniques described herein, such as for display by the display 140, storage for later display, or other use. In certain embodiments, one or more of the components described herein (e.g., one or more of the CFA 110, the image sensor 115, the controller 125, the processor 130, and the system memory 135) may be instantiated as elements of a single integrated system, such as a system-on-chip (SOC) or other integrated system. Also, in various embodiments, CFA 110 and image sensor 115 may be integrated, such as when the picture elements (pixels) of CFA 110 are formed or disposed on the surface of image sensor 115.

[0034] In operation, light 101 passes through lens 105 and CFA 110 and is sensed by image sensor 115, which uses the received light to generate information representative of multiple image frames 120. Lens 105 may include any suitable lens, including, by way of non-limiting example, a rectilinear lens, a wide-angle (or "fisheye") lens, a fixed focal length lens, a zoom lens, a fixed aperture or a variable aperture lens, etc. In various embodiments, image sensor 115 may include a complementary metal oxide semiconductor (CMOS) image sensor, a charge-coupled device (CCD) image sensor, or other suitable image sensor. In the illustrated embodiment, controller 125 may adjust the exposure time associated with each of the multiple image frames 120, such as under the direction of processor 130. Additionally, processor 130 may initiate the capture of one or more of the plurality of image frames 120 (autonomously or in response to user input), such as by initiating a "burst capture" of multiple image frames.

[0035] It will be understood that many variations and features of the imaging system shown in FIG. 1 may be utilized without departing from the scope of the present disclosure, including, for example, one or more elements such as an autofocus system, a mechanical or electromechanical shutter system, etc.

[0036] FIG. 2 illustrates computational processing of a captured burst of image data associated with four individual image frames (frame 1 210, frame 2 220, frame 3 230, and frame 4 240) according to one or more embodiments. Each image frame 220, 230, and 240 is shown with an exemplary point (not present in the actual image frame data) at the center of each representative pixel, offset from the previous image frame by a virtually randomized spatial offset resulting from involuntary movement of a user, such as a user holding or wearing an embodiment of the associated imaging system (e.g., imaging system 100 of FIG. 1). In certain embodiments, frame 1 210 may be utilized as a “base frame” for registering the remaining image frames 240, 230, and 240. To generate merged image 250, pixel density is increased using sampled color and luminance data from each of the four component image frames by utilizing computational processing to detect and register features of each component image frame.

[0037] FIG. 3 illustrates a portion of an example point-sampling color filter array (CFA) 300 according to one or more embodiments. The portion of CFA 300 shown shows a representative grid of 256 pixels divided into 16 subunits, each consisting of 4x4 pixels. In the illustrated embodiment, each subunit of CFA 300 includes only one wavelength-filtered pixel. For example, the top-left subunit 305 includes one pixel 301 wavelength-filtered to sample red (R) light, and all other pixels in that subunit include panchromatic pixels to provide luminance information. Similarly, subunit 310 includes one pixel 306 wavelength-filtered to sample green (G) light, and all other pixels in that subunit include panchromatic pixels to provide luminance information. Subunit 315 includes one pixel 311 wavelength-filtered to sample blue (B) light, and all other pixels in that subunit also include panchromatic pixels to provide luminance information. For clarity, the remaining subunits of CFA 300 are shown but will not be described individually, as each remaining subunit is identical to one of the described subunits 305, 310, and 315. It will be understood that each subunit of point-sampling CFA 300 contains only one wavelength-filtered pixel out of the subunit's respective 16 pixels, and therefore each subunit has a color sampling rate of 6.25%.

[0038] FIG. 4 illustrates a portion of an exemplary cluster-sampling CFA 400 according to one or more embodiments. The portion of the CFA 400 shown illustrates another representative grid of 256 pixels divided into 16 subunits, each consisting of 4x4 pixels, similar to that illustrated by the CFA 300 of FIG. 3. In the illustrated embodiment, each subunit of the CFA 400 includes a cluster of adjacent wavelength-filtered pixels, with all other pixels in each subunit being panchromatic pixels providing luminance information. An exemplary subunit 410, representative of all 16 subunits of the CFA 400, includes a cluster of wavelength-filtered pixels including a red (R) pixel 412, a green (G) pixel 414, and a blue (B) pixel 416. All other pixels in the subunit 410 are panchromatic pixels providing luminance information. In the illustrated embodiment, the "cluster spacing" of the CFA 400 includes the distance 418 between two pixels. That is, each cluster of wavelength-filtered pixels is separated from the corresponding cluster of the adjacent subunit by two panchromatic pixels. Thus, the cluster-sampling CFA 400 includes subunits in which three out of sixteen pixels are wavelength-filtered pixels, and each subunit therefore provides a color sampling rate of 18.75%.

[0039] FIG. 5 illustrates a portion of another exemplary cluster-sampling CFA 500 according to one or more embodiments. The portion of CFA 500 shown illustrates another representative grid of 256 pixels divided into subunits, similar to that illustrated by CFA 400 of FIG. 4. Each subunit of CFA 500 again includes a cluster of adjacent wavelength-filtered pixels, with all other pixels in each subunit being panchromatic pixels that provide luminance information. Exemplary subunit 510, representative of other subunits of CFA 500, includes a cluster of wavelength-filtered pixels including a red (R) pixel 512, a green (G) pixel 514, and a blue (B) pixel 516. All other pixels in subunit 510 are panchromatic pixels that provide luminance information. However, in the illustrated embodiment, in contrast to CFA 400 of Figure 4, CFA 500 includes subunits of 25 pixels each (5x5), with a cluster spacing distance 518 of three panchromatic pixels between corresponding wavelength-filtered clusters of adjacent subunits. Thus, cluster-sampling CFA 500 includes subunits in which three of the 25 pixels are wavelength-filtered pixels, and therefore each subunit provides a color sampling rate of only 12%.

[0040] FIG. 6 illustrates a portion of another exemplary cluster-sampling CFA 600 according to one or more embodiments. The portion of CFA 600 shown illustrates another representative grid of 256 pixels divided into subunits, similar to those illustrated by CFA 400 of FIG. 4 and CFA 500 of FIG. 5. Each subunit of CFA 600 again includes a cluster of adjacent wavelength-filtered pixels, with all other pixels in each subunit being panchromatic pixels that provide luminance information. Exemplary subunit 610, representative of other subunits of CFA 600, includes a cluster of wavelength-filtered pixels including a red (R) pixel 612, a green (G) pixel 614, and a blue (B) pixel 616. All other pixels in subunit 610 are panchromatic pixels that provide luminance information. In the illustrated embodiment, in contrast to CFA 400 of Figure 4 and CFA 500 of Figure 5, respectively, CFA 600 includes subunits of 36 pixels each (6x6), with a cluster spacing distance 618 of four panchromatic pixels between corresponding wavelength-filtered clusters of adjacent subunits. Thus, cluster-sampling CFA 600 includes subunits in which three of the 36 pixels are wavelength-filtered pixels, and therefore each subunit provides a color sampling rate of only 8.33%.

[0041] It will be understood that in various embodiments, the arrangement of color sampling clusters of wavelength-filtered pixels may be configured in various ways without departing from the techniques described herein. In particular, in certain embodiments, the color sampling clusters may include additional wavelength-filtered pixels (e.g., one or more additional green (G) or other wavelength-specific filtered pixels), may be spatially arranged in ways other than those shown with respect to the above embodiments, etc.

[0042] 7 is a block diagram illustrating an overview of a processor-based imaging system operating routine 700, according to one or more embodiments, which may be performed, for example, by an embodiment of the imaging system 100 of FIG. 1 or by some other embodiment.

[0043] The routine begins at block 705, where the processor-based imaging system initiates burst capture of multiple image frames with a sparse color filter array. As described elsewhere herein, the initiation of burst capture may be performed autonomously by the processor of the processor-based imaging system (e.g., based on one or more defined criteria), in response to one or more user inputs, or in some other manner.

[0044] The routine proceeds to block 710 where color-sampled image data representing a plurality of image frames is received by a processor-based imaging system. The routine then proceeds to block 715.

[0045] At block 715, the processor-based imaging system processes the color-sampled image data associated with the multiple image frames. In certain embodiments, such processing may include spatially registering each of the multiple image frames to compensate for spatial offsets between consecutive image frames of the multiple image frames (e.g., due to involuntary user movement). The routine then proceeds to block 720.

[0046] At block 720, the processor-based imaging system generates a single color image based on the multiple processed image frames, such as for display on a display device, storage, or other purposes.

[0047] In some embodiments, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored on or tangibly embodied on a non-transitory computer-readable storage medium. The software may include these instructions and specific data that, when executed by the one or more processors, operate the one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, a magnetic or optical disk storage device, a solid-state storage device such as flash memory, a cache, a random access memory (RAM), or one or more other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be in the form of source code, assembly language code, object code, or other instruction formats that are interpretable or executable by one or more processors.

[0048] A computer-readable storage medium may include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tape, or magnetic hard drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS)-based storage media. The computer-readable storage medium may be incorporated into the computing system (e.g., system RAM or ROM), permanently attached to the computing system (e.g., a magnetic hard drive), removably attached to the computing system (e.g., an optical disk or Universal Serial Bus (USB)-based flash memory), or coupled to the computer system via a wired or wireless network (e.g., Network Accessible Storage (NAS)).

[0049] It should be noted that not all of the operations or elements described above in the general description are required. Parts of certain operations or devices may not be required. One or more additional operations may be performed or one or more elements may be included in addition to those described above. Furthermore, the order in which operations are listed is not necessarily the order in which those operations are performed. Also, concepts are described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure, as set forth in the following claims. Accordingly, the specification and drawings should be regarded in an illustrative, rather than a restrictive, sense, and all such modifications are intended to be included within the scope of the present disclosure.

[0050] Benefits, other advantages, and solutions to problems have been described above with reference to specific embodiments. However, benefits, advantages, solutions to problems, and any features that may cause or make more pronounced any benefit, advantage, or solution should not be construed as key, required, or essential features of any or all claims. Moreover, the specific embodiments disclosed above are illustrative only. The disclosed subject matter may be modified and practiced in different but equivalent manners, as will be apparent to those skilled in the art having the benefit of the teachings herein. No limitations are intended to the details of construction or design shown herein, other than as set forth in the following claims. It is therefore apparent that the specific embodiments disclosed above may be altered or modified, and all variations are considered within the scope of the disclosed subject matter. The protection sought herein is therefore set forth in the following claims.

Claims

1. 1. A method comprising: Initiating burst capture of a plurality of image frames with one or more spatial offsets by a Color Filter Array (CFA) for use in a non-fixed imaging system, the CFA including a plurality of sub-units, each sub-unit including a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels, the method further comprising: receiving the plurality of image frames in response to the initiating step; processing each of the plurality of image frames; generating an image based on the processing step; 11. The method of claim 10, wherein processing each of the plurality of image frames comprises aligning subunits of the plurality of image frames to compensate for the one or more spatial offsets between respective image frames of the plurality of image frames.

2. 2. The method of claim 1 , wherein the one or more wavelength-filtered adjacent pixels of each sub-unit include a cluster of one red wavelength-filtered pixel, one blue wavelength-filtered pixel, and one or more green wavelength-filtered pixels.

3. The method of claim 1 , wherein the one or more wavelength-filtered adjacent pixels of each sub-unit consist of a single wavelength-filtered pixel.

4. 4. The method of claim 1, wherein the one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits are separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

5. The method of any one of claims 1 to 4, wherein each subunit of the plurality of subunits has a ratio of wavelength-filtered pixels to total pixels of the CFA of less than 1:

5.

6. The method of any one of claims 1 to 5, wherein each subunit of the plurality of subunits is associated with a color sampling rate of less than 20%.

7. A non-stationary imaging system, comprising: An image sensor; a color filter array (CFA), the CFA including a plurality of sub-units, each sub-unit including a small number of adjacent pixels that are one or more wavelength-filtered and a remaining majority of panchromatic pixels, the non-fixed imaging system further comprising: an image sensor coupled to the CFA; a processor coupled to the image sensor, the processor initiating burst capture of a plurality of image frames with the CFA and the image sensor, the image frames including one or more spatial offsets, and processing each of the plurality of image frames to generate a color image; processing each of the plurality of image frames includes compensating for the one or more spatial offsets between respective image frames of the plurality of image frames by aligning subunits of the plurality of image frames.

8. 8. The imaging system of claim 7, wherein the one or more wavelength-filtered adjacent pixels of each sub-unit include a cluster of one red wavelength-filtered pixel, one blue wavelength-filtered pixel, and one or more green wavelength-filtered pixels.

9. The imaging system of claim 7 , wherein the one or more adjacent wavelength-filtered pixels of each sub-unit consist of a single wavelength-filtered pixel.

10. 10. The imaging system of claim 7, wherein the one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits are separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

11. The imaging system of any one of claims 7 to 10, wherein each subunit of the plurality of subunits has a ratio of wavelength-filtered pixels to total pixels of the CFA of less than 1:

5.

12. The imaging system of any one of claims 7 to 11, wherein each subunit of the plurality of subunits is associated with a color sampling rate of less than 20%.

13. A computing device for use in a non-fixed imaging system, comprising: An image sensor; a color filter array (CFA), the CFA including a plurality of sub-units, each sub-unit including a small number of one or more wavelength-filtered adjacent pixels and a remaining majority of panchromatic pixels, the computing device further comprising: an image sensor coupled to the CFA; one or more lenses for directing light through the CFA onto the image sensor; one or more processors coupled to the image sensor, wherein the one or more processors initiate burst capture of a plurality of image frames with one or more spatial offsets by the CFA and the image sensor, and process each of the plurality of image frames to generate a color image; A computing device, wherein generating the color image includes compensating for the one or more spatial offsets between respective image frames of the plurality of image frames by spatially aligning subunits of the plurality of image frames.

14. 14. The computing device of claim 13, wherein the one or more wavelength-filtered adjacent pixels of each sub-unit include a cluster of one red wavelength-filtered pixel, one blue wavelength-filtered pixel, and one or more green wavelength-filtered pixels.

15. 15. The computing device of claim 13 or 14, wherein the one or more wavelength-filtered adjacent pixels of each subunit of the plurality of subunits are separated from the one or more wavelength-filtered adjacent pixels of each adjacent subunit of the plurality of subunits by at least two panchromatic pixels.

16. A computing device according to any one of claims 13 to 15, wherein each subunit of the plurality of subunits provides a color sampling rate of less than 20%.

17. The computing device of any one of claims 13 to 16, further comprising a Head-Wearable Display (HWD).

Images