Methods and appartus for using multiple optical chains in parallel with multiple different exposure times

Abstract

Methods and apparatus for capturing or generating images using multiple optical chains operating in parallel are described. Pixel values captured by individual optical chains corresponding to the same scene area are combined to provide an image with at least some of the benefits which would have been provided by capturing an image of the scene using a larger lens than that of the individual lenses of the optical chain modules. By using multiple optical chains in parallel at least some benefits normally obtained from using a large lens can be obtained without the need for a large lens. Furthermore in at least some embodiments, a wide dynamic range can be supported through the use of multiple sensors with the overall supported dynamic range being potentially larger than that of the individual sensors. Some lens and / or optical chain configurations are designed for use in small handheld devices, e.g., cell phones.

Description

RELATED APPLICATIONS[0001]The present application claims the benefit of U.S. Provisional Patent Application Ser. No. 61 / 749,314, filed Jan. 5, 2013, U.S. Provisional Patent Application Ser. No. 61 / 749,315, filed Jan. 5, 2013, U.S. Provisional Patent Application Ser. No. 61 / 749,316, filed Jan. 5, 2013, U.S. Provisional Patent Application Ser. No. 61 / 749,317, filed Jan. 5, 2013, and U.S. Provisional Patent Application Ser. No. 61 / 749,382, filed Jan. 6, 2013, and is related to U.S. Provisional Patent Application Ser. No. 61 / 923,755, filed Jan. 5, 2014 each of the forgoing patent applications being hereby expressly incorporated by reference in their entirety.FIELD[0002]The present application relates to image capture and generation methods and apparatus and, more particularly, to methods and apparatus related to camera apparatus including multiple optical chains or which processes the output of multiple optical chains.BACKGROUND[0003]High quality digital cameras have to a large extent r...

AI Technical Summary

Benefits of technology

[0022]The use of multiple optical chains in parallel, in various embodiments, provides many of the benefits associated with use of a large lens and / or large high quality sensor, through the use of multiple optical chains which can normally be implemented using smaller and / or lower cost components than commonly used with a high quality large lens single optical chain camera implementation.

[0023]In various embodiments an optical chain includes a combination of elements including one or more lenses and a sensor. The outer lenses of the optical chains in various embodiments have a known relationship, i.e., spacing, between them. This allows for pixels of an image captured by one optical chain to be easily matched to and combined with pixels corresponding to the same scene area captured by one or more of the other optical chains and then processed to generate a combined image from the various images captured by the sensors of the individual optical chains.

[0026]The methods and apparatus of the invention allow, in some embodiments, different, e.g., relatively low cost, optical chains to be used in parallel to provide many of the benefits of a large lens, e.g., a large light capture area, without the need for a large single large lens and many of the disadvantages associated with a single large lens not only in terms of cost, weight, size and / or other issues.

[0029]Various methods and apparatus of the present invention provide some or all of the benefits of using relatively large and / or long lens assemblies without the need for large lens and / or long lens assemblies, through the use of multiple optical chain modules in combination.

[0031]Using the methods and apparatus of the present invention, a handheld camera can provide improved still image and / or video generation results than might be achieved without the use of the methods described herein.

[0039]It should be appreciated that by starting the image exposure at the same time but using different exposure durations different optical chains will be able to reliably capture light (e.g., bright), midlevel, and dark image regions in a reliable manner allowing for the combined image to have a good level of quality over a wide range of light conditions.

Problems solved by technology

Individuals seeking to take quality photographs are often encouraged to invest in large bulky and often costly lenses for a variety of reasons.

Telephoto lenses tend to be large not only because of their large apertures but also because of their long focal lengths.

While large lenses have many advantages with regard to the ability to capture relatively large amounts of light compared to smaller lenses, support large zoom ranges, and often allow for good control over focus, there are many disadvantages to using large lenses.

Large lenses tend to be heavy requiring relatively strong and often large support structures to keep the various lenses of a camera assembly in alignment.

The heavy weight of large lenses makes cameras with such lenses difficult and bulky to transport.

Furthermore, cameras with large lenses often need a tripod or other support to be used for extended periods of time given that the sheer weight of a camera with a large lens can become tiresome for an individual to hold in a short amount of time.

In addition to weight and size drawbacks, large lenses also have the disadvantage of being costly.

This is because of, among other things, the difficulty in manufacturing large high quality optics and packaging them in a manner in which they will maintain proper alignment over a period of time which may reflect the many years of use a camera lenses is expected to provide.

However, for the vast majority of camera users, the drawbacks to cameras with large lenses means that camera users tend not to use large lenses with such lenses often being left to professionals and / or photo enthusiasts willing to incur the expense and trouble of buying and using large lenses.

For example, cell phone mounted cameras are often more readily available for use when an unexpected photo opportunity arises or in the case of a general family outing where carrying large bulky camera equipment may be uncomfortable or undesirable.

While a filter arrangement over a sensor array can be used to allow different sensor elements to capture different colors of light thus allowing a single sensor to capture a color image, the need to carefully align the filter area with individual pixel size sensor elements complicates the manufacture of sensor arrays as compared to arrays which do not require the use of a multi-color filter array.

Furthermore, the fact that multiple colors of light need to pass through the camera lenses to reach the sensor so that the sensor can measure multiple different colors of light means that the lens can not be optimized for a single color of light and that some chromatic aberration is likely to result.

Chromatic aberration is a type of distortion in which there is a failure of a lens to focus all colors to the same convergence point.

While small focal length lenses paired with relatively high resolution sensors have achieved widespread commercial success in cell phones and pocket cameras, they often leave their owners longing for better picture quality, e.g., picture quality that can only be achieved with a larger pixel area and a larger lens opening to collect more light.

However, there are a few disadvantages to using smaller sensors and lenses.

First, the small pixel size limits the dynamic range of the sensor as only a small amount of light can saturate the sensor.

Second, small lenses collect less total light which can result in grainy pictures.

Third, small lenses have small maximum apertures which make artistic effects like small depth of field for portrait pictures not possible.

The dynamic range of a photo sensor is somewhat limited.

As a result, details in dark areas of an area being photographed may be difficult to detect due to lack of sufficient light from such areas, e.g., underexposure with regard to such areas.

Similarly bright areas may appear overexposed due to the larger amount of light received from such portions of an image.

While some areas of an image quality may be captured using a reasonable exposure time, when the image includes areas covering a large dynamic range, i.e., there is a large difference in the light intensity coming from different portions of an area to be photographed, it may not be possible to achieve a good exposure for all portions of an image where the exposure time is limited by the use of a single optical chain.

Images