Multi-camera system for wide-angle imaging

Abstract

Systems and techniques for large field of view digital imaging are described. A first image sensor of a device captures a first image based on first light redirected from a first path to a redirected first path by a first light redirecting element, and a second image sensor of the device captures a second image based on second light redirected from a second path to a redirected second path by a second light redirecting element. A virtual extension of the first path beyond the first light redirecting element may intersect a virtual extension of the second path beyond the second light redirecting element. The device may modify the first image and the second image using perspective distortion correction and may generate a combined image by combining the first image and the second image. The combined image may have a larger field of view than the first image and / or the second image.

Description

Technical Field

[0001] Broadly speaking, the present disclosure relates to image or video capture devices, including multi-camera systems for generating images with a large (e.g., wide) field of view. Background Art

[0002] Many devices include one or more cameras. For example, a smartphone or a tablet includes a front camera for capturing self-portrait images and a rear camera for capturing images of scenes (e.g., landscapes or other scenes of interest to the user of the device). A user may wish to capture an image of a scene that does not fit within the field of view of the camera. Some devices include multiple cameras with different fields of view based on the curvature of the camera lenses that direct light to the image sensors. Thus, the user can use a camera with a desired field of view of the scene based on the curvature of the camera lens to capture an image. Summary of the Invention

[0003] Systems and techniques for digital imaging to generate images with a large field of view are described. For example, a device may include a first camera having a first image sensor that captures a first image based on first light redirected by a first light redirecting element. The first light redirecting element may redirect the first light from a first path to a redirected first path toward the first camera. The device may include a second camera having a second image sensor that captures a second image based on second light redirected by a second light redirecting element. The second light redirecting element may redirect the second light from a second path to a redirected second path toward the second camera. The first camera, the second camera, the first light redirecting element, and the second light redirecting element may be arranged such that a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element. These elements may be arranged such that a first lens of the first camera and a second lens of the second camera are virtually overlapped based on light redirection without physically overlapping. The device may use perspective distortion correction to modify the first image and / or the second image, e.g., to make the first image and the second image appear to be taken from the same angle of view of the scene. The device may generate a composite image from the first image and the second image, e.g., by aligning and stitching together the first image and the second image. The composite image may have a larger field of view than the first image, the second image, or both.

[0004] In one example, a device for digital imaging is provided. The device includes a memory and one or more processors coupled to the memory, the one or more processors (e.g., implemented in circuitry) being coupled to the memory. The one or more processors are configured to and can: receive a first image of a scene captured by a first image sensor, wherein a first light redirecting element redirects first light from a first path to a redirected first path toward the first image sensor, wherein the first image sensor captures the first image based on receiving the first light at the first image sensor; receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path toward the second image sensor, wherein the second image sensor captures the second image based on receiving the second light at the second image sensor, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; modify at least one of the first image and the second image using perspective distortion correction; and generate a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0005] In another example, a digital imaging method is provided. The method includes: receiving a first image of a scene captured by a first image sensor, wherein a first light redirecting element redirects first light from a first path to a redirected first path towards the first image sensor, and wherein the first image sensor captures the first image based on the first light received at the first image sensor. The method includes: receiving a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path towards the second image sensor, and wherein the second image sensor captures the second image based on the second light received at the second image sensor, and wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element. The method includes: modifying at least one of the first image and the second image using perspective distortion correction. The method includes: generating a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0006] In another example, a non-transitory computer-readable storage medium is provided, having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: receive a first image of a scene captured by a first image sensor, wherein a first light redirecting element redirects first light from a first path to a redirected first path towards the first image sensor, and wherein the first image sensor captures the first image based on the first light received at the first image sensor; receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path towards the second image sensor, and wherein the second image sensor captures the second image based on the second light received at the second image sensor, and wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; modify at least one of the first image and the second image using perspective distortion correction; and generate a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0007] In another example, a device for digital imaging is provided. The device includes: a unit for receiving a first image of a scene captured by a first image sensor, wherein a first light redirecting element redirects first light from a first path to a redirected first path towards the first image sensor, wherein the first image sensor captures the first image based on the first light received at the first image sensor. The device includes: a unit for receiving a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path towards the second image sensor, wherein the second image sensor captures the second image based on the second light received at the second image sensor, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element. The device includes: a unit for modifying at least one of the first image and the second image using perspective distortion correction. The device includes: a unit for generating a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0008] In some aspects, modifying at least one of the first image and the second image using the perspective distortion correction includes: using the perspective distortion correction to modify the first image from depicting a first perspective to depicting a common perspective; and using the perspective distortion correction to modify the second image from depicting a second perspective to depicting the common perspective, wherein the common perspective is between the first perspective and the second perspective.

[0009] In some aspects, modifying at least one of the first image and the second image using the perspective distortion correction includes: identifying a depiction of one or more objects in the image data of at least one of the first image and the second image; and modifying the image data by projecting the image data based on the depiction of the one or more objects.

[0010] In some aspects, generating the combined image from the first image and the second image, the one or more processors include: aligning a first portion of the first image with a second portion of the second image; and stitching the first image and the second image together based on the alignment of the first portion of the first image with the second portion of the second image.

[0011] In some aspects, the above methods, apparatuses, and computer-readable media further include: using brightness uniformity correction to modify at least one of the first image and the second image.

[0012] In some aspects, the above methods, apparatuses, and computer-readable media further include: the first image sensor; the second image sensor; the first light redirecting element; and the second light redirecting element.

[0013] In some aspects, the first light redirecting element includes a first reflective surface, wherein, in order to redirect the first light towards the first image sensor, the first light redirecting element uses the first reflective surface to reflect the first light towards the first image sensor; and the second light redirecting element includes a second reflective surface, wherein, in order to redirect the second light towards the second image sensor, the second light redirecting element uses the second reflective surface to reflect the second light towards the second image sensor.

[0014] In some aspects, the first light redirecting element includes a first prism configured to refract the first light; and the second light redirecting element includes a second prism configured to refract the second light. In some aspects, the first prism and the second prism are adjacent. In some aspects, a bridge connects a first edge of the first prism and a second edge of the second prism, wherein the bridge is configured to: prevent light from reflecting from at least one of the first edge of the first prism and the second edge of the second prism. In some aspects, the first prism includes at least one beveled edge, and wherein the second prism includes at least one beveled edge. In some aspects, the first prism includes at least one edge having a light absorbing coating, wherein the second prism includes at least one edge having a light absorbing coating. In some aspects, the first path is the path of the first light before the first light enters the first prism, wherein the second path is the path of the second light before the second light enters the second prism. In some aspects, the first prism includes a first reflective surface configured to reflect the first light, wherein the second prism includes a second reflective surface configured to reflect the second light. In some aspects, the first path is the path of the first light after the first light enters the first prism but before the first reflective surface reflects the first light, wherein the second path is the path of the second light after the second light enters the second prism but before the second reflective surface reflects the second light.

[0015] In some aspects, the first image and the second image are captured simultaneously. In some aspects, the first light redirecting element is fixed relative to the first image sensor, wherein the second light redirecting element is fixed relative to the second image sensor. In some aspects, a first flat surface of the first image sensor faces a first direction, wherein a second flat surface of the second image sensor faces a second direction parallel to the first direction.

[0016] In some aspects, the device includes a camera, a mobile handheld device, a smartphone, a mobile phone, a portable gaming device, another mobile device, a wireless communication device, a smartwatch, a wearable device, a head-mounted display (HMD), an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a personal computer, a laptop computer, a server computer, other devices, or a combination thereof. In some aspects, one or more processors include an image signal processor (ISP). In some aspects, the device includes one or more cameras for capturing one or more images. In some aspects, the device includes an image sensor that captures the image data. In some aspects, the device further includes a display for displaying images, one or more notifications associated with the processing of the images, and / or other displayable data.

[0017] This summary is not intended to identify key or essential features of the claimed invention, nor is it intended to be used alone to determine the scope of the claimed invention. The inventive subject matter should be understood by reference to the appropriate portions of the entire specification of this patent, any or all of the drawings, and each claim.

[0018] The foregoing and other features and embodiments will become more apparent by reference to the following specification, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Aspects of the present disclosure are illustrated by way of example and not limitation in the figures, and like reference numerals refer to like elements throughout. The illustrative embodiments of the present application are described in detail below with reference to the following figures, wherein:

[0020] Figure 1 is a conceptual diagram showing an example of distortion of an image captured by a camera using a lens with lens curvature.

[0021] Figure 2 is a conceptual diagram showing an example of a series of captured wide-angle images captured by a camera;

[0022] Figure 3 is a conceptual diagram showing an example of ghosting distortion in a wide-angle image generated using panoramic stitching;

[0023] Figure 4 is a conceptual diagram showing an example stitching distortion in a wide-angle image generated using panoramic stitching;

[0024] Figure 5 is a block diagram showing an example device configured to generate one or more wide-angle images;

[0025] Figure 6 is a conceptual diagram showing two image sensors of two cameras for capturing image frames and their associated lenses;

[0026] Figure 7 is a conceptual diagram showing an example redirecting element for redirecting light to a camera lens and the position changes of the camera lens and the associated image sensor based on the redirecting element;

[0027] Figure 8 is a conceptual diagram showing an example configuration of two cameras for generating a wide-angle image using a redirecting element including a mirror;

[0028] Figure 9 is a conceptual diagram showing an example configuration of two cameras for generating a wide-angle image using a redirecting element including a prism;

[0029] Figure 10A is a conceptual diagram showing an example perspective distortion in an image frame captured by one or more cameras;

[0030] Figure 10B is a conceptual diagram showing an example perspective distortion correction for bringing two image frames to a common perspective;

[0031] Figure 10C is a conceptual diagram showing an example digital alignment and stitching of two image frames captured by two cameras for generating a wide-angle image;

[0032] Figure 10D is a conceptual diagram showing an example brightness uniformity correction of a wide-angle image generated from two image frames captured by two cameras;

[0033] Figure 11 is a conceptual diagram showing an example light reflection from a camera lens that may cause scattering noise in a part of an image frame;

[0034] Figure 12A is a conceptual diagram showing an example redirecting element for redirecting light to a first camera and redirecting light to a second camera;

[0035] Figure 12B is showing Figure 12A a redirecting element in, which shows elimination of light scattering from the prism edge;

[0036] Figure 12C shows the Figure 12A conceptual diagram of the redirecting element from the perspective angle;

[0037] Figure 13A is a flowchart showing an example process for generating a composite image from multiple image frames;

[0038] Figure 13B is a flowchart showing an example method of digital imaging;

[0039] Figure 14 is a flowchart showing an example process for capturing multiple image frames to be combined to generate a composite image frame;

[0040] Figure 15 is a conceptual diagram showing examples of flat perspective distortion correction and curved perspective distortion correction;

[0041] Figure 16 is a conceptual diagram showing the pixel mapping from the image sensor image plane to the perspective-corrected image plane in flat perspective distortion correction and curved perspective distortion correction;

[0042] Figure 17 is a conceptual diagram showing three example composite images of a scene, each composite image having different degrees of curvature of curved perspective distortion correction applied;

[0043] Figure 18 is a conceptual diagram showing a graph comparing different degrees of curvature of curved perspective distortion correction relative to flat perspective distortion.

[0044] Figure 19 is a flowchart showing an example process for performing curved perspective distortion correction;

[0045] Figure 20 is a block diagram showing an example of the architecture of an image capture and processing device; and

[0046] Figure 21 is a block diagram showing an example of a system for implementing certain aspects of the present technology. DETAILED DESCRIPTION

[0047] Aspects of the present disclosure can be used in image or video capture devices. Some aspects include using multiple cameras to generate wide-angle images.

[0048] Smartphones, tablets, digital cameras, or other devices include a camera for capturing an image or video of a scene. The camera has a maximum field of view based on an image sensor and one or more camera lenses. For example, a single-lens or multi-lens system with a greater curvature in the camera lens can enable the image sensor to capture a larger field of view of the scene. Some devices include multiple cameras with different fields of view based on the curvature of the focusing lens. For example, a device may include a camera with a normal lens having a normal field of view and a different camera with a wide-angle lens having a wider field of view. A user of the camera or a software application running on the camera's processor can select between the fields of view of different cameras to choose the camera having the field of view most suitable for capturing a particular set of images or videos. For example, some smartphones include a telephoto camera, a wide-angle camera, and an ultra-wide-angle camera with different fields of view. Before capture, the user or software application can select which camera to use based on the field of view of each camera. Due to reliance on approximations, compensation for such distortion can be computationally expensive and inaccurate. Applying distortion compensation may retain some of the original distortion, may overcompensate, and / or may introduce other image artifacts.

[0049] However, an ultra-wide-angle camera may have a field of view less than the desired field of view of the scene to be captured. For example, many users desire to capture images or videos of a scene with a field of view greater than the field of view of the camera. Device manufacturers can increase the curvature of the camera lens to increase the field of view of the camera. However, the device manufacturer may also need to increase the size and complexity of the image sensor to accommodate the larger field of view.

[0050] In addition, lens curvature introduces distortion in the image frames captured from the camera. For example, lens curvature introduces radial distortion such as barrel distortion, pincushion distortion, or mustache distortion. In some cases, digital image processing can be used to perform software-based compensation for radial distortion by warping the distorted image using inverse distortion. However, software-based compensation for radial distortion can be difficult and computationally intensive to perform. In addition, software-based compensation typically relies on approximations and models that may not be applicable to all cases and may ultimately result in inaccurate or incomplete warping of the image. The resulting image after applying the compensation may still retain some radial distortion, may ultimately be distorted in the opposite way of the original image due to overcompensation, or may include other visual artifacts.

[0051] Systems and techniques for digital imaging to generate images with a large field of view are described. A device may include a first camera that captures a first image based on first light redirected by a first light redirecting element and a second camera that captures a second image based on second light redirected by a second light redirecting element. The first camera, the second camera, the first light redirecting element, and the second light redirecting element may be arranged such that a first lens of the first camera and a second lens of the second camera are virtually overlapped based on light redirection without physically overlapping. For example, a first center of a first entrance pupil of the first lens of the first camera and a second center of a second entrance pupil of the second lens of the second camera may be virtually overlapped without physically overlapping. The device may generate a combined image from the first image and the second image, such as by aligning and stitching the first image and the second image together. The combined image may have a wider field of view than the first image, the second image, or both.

[0052] The device does not rely on wide-angle lenses that increase lens curvature to generate a combined image of a wide-angle lens with a large field of view. Thus, the cameras in the device may use lenses that do not introduce the radial distortion introduced by wide-angle lenses and ultra-wide-angle lenses, in which case little or no radial distortion compensation needs to be applied. Thus, compared to using a camera with a curved lens that introduces radial distortion and a processor that subsequently compensates for that radial distortion to generate a comparable image, using the device to generate a combined image with a large field of view has a lower computational cost and is more accurate. Each camera in the device may also have a smaller and simpler image sensor, respectively, compared to the image sensor in a camera with a curved lens that introduces radial distortion. Thus, compared to a camera with a curved lens that introduces radial distortion, a single camera in the device may consume less power and require less power for processing.

[0053] Figure 1 Conceptual diagram 100 is an example showing the distortion of an image captured by camera 112 using lens 104 with a lens curvature. The distortion is based on the curvature of lens 104. Camera 112 includes at least lens 104 and image sensor 106. Lens 104 directs light from scene 102 to image sensor 106. Image sensor 106 captures one or more image frames. The captured image frame 108 is an example image frame depicting scene 102 and captured by image sensor 106 of camera 112. The captured image frame 108 includes barrel distortion, which is a type of radial distortion. The barrel distortion in the captured image frame 108 causes the center of scene 102 to appear stretched in the captured image frame 108 relative to the edges of the scene, while the corners of scene 102 appear squeezed towards the center of the captured image frame 108.

[0054] A device (such as camera 112 or another image processing device) can use distortion compensation to process the captured image frame 108 to reduce barrel distortion. However, this processing may introduce its own distortion effect on the captured image frame 108. For example, the center of the scene 102 in the captured frame 108 can be normalized or otherwise adjusted with reference to the edges of the scene in the captured image frame 108. Adjusting the center can include stretching the corners of the scene in the captured image frame 108 to more closely resemble a rectangle (or the shape of the image sensor, if different from a rectangle). In Figure 1 An example processed image frame 110 generated by processing the captured image frame 108 using distortion compensation is shown. The example processed image frame 110 shows an example where the distortion compensation overcompensates for the barrel distortion and introduces pincushion distortion, which is another type of radial distortion. Stretching the corners too much when processing the captured image frame 108 may introduce, for example, pincushion distortion. Using distortion compensation to process the image also introduces other image artifacts.

[0055] The lens curvature of the lens 104 can be increased to increase the field of view of the image frames captured by the image sensor 106. For example, wide-angle lenses, ultra-wide-angle lenses, and fish-eye lenses all typically exhibit a high level of lens curvature, which usually results in barrel distortion, other types of radial distortion, or other types of distortion. As a result, the distortion in each captured image frame 108 captured using such a lens increases, as shown by the barrel distortion in Figure 1 The possibility of distortion compensation introducing distortion or other image artifacts (such as the pincushion distortion shown in Figure 1 also increases with the increase in the curvature in the lens 104. Therefore, an image captured and / or generated using the lens 104 with increased lens curvature, which includes an image with a smaller field of view than desired (e.g., a cropped image), is typically distorted or includes artifacts.

[0056] Some devices also include software functions for generating an image with a wider field of view using a single camera based on the movement of the camera. For example, some camera applications include a camera movement panoramic stitching mode for generating an image with a wider field of view than the camera. For the camera movement panoramic stitching mode, the user moves the camera while the camera captures a series of image frames until all of the scene is included in at least one of these image frames. Then the image frames are stitched together to generate a wide-angle image.

[0057] Figure 2FIG. 200 is a conceptual diagram showing an example wide-angle image capture of a series of captured scenes 202 based on camera 206. User 204 wishes to capture an image of scene 202, but the field of view required to depict the entire scene 202 is greater than the field of view of camera 206. Accordingly, user 204 places camera 206 in a camera-moving panoramic stitching mode. User 204 positions camera 206 at a first position indicated by a dashed line in a first illustration of camera 206 such that the field of view of the camera is directed at scene portion 210. User 204 instructs camera 206 to begin image frame capture (e.g., by pressing a shutter button), and camera 206 captures a first image frame having scene portion 210. User 204 moves camera 206 (e.g., along camera movement arc 208) to move the field of view of the camera of scene 102 in direction 216. After capturing the first image frame, camera 206 captures a second image frame of scene portion 212 while camera 206 is in a second position indicated by a dashed line in a second illustration of camera 206. The second position of camera 206 is farther along direction 216 than the first position of camera 206. The second position of camera 206 is farther along camera movement arc 208 than the first position of camera 206. User 204 continues to move camera 206, and camera 206 captures a third image frame of scene portion 214 while camera 206 is in a third position indicated by a solid line in an illustration of camera 206. The third position of camera 206 is farther along direction 216 than the second position of camera 206. The third position of camera 206 is farther along camera movement arc 208 than the second position of camera 206. After translating camera 206 along camera movement arc 208 during image frame capture to capture image frames spanning scene 202, user 204 may stop image frame capture (e.g., by pressing the shutter button again or by releasing a shutter button that was continuously held during image frame capture). After capturing a series of image frames, camera 206 or another device may stitch the series of image frames together to generate a combined image of scene 102 having a wider field of view than each of the first image frame, the second image frame, and the third image frame. For example, the first image frame of scene portion 210, the second image frame of scene portion 212, and the third image frame of scene portion 214 (captured at different times) are stitched together to generate a combined image 202 depicting the entire scene, which may be referred to as a wide-angle image of the entire scene 202. Although three image frames are shown, two or more image frames may be captured and combined using the camera-moving panoramic stitching mode based on the desired field of view of the combined image.

[0058] For example, camera 206 or another device may recognize that both a first portion of a first image frame and a second portion of a second image frame depict a shared portion of scene 202. The shared portion of scene 202 is shown between two vertical dashed lines that fall within both a first scene portion 210 and a second scene portion 212. Camera 206 or other device may identify the shared portion of scene 202 within the first and second images by detecting features of the shared portion of scene 202 within the first and second images. Camera 206 or other device may align the first portion of the first image with the second portion of the second image. Camera 206 or other device may generate a combined image from the first and second images by stitching together the first portion of the first image and the second portion of the second image. Camera 206 may similarly stitch together the second image frame and the third image frame. For example, camera 206 or other device may identify a second shared portion of scene 202 depicted in a third portion of the third image frame and a fourth portion of the second image frame. Camera 206 or other device may stitch together the third portion of the third image frame and the fourth portion of the second image frame. Since a series of image frames are captured over a period of time as camera 206 moves along camera movement arc 208, Figure 2 the camera movement panoramic stitching mode shown in Figure 2 may be limited to generating still images rather than video because successive panoramic stitching combined images cannot be generated quickly enough to depict smooth motion. Additionally, the moving camera 206 and the passage of time during which a series of image frames are captured may introduce one or more distortions or artifacts into the generated images. Example distortions include ghosting distortion and stitching distortion. Ghosting distortion is an effect in which multiple instances of a single object may appear in the final image. Ghosting distortion may be the result of local motion within scene 202 during the capture of the series of image frames. Figure 3 An example of ghosting distortion is shown in Figure 3 . Stitching distortion is an effect in which edges may break, or objects may split, distort, overlap, etc., where two image frames are stitched together. Figure 4 An example of stitching distortion is shown in Figure 4 .

[0059] Distortion is also introduced by the changing depth of the scene by the entrance pupil of the camera as the camera moves. In other words, the moving camera changes the position of the camera entrance pupil relative to the scene. The entrance pupil associated with the image sensor is an image of the aperture in front of the camera (e.g., light is focused onto the image sensor by one or more lenses located in front of or at the aperture).

[0060] To keep the depth of an object in a scene from changing with reference to the movement of a camera between image captures, the camera needs to rotate about an axis centered on the camera's entrance pupil. However, when a person moves the camera, the person does not rotate the camera about an axis centered on the entrance pupil. For example, the camera can move about an axis at the torso of the person moving the camera (or rotation also includes translational movement). Since the camera rotation is not about an axis at the entrance pupil, the position of the entrance pupil changes between image frame captures, and the image frames are captured at different depths. Stitching distortion can be the result of parallax artifacts due to stitching together image frames captured at different depths. Stitching distortion can also be the result of global movement (which also includes changes in the camera's perspective when capturing a series of image frames).

[0061] Based on the change in the speed of movement of a user along a camera movement arc 208 of camera 206, distortion and artifacts can also be introduced into the composite image. For example, if the movement of camera 206 is fast, some of the image frames may include motion blur in some of the frames. Also, if the movement of camera 206 is fast, the shared portion of the scene depicted in two consecutive image frames may be very small, and distortion may be introduced due to poor stitching. If certain camera settings of camera 206 (such as focus or gain) change between image frame captures during the camera movement arc 208, distortion and artifacts can also be introduced into the composite image. Such changes in camera settings create visible seams between the images in the resulting composite image.

[0062] The figures shown herein depict each lens of each camera at the position of the camera's entrance pupil. For example, Figures 6 - 9 , Figure 11 and Figures 12A - 12C are the case. Although the camera lens is shown as a single camera lens in the figures to prevent confusion of aspects of the present disclosure, the camera lens can represent a single-element lens or a multi-element lens system of the camera. Additionally, the camera can have a fixed focal length, or the camera can be configured to autofocus (for which one or more camera lenses can move with reference to the image sensor). The present disclosure is not limited to the specific examples of the entrance pupil or its position depicted in the figures, or the specific examples of the camera lens or its position.

[0063] Figure 3 is a conceptual diagram 300 showing an example ghosting distortion 310 in a wide-angle image generated using panoramic stitching. Panoramic stitching can refer to Figure 2The panoramic stitching operation mode during camera movement. A device in the camera movement panoramic stitching mode generates an image 308 of a scene 302. The user positions the device such that the device's camera captures a first image frame including a first scene portion 304 at a first time. The user moves the device such that the device's camera captures a second image frame including a second scene portion 306 at a second time. Scene 302 includes a car moving from left to right in scene 302. As a result of the car moving in scene 302, the first image frame includes most of the car that is also included in the second image frame. When the two image frames are stitched together, the car may appear as multiple cars or multiple parts of a car in the resulting image 308 (shown as a ghosting distortion 310).

[0064] On the other hand, if the car in scene 302 moves from right to left instead of from left to right, then although the car is present in scene 302 during the capture of the first image frame and / or during the capture of the second image frame, the car may be at least partially omitted from image 308. For example, if the car is at least partially in the second scene portion 306 at the first time during the capture of the first image frame, the car may be at least partially omitted from the first image frame. If the car is at least partially in the first scene portion 304 at the second time during the capture of the second image frame, the car may be at least partially omitted from the second image frame. The combined image 308 may thus at least partially omit the car and, in some cases, may include more than one copy of the car that is partially omitted. This type of omission represents another type of distortion or image artifact that can be caused by camera movement panoramic stitching via the movement of camera 206 as Figure 2 shown.

[0065] Figure 4 is a conceptual diagram 400 showing an example stitching distortion 410 in a wide-angle image generated using panoramic stitching. Panoramic stitching may refer to Figure 2 the camera movement panoramic stitching operation mode in. Figure 4Also depicted is stitching distortion caused by parallax artifacts. A device in a camera motion panoramic stitching mode can generate a composite image 408 of a scene 402. A user positions the device such that the device's camera 206 captures a first image frame including a first scene portion 404 at a first time. The user moves the device such that the device's camera 206 captures a second image frame including a second scene portion 406 at a second time. Due to the movement of camera 206 between image frame captures (change in the position of the entrance pupil) and / or a change in the viewing angle of the first and second image frames of scene 402, there may be parallax-based and camera movement-based artifacts or distortions when the two image frames are stitched together. For example, composite image 408 is generated by stitching together the first image frame and the second image frame. As shown, there is a stitching distortion 410 where the left portion of the tree does not align with the right portion of the tree and the left portion of the ground does not align with the right portion of the ground. Although example stitching distortion 410 is shown as a lateral displacement between portions of the scene captured in the two image frames, stitching distortion 410 can also include rotational displacement or warping resulting from an attempt to align the image frames during stitching. In this way, lines that should be straight and unbroken in the scene may be broken at an angle in the final image, lines that should be straight may appear curved near the stitching, lines that should be straight may suddenly change direction near the stitching, or due to rotation, an object may appear distorted or warped on one side of the stitching compared to the other side. Distortion from stitching is exacerbated by a single camera moving over time to capture image frames. For example, in some cases, stitching distortion may cause an object in the scene to appear stretched, squeezed, tilted, skewed, warped, distorted, or otherwise inaccurate in composite image 408.

[0066] Another example of distortion is perspective distortion. Returning to Figure 2 when, the viewing angle of camera 206 is from the right side of scene portion 210, while the viewing angle of camera 206 is from the left side of scene portion 214. Thus, a horizontal edge (e.g., the horizon) may appear tilted in one direction in the first image frame, and the same horizontal edge (e.g., the horizon) may appear tilted in the opposite direction in the third image frame. The final image with the image frames stitched together can connect opposite tilted edges via an arc. For example, the horizon in a composite image generated using the camera motion panoramic stitching mode may appear curved rather than flat. This curvature is an example of perspective distortion. To exacerbate perspective distortion, the viewing angle changes based on camera movement, which may not be consistent between different instances of generating a wide-angle image via camera movement panoramic stitching. As a result, the camera viewing angle during the capture of a series of image frames may be different from the camera viewing angle during other series of captured image frames.

[0067] As described above, the distortion caused by increasing the lens curvature to increase the field of view degrades the quality of the resulting image, which has a negative impact on the user experience. In addition, the distortion caused by capturing a series of image frames over time (in the camera movement panoramic stitching mode) to generate a wide-angle image degrades the quality of the resulting image, which has a negative impact on the user experience. In addition, the camera movement panoramic stitching mode that requires capturing a series of image frames while the user manually moves the camera may prevent the camera from performing video capture, or may result in parallax artifacts that are difficult to remove due to camera movement. Therefore, there is a need for a unit for generating a wide-angle image (including a series of wide-angle images with a large field of view for video) that prevents or reduces the above-mentioned distortion.

[0068] In some examples of panoramic stitching, multiple cameras are used to capture image frames, which allows panoramic stitching to be performed without camera movement. The image frames captured by different cameras can be stitched together to generate a combined image with a field of view larger than that of any one of the multiple cameras. As used hereinafter, such a combined image (with a field of view larger than that of any one of the multiple cameras) is referred to as a wide-angle image. The multiple cameras can be placed such that the centers of their entrance pupils overlap (e.g., virtually overlap). In this way, there is no need to move the multiple cameras or the device including the multiple cameras (which may cause a change in the position of one or more entrance pupils). Therefore, no distortion caused by device movement is introduced into the generated wide-angle image. In some embodiments, the multiple cameras are configured to capture image frames concurrently and / or simultaneously. As used herein, the concurrent capture of image frames can refer to the simultaneous capture of image frames. As used herein, the concurrent and / or simultaneous capture of image frames can refer to at least a portion of the exposure window overlapping the corresponding image frames captured by the multiple cameras. As used herein, the concurrent and / or simultaneous capture of image frames can refer to at least a portion of the exposure windows of the corresponding image frames falling within a shared time window. The shared time window can have a duration of, for example, one or more picoseconds, one or more nanoseconds, one or more milliseconds, one or more centiseconds, one or more deciseconds, one or more seconds, or a combination thereof. In this way, there is no or less distortion in the generated wide-angle image caused by the passage of time in capturing a series of image frames.

[0069] In addition to overlapping the centers of the entrance pupils, the cameras can be placed relative to each other to capture the desired field of view of the scene. Since the positions of the cameras relative to each other are known, the device can be configured to reduce or remove perspective distortion based on the known positioning. In addition, since the images captured concurrently and / or simultaneously by the multiple cameras do not require each camera to be like Figure 2does not capture a series of image frames in the manner of the camera movement panoramic stitching mode of , so a device with multiple cameras can be configured to generate a wide-angle video including a series of wide-angle video frames. Each video frame can be a combined image generated by stitching together two or more images from two or more cameras.

[0070] In the following description, numerous specific details (such as examples of specific components, circuits, and processes) are set forth in order to provide a thorough understanding of the present disclosure. As used herein, the term "coupled" means directly connected or connected through one or more intermediate components or circuits. Additionally, in the following description and for purposes of explanation, specific terms are set forth in order to provide a thorough understanding of the present disclosure. However, those skilled in the art will appreciate that implementing the teachings disclosed herein may not require these specific details. In other instances, well-known circuits and devices are shown in block diagram form in order to avoid obscuring the teachings of the present disclosure. Some portions of the following detailed embodiments are presented in terms of programs, logic blocks, processing, and other symbolic representations of operations on data bits within a computer memory. In the present disclosure, programs, logic blocks, processes, etc. are conceived of as a self-consistent sequence of steps or instructions that result in a desired outcome. These steps are those requiring physical manipulation of physical quantities. Although not necessarily, typically these quantities take the form of electrical or magnetic signals that can be stored, transmitted, combined, compared, and otherwise manipulated in a computer system.

[0071] However, it should be borne in mind that all such and similar terms are to be associated with appropriate physical quantities and are merely convenient labels applied to those quantities. Unless specifically stated otherwise, it will be apparent from the following discussion that throughout this application, discussions using terms such as "access," "receive," "send," "use," "select," "determine," "normalize," "multiply," "average," "monitor," "compare," "apply," "update," "measure," "derive," "set," "generate," etc. refer to the actions and processes of a computer system or similar electronic computing device that manipulates and transforms data represented as physical (electronic) quantities within the registers and memories of the computer system into other data similarly represented as physical quantities within the memories or registers or other such information storage, transmission, or display devices of the computer system.

[0072] In the figures, a single block can be described as performing a function or functions; however, in actual practice, the function or functions performed by that block can be performed in a single component or multiple components, and / or can be performed using hardware, using software, or using a combination of hardware and software. To clearly represent this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps are described generally in terms of their functionality hereinafter. Whether such functionality is implemented as hardware or as software depends upon the particular application and the design constraints imposed on the overall system. Those of ordinary skill in the art can implement the described functionality in a variety of ways for each particular application, but such implementation decisions should not be construed as causing a departure from the scope of the present disclosure. Moreover, example devices can include components other than those shown, including well-known components such as processors, memories, and the like.

[0073] Aspects of the present disclosure are applicable to any suitable electronic device, including or coupled to a plurality of image sensors capable of capturing images or video (such as security systems, smartphones, tablets, laptop computers, digital video and / or still cameras, image capture device 2005A, image processing device 2005B, image capture and processing device 2000, computing system 2100, etc.). The terms “device” and “apparatus” are not limited to one or a particular number of physical objects (such as a single smartphone, a camera controller, a processing system, etc.). As used herein, a device can be any electronic device having one or more parts that can implement at least some portions of the present disclosure. Although the description and examples hereinafter use the term “device” to describe aspects of the present disclosure, the term “device” is not limited to a particular configuration, type, or number of objects. As used herein, an apparatus can include a device or a portion of a device for performing the described operations.

[0074] The depictions in the figures may not be drawn to scale or proportion, and embodiments may vary in size or dimension from those depicted in the figures. Some of the figures depict a camera lens indicating the entrance pupil of the camera. However, the camera and the entrance pupil can be positioned relative to each other (and the image sensor) in any suitable orientation to perform aspects of the present disclosure. The lenses depicted in the figures can indicate single-element lenses or multi-element lenses (even if the lenses may appear to be depicted as single-element lenses in the figures). Accordingly, the present disclosure is not limited to the examples explicitly depicted in the figures.

[0075] Figure 5FIG. 0 is a block diagram showing an example device 500 configured to generate one or more wide-angle images. The example device 500 includes (or is coupled to) cameras 501 and 502. Although two cameras are depicted, device 500 may include any number of cameras (e.g., 3 cameras, 4 cameras, etc.). The first camera 501 and the second camera 502 may be included in a single camera module or may be part of separate camera modules of device 500. In an example of a smartphone or a tablet, the first camera 501 and the second camera 502 may be associated with one or more apertures on the same side of the device to receive light for capturing image frames of a scene. The first camera 501 and the second camera 502 may be positioned relative to each other to allow capturing of a scene by generating a field of view that is wider than the field of view of the first camera 501 and / or the second camera 502 by combining images from camera 501 and camera 502. In some embodiments, device 500 includes (or is coupled to) one or more light redirecting elements 503. At least a first subset of the one or more light redirecting elements 503 may redirect light towards the first camera 501. At least a second subset of the one or more light redirecting elements 503 may redirect light towards the second camera 502. The first camera 501 may capture a first image based on the incident light redirected by the one or more light redirecting elements 503. The second camera 502 may capture a second image based on the incident light redirected by the one or more light redirecting elements 503. Device 500 may combine the first image and the second image to generate a combined image having a combined image field of view that is wider and / or larger than the first field of view of the first image, the second field of view of the second image, or both. The combined image may be referred to as a wide-angle image. The combined image field of view may be referred to as a large field of view, a wide field of view, or a combination thereof.

[0076] Device 500 may generate the combined image by combining the first image and the second image (e.g., by stitching the first image and the second image together without moving the first camera 501 and / or the second camera 502). For example, the device or another device may identify that a first portion of the first image captured by the first camera 501 and a second portion of the second image captured by the second camera 502 both depict a shared portion of the captured scene. Device 500 may identify the shared portion of the scene within the first image and the second image by detecting features of the shared portion of the scene within both the first image and the second image. Device 500 may align the first portion of the first image with the second portion of the second image. Device 500 may generate the combined image from the first image and the second image by stitching the first portion of the first image and the second portion of the second image together.

[0077] The first camera 501 and the second camera 502 can be proprietary cameras, dedicated cameras, or any type of camera. In some aspects, the first camera 501 and the second camera 502 can be cameras of the same type as each other. For example, the first camera 501 and the second camera 502 can be of the same brand and model. In some aspects, the first camera 501 and the second camera 502 can be cameras of different types, brands, and / or models. Although the examples below depict two similar cameras 501 and 502, any suitable number, type, or configuration of cameras can be used to implement aspects of the present disclosure. The first camera 501 and the second camera 502 can be used to receive and capture at least one spectrum, such as the visible light spectrum, the infrared light spectrum, the ultraviolet light spectrum, the microwave spectrum, the radio wave spectrum, the X-ray spectrum, the gamma ray spectrum, another subset of the electromagnetic spectrum, or a combination thereof.

[0078] The first camera 501, the second camera 502, and one or more redirecting elements 503 can be arranged such that the centers of the entrance pupils associated with the first camera 501 and the second camera 502 are virtually overlapped. For example, each camera includes an image sensor coupled to one or more lenses for focusing light onto the corresponding image sensor, and the lens and the entrance pupil are located at the same position of the camera. When using one or more redirecting elements 503, the first camera 501 and the second camera 502 can be arranged such that their lenses are virtually overlapped (e.g., the centers of their respective entrance pupils are virtually overlapped), while their lenses do not physically overlap or otherwise occupy the same space. For example, the light captured by the first camera 501 and the second camera 502 can be redirected (e.g., reflected and / or refracted) by one or more redirecting elements 503 such that the lenses of the first camera 501 and the second camera 502 can be physically separated while maintaining the virtual overlap of the lenses (e.g., the virtual overlap of the centers of the entrance pupils). Due to the virtual overlap of the centers of the entrance pupils associated with the cameras, the parallax effect between the image frames captured by different cameras 501 and 502 is reduced (or eliminated).

[0079] As used herein, virtual overlap can mean that if the light is not redirected (e.g., reference Figure 7The positions of a plurality of objects (such as camera lenses) are included. For example, the first lens of the first camera 501 and the second lens of the second camera 502 that are virtually overlapped may include a first virtual position of the first lens that overlaps with a second virtual position of the second lens. Before the first light redirecting element of the light redirecting element 503 redirects the first light ray away from the first path and towards the first camera 501, the first light ray travels along the first path. Before the second light redirecting element of the light redirecting element 503 redirects the second light ray away from the second path and towards the second camera 502, the second light ray travels along the second path. The virtual extension of the first path beyond the first light redirecting element intersects the first virtual position of the first lens. The virtual extension of the second path beyond the second light redirecting element intersects the second virtual position of the first lens.

[0080] The device 500 may also include one or more additional lenses, one or more apertures, one or more shutters, or other suitable components associated with the first camera 501 and the second camera 502. The device 500 may also include a flash, a depth sensor, or any other suitable imaging component. Although two cameras are shown as part of the device 500, the device 500 may include or be coupled to other image sensors not shown. In this way, wide-angle imaging may include using more than two cameras (such as three or more cameras). The two cameras are illustrated in the examples below to clearly explain the various aspects disclosed, but the present disclosure is not limited to the specific example of using two cameras.

[0081] The example device 500 also includes a processor 504, a memory 506 that stores instructions 508, and a camera controller 510. In some embodiments, the device 500 may include a display 514, a plurality of input / output (I / O) components 516, and a power supply 518. The device 500 may also include additional features or components not shown. In one example, a wireless interface that may include a number of transceivers and baseband processors may be included for a wireless communication device. In another example, one or more motion sensors (such as gyroscopes), position sensors (such as global positioning system sensors (GPS)), and sensor controllers may be included in the device.

[0082] The memory 506 can be a non-transitory or non-temporary computer-readable medium storing all or part of the computer-executable instructions 508 for performing one or more operations described in the present disclosure. In some embodiments, the instructions 508 include instructions for operating the device 500 in a wide-angle capture mode using the first camera 501 and the second camera 502. The instructions 508 may also include other applications or programs executed by the device 500, such as an operating system, a camera application, or other applications or operations executed by the device 500. In some examples, the memory 506 stores image frames (as a frame buffer) of the first camera 501 and / or the second camera 502.

[0083] In some examples, the memory 506 stores camera luminance uniform calibration data. Using the camera luminance uniform calibration data, the device 500 (e.g., the camera controller 510, the ISP 512, and / or the processor 504) can adjust the luminance level in the first image from the first camera 501 and / or the luminance level in the second image from the second camera 502. For example, the device 500 can remove vignetting or other luminance non-uniformities from the first image, the second image, or both. The device 500 can also increase or decrease the overall luminance of the first image, the second image, or both, such that the overall luminance matches between the first image and the second image. Such luminance adjustment can ensure that there are no visible seams in the combined image (e.g., between the portion from the first image in the combined image and the portion from the second image in the combined image). In some examples, the memory 506 stores perspective distortion correction data. The perspective distortion correction data can include data such as angles, distances, directions, amplitudes, distortion correction vectors, curvatures, or combinations thereof. Using the perspective distortion correction data, the device 500 (e.g., the camera controller 510, the ISP 512, and / or the processor 504) can perform perspective distortion correction (e.g., perspective distortion correction 1022, flat perspective distortion correction 1515, curved perspective distortion correction 1525, curved perspective distortion correction 1630).

[0084] The processor 504 can be one or more suitable processors capable of executing scripts or instructions of one or more software programs (e.g., the instructions 508) stored in the memory 506. In some aspects, the processor 504 can be one or more general-purpose processors executing the instructions 508. For example, the processor 504 can be an application processor and can execute a camera application. In some embodiments, the processor 504 is configured to instruct the camera controller 510 to perform one or more operations with reference to the first camera 501 and the second camera 502. In additional or alternative aspects, the processor 504 can include integrated circuits or other hardware for performing functions or operations without using software.

[0085] Although in Figure 5In the example shown, they are coupled to each other via processor 504. However, processor 504, memory 506, camera controller 510, optional display 514, and optional I / O component 516 can be coupled to each other in various arrangements. For example, processor 504, memory 506, camera controller 510, optional display 514, and / or optional I / O component 516 can be coupled to each other via one or more local buses (not shown for simplicity).

[0086] If device 500 includes display 514, then display 514 can be any suitable display or screen that allows user interaction and / or presents items for the user to view (such as images, videos, or preview images captured from one or more of first camera 501 and second camera 502). In some aspects, display 514 is a touch-sensitive display. Optional I / O component 516 can include any suitable mechanism, interface, or device for receiving input (such as commands) from the user and providing output to the user. For example, I / O component 516 can include a graphical user interface (GUI), keyboard, mouse, microphone, and speaker, squeezable bezels, one or more buttons (such as a power button), sliders, or switches.

[0087] Camera controller 510 can include image signal processor 512, which can be one or more image signal processors for processing the captured image frames provided by one or more cameras 501 and 502. In some example embodiments, camera controller 510 (such as image signal processor 512) can also control the operation of first camera 501 and second camera 502. For example, camera controller 510 (such as image signal processor 512) can receive instructions from processor 504 to perform wide-angle imaging, and camera controller 510 can initialize first camera 501 and second camera 502 and instruct first camera 501 and second camera 502 to capture one or more image frames, which camera controller 510 and / or processor 504 use panoramic stitching to combine into a combined image for wide-angle imaging. Camera controller 510 can control other aspects of first camera 501 and second camera 502, such as operations for performing one or more of automatic white balance, autofocus, or autoexposure operations.

[0088] In some aspects, the image signal processor 512 includes one or more processors configured to execute instructions from a memory (e.g., instructions 508 from memory 506, instructions stored in a separate memory coupled to the image signal processor 512, or instructions provided by processor 504). For example, the image signal processor 512 may execute instructions to process image frames from the first camera 501 and the second camera 502 to generate a wide-angle image. In addition to or instead of the image signal processor 512 including one or more processors configured to execute software, the image signal processor 512 may include specific hardware for performing one or more operations described in this disclosure. The image signal processor 512 may alternatively or additionally include a combination of specific hardware and the ability to execute software instructions.

[0089] Although the image signal processor 512 is described as part of the camera controller 510, the image signal processor 512 may be separate from the camera controller 510. For example, the camera controller 510 that controls the first camera 501 and the second camera 502 may be included in the processor 504 (e.g., embodied in instructions 508 executed by the processor 504 or embodied in one or more integrated circuits of the processor 504). The image signal processor 512 may be part of an image processing pipeline from an image sensor (for capturing image frames) to a memory (for storing image frames) and separate from the processor 504.

[0090] Although reference Figure 5 is made to the example device 500 in the following examples for performing wide-angle imaging or image capture, any suitable device or apparatus may be used. For example, a device for performing wide-angle imaging may be part of the device 500 (e.g., a component of a system-on-chip or an imaging processing pipeline). In another example, the device 500 may include a different configuration of the depicted components or additional components.

[0091] Device 500 is configured to generate one or more wide-angle images using a first camera 501 and a second camera 502. For example, the first camera 501 and the second camera 502 are configured to capture image frames, and the device 500 (such as an image signal processor 512) is configured to process the image frames to generate wide-angle images. As used herein, a wide-angle image refers to an image having a wider field of view than the first camera 501 or the second camera 502. When processing the image frames, the device 500 combines the image frames to generate a wide-angle image (which may also be referred to as a combined image). The first camera 501 and the second camera 502 may be positioned such that the centers of the associated entrance pupils are virtually overlapped. In this way, the parallax effect can be reduced or eliminated. The processing may also include reducing distortion in the image frames of the combined image (e.g., reducing perspective distortion based on the positional differences between the first camera 501 and the second camera 502, and non-uniform luminance distortion caused by the configuration of one or more camera lenses that focus light onto the image sensors of the cameras 501 or 502). In some embodiments, the first camera 501 and the second camera 502 may be configured to capture image frames concurrently and / or simultaneously. In this way, distortion caused by global motion or local motion can be reduced or eliminated. As described above, concurrently and / or simultaneously captured image frames may refer to at least a portion of the exposure windows of the image frames overlapping. The exposure windows may overlap in any suitable manner. For example, the start of frame (SOF) of the image frames may be coordinated, the end of frame (EOF) of the image frames may be coordinated, or there may be a time range during which all the image frames are within their exposure windows. As used herein, concurrent and / or simultaneous capture of image frames may refer to at least a portion of the exposure windows of the corresponding image frames falling within a shared time window. The shared time window may have a duration, for example, of one or more picoseconds, one or more nanoseconds, one or more milliseconds, one or more centiseconds, one or more deciseconds, one or more seconds, or a combination thereof.

[0092] In some embodiments, the first camera 501 and the second camera 502 are configured to: capture image frames such that the image sensors of the first camera 501 and the second camera 502 appear to be adjacent to each other. In some embodiments, the first camera 501 and the second camera 502 may be angled with respect to each other to capture different portions of a scene. For example, if a smartphone is in landscape mode, the first camera 501 and the second camera 502 may be horizontally adjacent and offset from each other by a certain angle. The first camera 501 may capture the right portion of the scene, while the second camera 502 may capture the left portion of the scene.

[0093] In some examples, the first camera 501, the second camera 502, or both are stationary. In some examples, the lens of the first camera 501, the lens of the second camera 502, or both are stationary. In some examples, the image sensor of the first camera 501, the image sensor of the second camera 502, or both are stationary. In some examples, each of one or more light redirecting elements 503 is stationary.

[0094] Figure 6 is a conceptual diagram 600 showing the first and second cameras. The first camera includes a first image sensor 602 and an associated first camera lens 606, which are shown using dashed lines in Figure 6 The first camera lens 606 is located at the entrance pupil of the first camera. The second camera includes a second image sensor 604 and an associated second camera lens 608, which are shown using solid lines in Figure 6 The second camera lens 608 is located at the entrance pupil of the second camera. As described above, although a camera lens may be described as a single lens, a camera lens may be a single-element lens or a multi-element lens system.

[0095] Conceptual diagram 600 may be an example of a conceptual configuration of the first camera 501 and the second camera 502 of device 500. The conceptual description of the overlapping lenses 606 and 608 shows that the entrance pupil of the first camera is virtually overlapping with the entrance pupil of the second camera. The overlapping entrance pupil centers reduce or eliminate the parallax of the image frames captured by different image sensors 602 and 604. The corresponding image frames from image sensors 602 and 604 may be combined to generate an image with a larger field of view than a single image frame. For example, the images may be stitched together. As described above, reducing or removing parallax reduces the number and effect of artifacts or distortions that may be present in the combined image.

[0096] In some embodiments, the field of view of the first image sensor 602 overlaps with the field of view of the second image sensor 604. For example, the right edge of the field of view of the first image sensor may overlap with the left edge of the field of view of the second image sensor.

[0097] Since the first image sensor 602 can capture the right portion of the scene in the wide-angle image and the second image sensor 604 can capture the left portion of the scene in the wide-angle image, a viewing angle of the wide-angle image can be generated between the viewing angle of the first image sensor 602 and the viewing angle of the second image sensor 604. The image sensors 602 and 604 are not parallel to each other, and the image frames captured by the image sensors 602 and 604 include perspective distortion relative to each other. To generate a wide-angle image with a viewing angle between the two viewing angles, the device 500 can perform perspective distortion correction on the image frames from both the image sensors 602 and 604 to generate image frames with a desired viewing angle. In some other embodiments, the device 500 can perform perspective distortion correction on the image frames from one image sensor to generate image frames with a viewing angle similar to that of the other image sensor. In this way, the wide-angle image can have the viewing angle of one of the image sensors.

[0098] In addition to reducing or removing parallax artifacts, the device 500 can use the configuration shown in the conceptual diagram 600 to more successfully reduce perspective distortion than using a single camera in a camera-motion panoramic stitching mode that relies on a physically moving single camera (as Figure 2 shown) or a camera lens with a greater curvature to increase the field of view. Since the cameras have fixed positions relative to each other, the angle between the image sensors 602 and 604 is static. Using Figure 6 the configuration shown, the device 500 can process the captured image frames to reduce angle-based perspective distortion. Since the angle is static, perspective distortion can be corrected digitally (e.g., during processing of the captured image frames). For example, the device 500 can perform perspective distortion correction as a predefined filter configured based on the angle between the image sensors 602 and 604 (e.g., in the image signal processor 512). In contrast, when moving between image frame captures, the angle between image sensor instances (for a camera-motion panoramic stitching mode that relies on Figure 2 a physically moving single camera as shown) can vary depending on the device movement. Thus, a device using a camera-motion panoramic stitching mode that relies on a physically moving single camera (as Figure 2 shown) cannot use a predefined filter based on a static angle to eliminate perspective distortion because no static angle exists. This makes compensating for perspective distortion in the combined image generated using camera-motion panoramic stitching that relies on a physically moving single camera as Figure 2 shown very difficult and computationally intensive. The device 500 having fixed positions for the first camera 501, the second camera 502, and / or one or more light redirecting elements 503 can thus perform perspective distortion correction more quickly, reliably, and with reduced computational overhead.

[0099] Return referenceFigure 6 , the first camera and the second camera may have the same focal length. In this way, the depth range of the focused scene is the same for the image sensors 602 and 604. However, the lenses 606 and 608 may not physically occupy the same space. In some embodiments, a prism and / or a reflective surface may be configured to perform the functions of two lenses with spatial overlap (without physical contact between the separate lenses). For example, the prism and / or the reflective surface may be shaped to direct light from a first direction to the first camera lens 606 and direct light from a second direction to the second camera lens 608, such that the virtual images of the entrance pupils associated with the camera lenses 606 and 608 overlap at their centers.

[0100] In some other embodiments, the cameras may be configured such that the centers of the entrance pupils overlap virtually, while the camera lenses of the cameras are spatially separated from each other. For example, one or more light redirecting elements may be used to redirect light to the camera lenses 606 and 608. Based on the characteristics and positions of the light redirecting elements, the first camera lens 606 may be spatially separated from the second camera lens 608 while the centers of the entrance pupils overlap virtually. In this way, the image sensors may still be configured to capture image frames that conform to Figure 6 the conceptual diagram 600 with overlapping camera lenses 606 and 608. In some embodiments, the first image sensor 602 may be associated with a first redirecting element, and the second image sensor 604 may be associated with a second redirecting element. In some implementations, the first redirecting element and the second redirecting element may be the same redirecting element (e.g., the redirecting element 1210 as in Figures 12A - 12C ).

[0101] As used herein, a redirecting element may be any suitable element configured to redirect light traveling along a first path to a second path. The redirecting element may reflect or refract light. In some embodiments, the redirecting element may include a mirror for reflecting light. As used herein, a mirror may refer to any suitable reflective surface (e.g., a reflective coating, a mirror glass, etc.).

[0102] Figure 7FIG. 700 is a conceptual diagram showing a redirecting element 706 that redirects light to an image sensor 702 and a change in the position of the image sensor 702 based on the redirecting element 706. As depicted, the redirecting element 706 may include a mirror for reflecting the received light toward the lens 704 (and the image sensor 702). The path of the light is represented by solid lines, where the arrow indicators indicate the direction of the light. If the redirecting element 706 is removed, omitted, or otherwise absent, the light would instead travel to the position of the virtual image sensor 708 (via the virtual entrance pupil of the virtual camera lens 710) along an extension of the original path of the light (shown using a dashed line) before the light is redirected by the light redirecting element 706. For example, returning to reference Figure 6 , the light directed to the second image sensor 604 is close to the position of the camera lens 608. Referring to Figure 7 , if a light redirecting element 706 is used to direct light through the camera lens 704 to the image sensor 702, the image sensor 702 is placed as Figure 7 shown, rather than at the position of the virtual image sensor 708, for the image sensor 702 to capture the same image frame. In this way, the position of the camera lens 704 is as Figure 7 shown, rather than at the position of the virtual camera lens 710. In this way, the lenses for multiple image sensors can be spatially separated while the lenses and / or the entrance pupils still virtually overlap.

[0103] For example, a first light ray follows an initial path 720 before reaching the light redirecting element 706 and is redirected onto a redirected path 722 that points to the camera lens 704 and the image sensor 702. The first light ray reaches the camera lens 704 and the image sensor 702 along the redirected path 722. The initial path 720 extends beyond the virtual extension 724 of the light redirecting element 706 and is shown as a dashed line and instead points to and reaches the virtual camera lens 710 and the virtual image sensor 708. The second and third light rays are also shown in Figure 7 . The light redirecting element 706 redirects the second and third light rays from their initial paths toward the camera lens 704 and the image sensor 702. The second and third light rays thus reach the camera lens 704 and the image sensor 702. The initial paths of the second and third light rays extending beyond the virtual extension of the light redirecting element 706 are shown as dashed lines and instead point to and reach the virtual camera lens 710 and the virtual image sensor 708.

[0104] The reflective surface (e.g., a mirror) of the redirecting element 706 may form a virtual image located behind the reflective surface (e.g., a mirror) of the redirecting element 706 (to the right of the redirecting element 706, as Figure 7 shown). The virtual camera lens 710 may be from Figure 7The direction of the initial path 720 depicted is a virtual image of the camera lens 704 observed through the reflective surface (e.g., a mirror) of the redirecting element 706. The virtual image sensor 708 can be from Figure 7 a virtual image of the image sensor 702 observed through the reflective surface (e.g., a mirror) of the redirecting element 706 in the direction of the initial path 720 depicted.

[0105] Figure 8 Conceptual diagram 800 is an example configuration of two cameras that shows the generation of a wide-angle image using redirecting elements 810 and 812. The first camera includes a first camera lens 806 (which can be one or more camera lenses) and a first image sensor 802. The second camera includes a second camera lens 808 (which can be one or more camera lenses) and a second image sensor 804.

[0106] Figure 8 The depiction 800 in can implement the same functions as Figure 6 conceptual diagram 600 in, where the first lens 806 and the second lens 808 are virtually overlapped (e.g., the centers of the entrance pupils of the camera lenses 806 and 808 are virtually overlapped), while being physically separated in space to remove or reduce parallax artifacts in the combined image from the image frames captured by the image sensors 802 and 804. When comparing the depiction 800 with Figure 6 conceptual diagram 600 in, the first image sensor 802 (associated with the first redirecting element 810) is configured to capture a portion of the scene, similar to the first image sensor 602. The second image sensor 804 (associated with the second redirecting element 812) is configured to capture other portions of the scene, similar to the second image sensor 604. Based on the use of the first redirecting element 810 and the second redirecting element 812, the first camera lens 806 is spatially separated from the second camera lens 808, and the first image sensor 802 is spatially separated from the second image sensor 804.

[0107] In some embodiments, the redirecting elements 810 and 812 may be located outside the device. For example, a component including the redirecting elements may be coupled to the device 500 to direct light through one or more openings in the device 500 to the image sensors of the first camera 501 and the second camera 502. In some examples, the device 500 may include redirecting elements disposed on the outer surface of the device 500. In some examples, the redirecting elements may be disposed inside the device. For example, the device may include one or more openings and / or apertures to allow light to enter the device (e.g., light from a scene is captured for generating a wide-angle image). The opening / aperture may include glass or another transparent material to allow light to pass through, and may be shaped as one or more lenses. The opening may or may not include one or more lenses or other components for adjusting the direction of light entering the device. The redirecting elements 810 and 812 may be positioned along the optical path between the device opening and the associated image sensor 802 or 804.

[0108] Although the redirecting elements 810 and 812 are shown as two separate mirrors, the redirecting elements 810 and 812 may be a single redirecting element. For example, the redirecting elements 810 and 812 may be physically connected on one side to form a single redirecting element. Additionally, the arrangement of the image sensors 802 and 804 is shown as being oriented towards each other. For example, the optical axes of the image sensors 802 and 804 may be aligned and / or may be parallel to each other. However, the image sensors and lenses may be arranged in any suitable manner to receive light from a desired field of view of the scene. For example, the optical axes of the image sensors 802 and 804 may not be aligned and / or may not be parallel to each other and / or may be at an angle to each other. The present disclosure is not limited to Figure 8 the component arrangements shown.

[0109] In some embodiments, the image sensors 802 and 804 are configured to concurrently and / or simultaneously capture image frames (e.g., at least a portion of the exposure windows overlap for the image frames). In this way, local and global motion are reduced (thereby reducing distortion in the generated wide-angle image). In some embodiments, the image sensors 802 and 804 are configured to concurrently, simultaneously, and / or within a shared time window capture image frames. The shared time window may have a duration of, for example, one or more picoseconds, one or more nanoseconds, one or more milliseconds, one or more centiseconds, one or more deciseconds, one or more seconds, or a combination thereof. Additionally, since the angle between the image sensors 802 and 804 is static, the device may be configured to reduce perspective distortion based on the known angle.

[0110] In some embodiments, the light to the first image sensor 802 and the light to the second image sensor 804 can be refracted (e.g., through a high refractive index medium) to reduce perspective distortion and / or vignetting at the camera aperture. Light propagating in a high refractive index material has a smaller divergence angle before leaving the medium, thereby reducing vignetting at the lens aperture located near the exit surface of the high refractive index medium. Refraction can alternatively or additionally be used to adjust the field of view of the image sensors 802 and 804. For example, the field of view can be widened to widen the field of view of a wide-angle image. In another example, the field of view can be shifted to allow for different spacings between the image sensors 802 and 804. Refraction can be used to allow for further physical separation between the camera lenses 806 and 808 while still allowing for the virtual overlap of the centers of the entrance pupils. For example, a prism can refract the light intended for the respective image sensors, and the prism can affect the position of the entrance pupil associated with the image sensor. Based on the refraction, an additional physical spacing between the camera lenses can be allowed while still allowing for the virtual overlap of the entrance pupil centers. In some embodiments, the redirecting element can include a prism. At least one surface of the prism can include a reflective surface, such as a mirror. In this way, one or more redirecting elements including a prism can be configured to refract and / or reflect the light directed towards the first image sensor 802 or the second image sensor 804.

[0111] Figure 9 FIG. 900 is a conceptual diagram showing an example configuration of two cameras and two redirecting elements 910 and 912. The two cameras are used to generate wide-angle images. The first camera includes a first image sensor 902 and a first camera lens 906. The second camera includes a second image sensor 904 and a second camera lens 908.

[0112] The redirecting elements 910 and 912 can include one or more prisms. Each prism can include a high refractive index medium (e.g., having a refractive index above a threshold). As depicted, the first redirecting element 910 redirects first light (e.g., including one or more light rays) from a first path proximate to the first redirecting element 910 to a redirected first path towards the first image sensor 902. The first path can be referred to as an initial first path. The second redirecting element 912 redirects second light (e.g., including one or more light rays) from a second path proximate to the second redirecting element 912 to a redirected second path towards the second image sensor 904. The second path can be referred to as an initial second path. The positions of the redirecting elements 910 and 912 can be as described with reference to Figure 8 above. For example, the redirecting elements 910 and 912 can be external to the device, or the redirecting elements 910 and 912 can be internal to the device and configured to receive light passing through an opening in the device.

[0113] InFigure 9 In [description], the first lens 906 may also represent the aperture and / or the position of the entrance pupil of the first camera. The second lens 908 may also represent the aperture and / or the position of the entrance pupil of the second camera. In illustration 900, the first redirecting element 910 includes a first prism, and the second redirecting element 912 includes a second prism. The first prism is configured to refract the first light going to the first image sensor 902 to redirect the first light from a first path close to the prism to a refracted first path. The second prism is configured to refract the second light going to the second image sensor 904 to redirect the second light from a second path close to the prism to a refracted second path. In some embodiments, the first redirecting element 910 further includes a first mirror on a side 918 of the first prism. The first mirror is configured to reflect the first light towards the first image sensor 902 by redirecting the first light from the refracted first path to a reflected first path. The second redirecting element 912 further includes a second mirror on a side 920 of the second prism. The second mirror is configured to reflect the second light towards the second image sensor 904 by redirecting the second light from the refracted second path to a reflected second path. After being reflected by the first mirror on the side 918, the first light exits the first prism 922.

[0114] Due to the refraction of the first prism 922, the first light may be redirected from the reflected first path to a post-prism first path when exiting the first prism 922. Similarly, after being reflected by the second mirror on the side 920, the second light exits the second prism 922. Due to the refraction of the second prism 924, the second light may be redirected from the reflected second path to a post-prism second path when exiting the second prism 924.

[0115] In some examples, the first light may also be redirected (e.g., via refraction) from the post-prism first path to a post-lens first path by the first lens 906. In some examples, the second light may also be redirected (e.g., via refraction) from the post-prism second path to a post-lens second path by the second lens 908. In this way, each redirecting element 910 and 912 may include a prism, where one side of the prism includes a reflective coating. Light passing through the prism and reaching the reflective coating is reflected or refracted back to the corresponding image sensor. In some other embodiments, the redirecting element may include separate reflective and refractive components. For example, the first mirror or the second mirror may be separate components distinct from the first prism and the second prism, respectively.

[0116] As used herein, a prism can refer to any suitable light-refracting object, such as a glass or plastic prism of a suitable shape. Suitable shapes can include triangular prisms, hexagonal prisms, etc., the surface angles of which are configured to refract light from a scene as needed. In some embodiments, the redirecting element includes an equilateral triangular prism (or other suitable edged triangular prism for refracting light). In illustration 900, side 922 of the first redirecting element 910 is generally aligned in the same plane as side 924 of the second redirecting element. The prism can be configured such that each camera includes a field of view of approximately 70 degrees (a field of vision having an angle of approximately 70 degrees). In some embodiments, sides 922 and 924 are coated with an anti-reflection coating to prevent reflected light from being captured by image sensors 902 and 904. In some embodiments, the prism surfaces facing the camera lenses are also coated with an anti-reflection coating to prevent light from reflecting from these surfaces.

[0117] In some examples, the first path after the lens can be referred to as the redirected first path. In some examples, the first path after the prism can be referred to as the redirected first path. In some examples, the reflected first path can be referred to as the redirected first path. In some examples, the refracted first path can be referred to as the redirected first path. In some examples, the second path after the lens can be referred to as the redirected second path. In some examples, the second path after the prism can be referred to as the redirected second path. In some examples, the reflected second path can be referred to as the redirected second path. In some examples, the refracted second path can be referred to as the redirected second path. In some examples, the first path approaching the prism can be referred to as the first path or the initial first path. In some examples, the refracted first path can be referred to as the first path or the initial first path. In some examples, the second path approaching the prism can be referred to as the second path or the initial second path. In some examples, the refracted second path can be referred to as the second path or the initial second path.

[0118] The first prism or the second prism can be configured to: refract light from a portion of a scene to adjust the focal length. For example, the first prism and the second prism can be shaped such that the incident and exit angles of light of the prism allow the associated camera lenses 906 and 908 to be in different positions while still having the same effect as Figure 6 conceptual diagram 600 therein. In this way, the lenses can be spatially separated while the centers of the entrance pupils still virtually overlap (as Figure 6 shown). The virtual overlap of the centers of the entrance pupils of the first lens 906 and the second lens 908 (shown as the actual overlap of the entrance pupils of the first virtual lens 926 and the second virtual lens 928) can provide the following technical benefits: when the entrance pupils do not have an image Figure 9In the case of virtual overlap as in [reference], reduce or eliminate parallax artifacts that might otherwise be present (and pose technical problems) in the combined image. For example, as a result of redirecting elements 910 and 912, if the first redirecting element 910 were absent, the first image sensor 902 could be conceptualized as a first virtual image sensor 914, and if the second redirecting element 912 were absent, the second image sensor 904 could be conceptualized as a second virtual image sensor 914. Similarly, if the redirecting elements 910 and 912 were absent, the lenses 906 and 908 could be conceptualized as virtual lenses 926 and 928. The overlapping virtual lenses 926 and 928 indicate overlapping entrance pupils, as Figure 6 shown.

[0119] The first virtual lens 926 can be conceptualized as having the virtual position, orientation, and / or pose that the first lens 906 would have in order to receive the first light that the first lens 906 actually receives if the first light continued along the virtual extension of its first path (extending beyond the first redirecting element 910) rather than being redirected by at least a portion of the first redirecting element 910 towards the first lens 906 and the first image sensor 902. The second virtual lens 928 can be conceptualized as having the virtual position, orientation, and / or pose that the second lens 908 would have in order to receive the second light that the second lens 908 actually receives if the second light continued along the virtual extension of its second path (extending beyond the second redirecting element 912) rather than being redirected by at least a portion of the second redirecting element 912 towards the second lens 908 and the second image sensor 904.

[0120] Similarly, the first virtual image sensor 914 can be conceptualized as having the virtual position, orientation, and / or pose that the first image sensor 902 would have in order to receive the first light that the first image sensor 902 actually receives if the first light continued along the virtual extension of its first path rather than being redirected by at least a portion of the first redirecting element 910 towards the first lens 906 and the first image sensor 902. The second virtual image sensor 916 can be conceptualized as having the virtual position, orientation, and / or pose that the second image sensor 904 would have in order to receive the second light that the second image sensor 904 actually receives if the second light continued along the virtual extension of its initial second path rather than being redirected by at least a portion of the second redirecting element 912 towards the second lens 908 and the second image sensor 904.

[0121] In some examples, the distance between the first redirecting element 910 and the first lens 906 is equal to the distance between the first redirecting element 910 and the first virtual lens 926. In some examples, the distance between the first redirecting element 910 and the first image sensor 902 is equal to the distance between the first redirecting element 910 and the first virtual image sensor 914. In some examples, the distance between the second redirecting element 912 and the second lens 908 is equal to the distance between the second redirecting element 912 and the second virtual lens 928. In some examples, the distance between the second redirecting element 912 and the second image sensor 904 is equal to the distance between the second redirecting element 912 and the second virtual image sensor 916.

[0122] In some examples, the optical distance between the reflective surface 918 of the first redirecting element 910 and the first lens 906 is approximately equal to the optical distance between the reflective surface of the first redirecting element 910 and the first virtual lens 926. In some examples, the optical distance between the reflective surface of the first redirecting element 910 and the first image sensor 902 is approximately equal to the optical distance between the reflective surface of the first redirecting element 910 and the first virtual image sensor 914. In some examples, the optical distance between the reflective surface of the second redirecting element 912 and the second lens 908 is approximately equal to the optical distance between the reflective surface of the second redirecting element 912 and the second virtual lens 928. In some examples, the optical distance between the reflective surface of the second redirecting element 912 and the second image sensor 904 is approximately equal to the optical distance between the second reflective surface of the redirecting element 912 and the second virtual image sensor 916.

[0123] Identifying the virtual positions, orientations, and / or poses corresponding to the first virtual lens 926, the second virtual lens 928, the first virtual image sensor 914, and the second virtual image sensor 916 may include conceptually removing or omitting at least a portion of the first redirecting element 910 and at least a portion of the second redirecting element 912, such as conceptually removing the reflective surface (e.g., mirror) on the side 918 of at least the first prism, the reflective surface (e.g., mirror) on the side 920 of the second prism, the first prism itself, the second prism itself, or combinations thereof. The antecedent path of the first light may include the path of the first light before entering the first prism or the path of the first light after the first light enters the first prism but before the first light is redirected by the reflective surface (e.g., mirror) on the side 918 of the first prism. The antecedent path of the second light may include the path of the second light before entering the second prism or the path of the second light after the second light enters the second prism but before the second light is redirected by the reflective surface (e.g., mirror) on the side 920 of the second prism.

[0124] The first virtual lens 926 can be referred to as the virtual lens of the first lens 906, the virtual position of the first lens 906, the virtual orientation of the first lens 906, the virtual pose of the first lens 906, or a combination thereof. The second virtual lens 928 can be referred to as the virtual lens of the second lens 908, the virtual position of the second lens 908, the virtual orientation of the second lens 908, the virtual pose of the second lens 908, or a combination thereof. The first virtual image sensor 914 can be referred to as the virtual image sensor of the first image sensor 902, the virtual position of the first image sensor 902, the virtual orientation of the first image sensor 902, the virtual pose of the first image sensor 902, or a combination thereof. The second virtual image sensor 916 can be referred to as the virtual image sensor of the second image sensor 904, the virtual position of the second image sensor 904, the virtual orientation of the second image sensor 904, the virtual pose of the second image sensor 904, or a combination thereof. Based on refraction, the spacing between the first camera lens 906 and the second camera lens 908 can be less than Figure 8 the spacing between the first camera lens 806 and the second camera lens 808 in Figure 8 where the light redirecting element may not refract light. Similarly, the spacing between the first image sensor 902 and the second image sensor 904 can be less than

[0125] the spacing between the first image sensor 802 and the second image sensor 804 in Figure 9 The reflective surface (e.g., a mirror) on the side 918 of the first redirecting element 910 can form a virtual image behind the reflective surface (e.g., a mirror) located on the side 918 of the first redirecting element 910 (as Figure 9 shown below and to the right of the first redirecting element 910). The reflective surface (e.g., a mirror) on the side 920 of the second redirecting element 912 can form a virtual image behind the reflective surface (e.g., a mirror) located on the side 920 of the second redirecting element 912 (as Figure 9 shown below and to the left of the second redirecting element 912). Figure 9 depicts that the first virtual lens 926 can be a virtual image of the first lens 906, as observed from the direction in which light approaches the first redirecting element 910 through the reflective surface (e.g., a mirror) on the side 918 of the first redirecting element 910. Figure 9Depicted therein, the second virtual lens 928 can be a virtual image of the second lens 908, as observed from the direction in which light approaches the second redirecting element 912 through a reflective surface (e.g., a mirror) on the side surface 920 of the second redirecting element 912. Figure 9 Depicted therein, the second virtual image sensor 916 can be a virtual image of the second image sensor 904, as observed from the direction in which light approaches the second redirecting element 912 through a reflective surface (e.g., a mirror) on the side surface 920 of the second redirecting element 912.

[0126] In some embodiments, the first prism and the second prism are physically separated from each other (e.g., by a distance of 1 / 2 millimeter (mm)). The spacing can be to prevent damage to the prisms caused by the prisms colliding with each other. In some other embodiments, the first prism and the second prism can be physically connected. For example, the first prism and the second prism can be connected at one of their corners such that the first redirecting element 910 and the second redirecting element 912 are the same redirecting element having a plurality of prisms and mirrors for refracting and reflecting light for the first image sensor 902 and the second image sensor 904.

[0127] Similar to that described above with reference to Figure 8 perspective distortion can be reduced by digitally performing perspective distortion correction on the captured image frames. The image frames (with corrected distortion) can be combined (e.g., digitally) by the device to generate a wide-angle image (which can also be referred to as a combined image). Similar to Figure 8 , the image sensors 902 and 904 can be configured to concurrently and / or simultaneously capture image frames, and / or capture image frames within a shared time window to reduce distortion from motion or other distortion in the combined image.

[0128] As described above, the image frames captured by the image sensors 802, 804, 902, or 904 can include perspective distortion. However, since the perspectives captured by the image sensors 802, 804, 902, and 904 are known and static, perspective distortion compensation techniques can be consistently applied to each image captured by each of the image sensors 802, 804, 902, and 904 in some cases.

[0129] Figure 10A is a conceptual diagram 1000 showing an example of perspective distortion in an image frame 1006 captured by an image sensor 1004. The image sensor 1004 can be Figure 8 or Figure 9Implementation of any image sensor. As shown, the image sensor 1004 captures the scene 1002 at an angle perpendicular to the scene 1002. A lens (not shown) may be located between the scene 1002 and the image sensor 1004. The lens can be any lens, such as the first camera lens 606, the second camera lens 608, the camera lens 704, the first camera lens 806, the second camera lens 808, the first lens 906, the second lens 908, the first lens 1106, the second lens 1108, the first lens 1206, the second lens 1208, the lens 1660, the lens 2015, or another lens. Since the right side portion of the scene 1002 is closer to the image sensor 1004 than the left side portion of the scene 1002, the captured image frame 1006 includes perspective distortion. The perspective distortion is shown as the right side portion of the scene 1002 in the image frame 1006 looking larger than the left side portion of the scene 1002 in the image frame 1006. Since the angle of the image sensor 1004 relative to another image sensor is known (e.g., between the image sensors 602 and 604 in the Figure 6 conceptual illustration), the device 500 (e.g., the image signal processor 512) can perform perspective distortion correction 1022 to generate the processed image 1008. The device 500 can use the perspective distortion correction 1022 to modify the captured image frame 1006 to generate the processed image 1008. For example, during the perspective distortion correction 1022, the device 500 can map the trapezoidal region of the captured image frame 1006 to a rectangular region (or vice versa), which can be referred to as trapezoidal perspective distortion correction, trapezoidal projective transformation, or trapezoidal distortion. In some cases, the perspective distortion correction 1022 can be referred to as perspective distortion, perspective transformation, projective distortion, projective transformation, transformation, warping, or some combination thereof.

[0130] When capturing scene 1002, image sensor 1004 may also capture areas outside of scene 1002 (such as shown by the white triangle in image frame 1006 from the sensor). In some embodiments of perspective distortion correction 1022, device 500 processes the captured image frame 1006 such that the resulting processed image 1008 only includes the illustrated portion of scene 1002, without additional captured scene information in the captured image frame 1006. Device 500 takes the left portion of the captured image frame 1006, which includes the illustrated portion of scene 1002 (excluding the additional portions of the captured scene above and below scene 1002 shown by the white triangle), and adjusts the remaining portion of the captured image frame 1006 to the left portion of scene 1002 in the captured image frame 1006 to generate image 1008. The portion taken from the left of the captured image frame 1006 (corresponding to the illustrated portion of scene 1002) may be based on the field of view of the image sensor, the common perspective to which the captured image frame 1006 is to be adjusted, and the perspective of another image sensor capturing different portions of the captured scene, not shown. For example, based on two perspectives of the camera, the common perspective, and the field of view, device 500 may use the range of image pixels in the left column of the image pixels of the captured image frame 1006 for the processed image 1008.

[0131] Similarly, the portion taken from the right of the image frame 1006 (corresponding to the illustrated portion of scene 1002) may be based on the field of view of the image sensor, the common perspective to which the image frame 1006 is to be adjusted, and the perspective of another image sensor capturing different portions of the captured scene, not shown. For example, based on two perspectives of the camera, the common perspective, and the field of view, device 500 may use the range of image pixels in the right column of the image pixels of the captured image frame 1006 for the processed image 1008. In the example captured image frame 1006, all pixels in the rightmost column of the captured image frame 1006 include information from the illustrated portion of scene 1002 (indicating that the white triangle representing the additional portion of the captured scene in the captured image frame 1006 ends at the right column of the image pixels in the image frame 1006).

[0132] As shown in the figure, the illustrated portion of scene 1002 skews in image frame 1006 from a smaller range of image pixels in the left column of image pixels of image frame 1006 to a larger range of image pixels in the right column of image pixels of image frame 1006. When moving from left to right through the columns of image pixels, the rate at which the number of pixels in this range increases can be linear (device 500 can determine this rate of increase based on a linear regression of the pixel range, which is based on the definition mapping of the column or pixel range per column). In this way, the image pixels in the columns of image pixels of image frame 1006 that will be used for processed image 1008 can be based on a mapping of the pixel column to the distances from the left column and the right column. For example, if image frame 1006 includes scene information of 100 pixels in 100 columns for image 1008 and the left column includes scene information of 50 pixels for image 1008, then the 50th column can include scene information of approximately 75 pixels for image 1008 (0.5 * 50 + 0.5 * 100). Additionally, the pixels of the scene information to be used for processed image 1008 can be centered around the center of the columns of image frame 1006. Continuing with the previous example, the 50th column can include 12 or 13 pixels at the bottom of the column that will not be used and can include 13 or 12 pixels at the top of the column that will not be used.

[0133] Based on the desired common view of the composite image, the device can use the selected pixels of the scene information to adjust the pixel values of the captured image frame (e.g., image frame 1006) to generate a processed image 1008. The device 500 can generate a composite image in response to the modification of the captured image frame 1006 to generate the processed image 1008. Adjusting the pixel values causes the parallel horizontal lines in the scene 1002 (which are shown as being inclined to each other in the image frame 1006 due to perspective distortion) to be parallel again in the image 1008. To adjust the pixel values of the image 1008 (such that, in the example, the horizontal lines are parallel in the image 1008), the device 500 can "stretch" the pixel values in the image frame 1006 to cover multiple pixels. For example, stretching the pixel values in the image frame 1006 to cover multiple pixel values in the processed image 1008 can include: using the pixel values at multiple pixel positions in the image 1008. Conversely, the device 500 can combine multiple pixel values in the image frame 1006 for fewer pixel values in the image 1008 (e.g., by taking an average or other combining means). The perspective distortion correction 1022 process based on merging or filtering (e.g., taking an average, median filtering, etc.) can be applied to the pixel values to adjust the captured image of the scene 1002 in the image frame 1006 to generate the processed image 1008. In the example, the process is shown as being performed in the vertical direction. However, the process can also be applied in the horizontal direction to prevent the scene 1002 from appearing stretched in the processed image 1008. Although some example filters for perspective distortion correction 1022 are described, any suitable filter can be used to combine the pixel values to generate the processed image 1008 in the correction of perspective distortion. As a result of perspective distortion correction, the processed image 1008 can be smaller or larger than the image frame 1006 (in terms of the number of pixels) in the horizontal and / or vertical directions.

[0134] While the above-described embodiments describe determining a portion of an image frame to be adjusted during perspective distortion correction, in some embodiments, one or more image sensors may be configured to adjust readings for an image frame based on perspective distortion correction. For example, image sensor 1004 may be configured to read out from particular image sensor pixels (e.g., image sensor pixels that exclude scene information in the white triangle that captures image frame 1006). In some embodiments, the device may be configured to adjust which rows (or portions of rows) of the image sensor's pixels are to be read out based on the portion of scene 1002 to be included in the processed image 1008. Perspective distortion may then be performed on the image frame that includes only a subset of the pixel data from image sensor 1004. The perspective distortion function may be based on the number of pixels read out from the image sensor. Since the image frames from the two cameras include perspective distortion of the expected perspective of the reference combined image, device 500 may perform perspective distortion correction on the image frames from the two cameras.

[0135] Figure 10B FIG. 1020 is a conceptual diagram showing an example perspective distortion correction 1022 of a common perspective of two image frames 1024 into a combined image 1026. As shown by the two image frames 1024, the first image and the second image have perspective distortions that are opposite to each other. Device 500 is used to correct the perspective distortion (e.g., as described above) of each of the first image and the second image to a common perspective (e.g., as shown in combined image 1026) using perspective distortion correction 1022. After correcting the perspective distortion, device 500 may stitch the corrected image 1 and the corrected image 2 to generate a combined (wide-angle) image.

[0136] The stitching can be any suitable stitching process for generating a combined image. In some embodiments, the field of view of the first camera 501 overlaps with the field of view of the second camera 502. For example, the first camera 501, the second camera 502, and one or more redirecting elements 503 are arranged such that the field of view overlap is from 1 / 2 degree to 5 degrees. After correcting the perspective distortion, the device 500 uses the overlapping portions in the captured frames from the two cameras 501 and 502 to align and combine the two image frames to generate a combined image. Due to the overlap, the device 500 can reduce stitching errors based on aligning the captured image frames. In some embodiments, the device 500 can compensate for changes in the overlap over time (e.g., if the device 500 drops or collides, repeated temperature changes cause displacement of one or more components, etc.). For example, at the time of device production, the overlap may start at 5 degrees, but over time, the overlap may increase to 7 degrees. The device 500 can use object detection and matching in the overlapping scene portions of the two image frames to align the image frames and generate a combined image (instead of using a static merging filter based on a fixed overlap and arrangement). By aligning and matching objects in the overlapping scene portions of the two image frames, the device 500 can stitch the image frames together with any overlap (as long as it has a sufficient size, e.g., 1 / 2 degree) to generate a combined image.

[0137] Figure 10C is a conceptual diagram 1040 showing an example digital alignment and stitching 1042 of two image frames captured by two cameras for generating a wide-angle image. To illustrate the operation of the digital alignment and stitching, the scene is described as two instances of the English alphabet (from A-Z twice). The correct instance of the alphabet in the scene is shown with each of its letters circled. The left instance of the alphabet in the scene has no circles around its letters. Camera 1 (e.g., the first camera 501) captures the left instance of the alphabet in the scene. Camera 2 (e.g., the second camera 502) captures the right instance of the alphabet in the scene. The overlapping fields of view of the two cameras may cause the two cameras to capture (the letter "A" with a circle). The overlap is based on the angle between the two cameras (e.g., by the Figure 9as shown by the virtual lenses and image sensors of the lens 906 and sensor 902 of one of the cameras and the lens 908 and sensor 904 of the other camera. The device 500 performs digital alignment and stitching 1042 by using object or scene recognition and matching towards the right edge of the image frame of camera 1 and matching towards the left edge of the image frame of camera 2 to align the matching objects / scenes. Alignment can include referencing another image frame to shift and / or rotate one or both image frames to overlap the pixels between the image frames until the matching objects or parts of the scene overlap. In the case where the image frames are aligned based on the matching objects / scenes, the two image frames are stitched together to generate a digitally aligned and stitched image (which can include saving the shifted and / or rotated image frames together as a combined image). Stitching can include averaging the overlapping image pixel values, selecting one of the image pixel values as the combined image pixel value, or otherwise mixing the image pixel values.

[0138] In addition to reducing stitching distortion and reducing perspective distortion, the device 500 can reduce non-uniform brightness distortion in the combined image. One or more camera lenses can be configured to image a scene onto an image sensor. The relative illuminance of the image formed by the lens can follow I(θ) = Io × cos 4 the low or minimum value of θ, where θ is the angle between the incident light ray and the lens normal, Io is a constant, and I(θ) is the illuminance of the image pixel illuminated by the incident light at the angle θ. Light perpendicular to the lens (θ = 0) will be focused at the center of the sensor, and light at the maximum angle (e.g., θ = 30°) will be focused at the edge of the sensor). Thus, the image brightness at the edge is cos 4 (30°) = 0.56. Additionally, a light redirecting component (such as Figure 8 a mirror in Figure 9The prism in ) may introduce vignetting, which may further reduce the brightness of the image pixels near the edges. Therefore, more light may reach the center of the image sensor than the edges of the image sensor. The light reaching the edges of the image sensor (and especially the corner pixels) may not be as much as the light reaching the center of the image sensor. Therefore, the image frames captured from the first camera 501 and the second camera 502 may have non-uniform brightness across the image pixels. Vignetting or other brightness non-uniformities in the first image frame from the first camera 501 and / or the second image frame from the second camera 502 may result in visible seams in the combined image generated by combining the first image and the second image. After capture (such as before or after correcting the perspective distortion and / or before or after stitching the image frames together), the device 500 may correct the brightness non-uniformities of the image frames of the combined image. For example, the device 500 may adjust the brightness in the first image frame from the first camera 501 to eliminate vignetting from the first image, and may adjust the brightness in the second image frame from the second camera 502 to eliminate vignetting from the second image, or both. The device 500 may perform these brightness adjustments before the device 500 combines the first image and the second image to generate the combined image. Removing vignetting through such brightness adjustments can ensure that there are no visible seams in the combined image (e.g., between the portion of the combined image from the first image and the portion of the combined image from the second image).

[0139] In addition, in some cases, the first camera 501 and the second camera 502 may receive unequal amounts of light, may process light and / or image data in different ways (e.g., due to differences in camera hardware and / or software), and / or may be mis-calibrated. Unequal brightness levels or another image property between the first image frame from the first camera 501 and the second image frame from the second camera 502 may result in visible seams in the combined image generated by combining the first image with the second image. In some examples, the device 500 may increase or decrease the brightness in the first image frame from the first camera 501, may increase or decrease the brightness in the second image frame from the second camera 502, or both. The device 500 may perform these brightness adjustments before the device 500 combines the first image and the second image to generate the combined image. Such brightness adjustments can ensure that there are no visible seams in the combined image (e.g., between the portion of the combined image from the first image and the portion of the combined image from the second image). Figure 10D is a conceptual diagram 1060 showing an example brightness uniformity correction 1062 of a wide-angle image generated from two image frames captured by two cameras. The brightness uniformity correction 1062 may correct the above for Figure 10CThe vignetting or other luminance non-uniformities discussed. FIG. 1064 shows the relative illuminance of an image sensor based on the illuminance at the center of the image sensors of the first camera 501 and the second camera 502. The center of each image sensor is illuminated the most (represented by positioning the center of the image sensor at a 30-degree angle from the center of the combined image). The angle between the incident light and the normal to the top surface of the prism (e.g., the side surfaces 922 and 924 in Figure 9 and the side surface 1220 in Figures 12A - 12C discussed herein) can be measured. In some examples, the lens can be tilted 30 degrees relative to the normal of the top surface of the prism, such as Figure 9 indicated by the angles of the first virtual lens 926 and the second virtual lens 928 in

[0140] 30-degree incident light can be perpendicular to the lens and can thus be focused at the center of the sensor and have the maximum illuminance / brightness in the resulting image. If each image sensor has a 70-degree field of view, the fields of view of the two image sensors may overlap by 10 degrees. As moving from the center of the image sensor (e.g., the centers corresponding to -30 degrees and 30 degrees respectively in FIG. 1064) to the edges of the image sensor (e.g., the edges represented by 0 in the middle of FIG. 1064 and the two ends of FIG. 1064), the illuminance of the image sensor decreases. Although FIG. 1064 is shown along one axis of the image sensor for illustrative purposes, FIG. 1064 can include additional dimensions or can be drawn in other ways to indicate the illuminance variation based on a two-dimensional image sensor.

[0141] In some implementations, device 500 increases the brightness of image pixels in an image frame (e.g., increases the luminance value in the YUV color space or similarly increases the RGB values in the RGB color space). The amount by which the brightness of the image pixels is increased can be a fraction of dividing the current luminance value by the illuminance between the associated image sensor pixel and the center of the image sensor (e.g., based on FIG. 1064). In this way, the brightness of each image pixel can be increased to be similar to the brightness of the image pixels at the center of the image sensor (as shown in FIG. 1066).

[0142] Device 500 can thus use one or more redirecting elements 503 to direct light to the first camera 501 and the second camera 502 for image frame capture to generate a combined image including corrected perspective distortion, reduced stitching artifacts, and reduced luminance distortion (non-uniform luminance).

[0143] Some implementations of one or more redirecting elements and cameras may result in scattered noise in the combined image.

[0144] Figure 11 Conceptual diagram 1100 shows an example light reflection from the first camera lens 1106 that may cause scattered noise in a portion of the image frame. The first camera includes a first image sensor 1102 and a first camera lens 1106. The first camera can be Figure 9An embodiment of the first camera (including the first image sensor 902 and the first camera lens 906). The first redirecting element 1110 is located outside the first camera to direct light towards the first image sensor 1102. As shown, light received on one side of the first redirecting element 1110 is refracted by the first prism of the first redirecting element 1110, reflected by the first mirror on the side 1112 of the first prism, and directed towards the camera lens 1106. The first camera lens 1106 can reflect a small portion of the light back towards the first prism by Fresnel reflection. The light received towards the top of the image sensor 1102 represents the remainder of the light that is allowed to pass through the lens 1106. The light reflected by the first camera lens 1106 is sent back through the first prism towards the upper right edge of the prism. The upper right edge of the first prism can be referred to as the edge of the first prism that is closest to the second prism of the second redirecting element 1120. The first prism and / or the second prism can include a high refractive index medium (e.g., having a refractive index higher than a threshold). Although not shown, one or more edges of the prism of the redirecting element can be chamfered (to reduce breakage). The upper right edge of the prism (which can be chamfered) can reflect and scatter the light from the camera lens 1106 back towards the camera lens 1106, and the camera lens 1106 can direct the light towards the bottom end of the image sensor 1102. In this way, light intended for a portion of the image sensor 1102 may be incorrectly received by a different portion of the image sensor 1102. Light received in an unexpected location in the image sensor 1102 may cause the first camera to capture an image frame with distorted brightness in the form of scattered noise and associated image artifacts. Although scattered noise is shown only for the first camera (with the first lens 1106 and the first image sensor 1102) and the first redirecting element 1110, scattered noise can occur for the second camera (with the second lens 1108 and the second image sensor 1104) and the second redirecting element 1120. Additionally, scattered noise may appear in the portion of the image sensor corresponding to the overlapping field of view of the cameras. Thus, the combined image may include scattered noise near the stitching line or on one side of the combined image. This may result in a visible stitching line in the combined image, which is undesirable as it disrupts the continuity of the image data in the combined image.

[0145] One or more redirecting elements 503 are configured to prevent redirecting light back towards the camera lens from the camera lens. For example, the redirecting element 1110 can be configured to prevent reflecting light from the camera lens 1106 back towards the camera lens 1106 (and similarly for other redirecting elements). In some embodiments, a portion of one or more edges of the prism is prevented from scattering light. In the process of preventing these portions from scattering light, one or more chamfered edges of the prism are prevented from scattering light. For example, in Figure 11In the example of, the light absorption coating can be applied to the upper right beveled edge of the prism. In some embodiments, one or both of the other two corner edges of the prism (which are not in the Figure 11 optical path shown and can be beveled or not beveled) can also be coated with a light absorption coating to prevent light from scattering from the surfaces at these locations. In this way, the light received at the upper right edge of the left prism in Figure 11 is absorbed and does not scatter towards the camera lens 1106 and the sensor 1102. In some examples, the light absorption coating can be opaque. In some examples, the light absorption coating can be black, dark gray, or dark-colored.

[0146] In some other embodiments, to reduce the scattered noise caused by reflections from the camera lens and subsequently scattered by the prism edges, the first redirecting element and the second redirecting element can be combined into a single redirecting element such that the upper right corner of the left prism and the upper left corner of the right prism are effectively eliminated (physically do not exist).

[0147] Figure 12A is a conceptual diagram 1200 showing an example redirecting element 1210 for redirecting light to a first camera and for redirecting light to a second camera. The first camera includes a first image sensor 1202 and a first camera lens 1206, and the first camera can be an Figure 9 example implementation of the first camera in. The second camera includes a second image sensor 1204 and a second camera lens 1208, and the second camera can be an Figure 9 example implementation of the second camera in. For example, the viewing angle Theta of both cameras can be 70 degrees.

[0148] The redirecting element 1210 includes a first prism 1212 for refracting the light intended for the first image sensor 1202, and a second prism 1214 for refracting the light intended for the second image sensor 1204. The first mirror can be on the side 1216 of the first prism 1212, and the second mirror can be on the side 1218 of the second prism 1218 (similar to Figure 9the redirecting elements 910 and 912). The first prism 1212 and / or the second prism 1218 may include a high refractive index medium (e.g., having a refractive index above a threshold). The first prism 1212 and the second prism 1214 are adjacent. The first prism 1212 and the second prism 1214 are physically connected and / or joined and / or bridged at the top of the sides 1216 and 1218. For example, the prisms 1212 and 1214 are connected to overlap at the top edges of the two prisms. For example, the edge of the first prism 1212 closest to the second prism 1214 overlaps and joins with the edge of the second prism 1214 closest to the first prism 1212. In some embodiments, the overlapping portion of the prisms 1212 and 1214 may have a height of 1 / 2 mm to 1 mm of the redirecting element 1210. The overlapping portion of the prisms 1212 and 1214 may be referred to as a bridge connecting the first prism 1212 and the second prism 1214.

[0149] In this way, light received near the center of the side 1220 of the redirecting element can be reflected towards the first image sensor 1202 or the second image sensor 1204 based on which of the sides 1216 or 1218 receives the light. The light reflected back towards the redirecting element 1210 by the camera lenses 1206 and 1208 does not hit the prism corner edge (as Figure 11 shown), because there is no prism corner edge in the redirecting element 1210.

[0150] In some embodiments of manufacturing the redirecting element 1210, an injection mold of a desired shape (e.g., including two adjacent / overlapping triangular or equilateral triangular prisms) is filled with plastic having a desired refractive index. After creating the plastic element of the desired shape, reflective coatings are applied to two surfaces (e.g., sides 1216 and 1218) of the plastic element. In some embodiments, an anti-reflective coating is applied to the top side to receive light from the scene (e.g., side 1220). The anti-reflective coating may also be applied to the sides of the prism facing the camera lenses 1206 and 1208. In some embodiments, the near side and the far side of the redirecting element 1210 also include non-reflective and / or light-absorbing coatings. In some examples, the coating may be opaque. In some examples, the coating may be black, dark gray, or dark-colored. As the apex angles of the prisms 1212 and 1214 are closest to overlapping with each other, the camera can be placed to ensure that the virtual centers of the first lens 1206 and the second lens 1208 virtually overlap while remaining physically separated, as Figure 9 shown (e.g., as Figure 9 shown, the center of the first entrance pupil of the first lens 1206 and the center of the entrance pupil of the second lens 1208 overlap). In some embodiments, the orientation of the camera is the same as Figure 9be the same or similar to ensure that the scene in the central stitching area of the combined image of the two images captured by the image sensors 1202 and 1204 overlaps by 0.5 - 5 degrees. Although not shown in Figure 12A (or other embodiments of the prism of the redirecting element), one or more corner edges may be chamfered to prevent cracking.

[0151] Although not shown in Figure 12A a virtual lens corresponding to the first lens 1206 and the second lens 1208, it should be understood that the position of such a virtual lens will be similar to Figure 9 the positions of the first virtual lens 926 and the second virtual lens 928 of Figure 12A Although not shown in Figure 9 a virtual image sensor corresponding to the first image sensor 1202 and the second image sensor 1204, it should be understood that the position of such a virtual image sensor will be similar to Figure 12A the positions of the first virtual image sensor 914 and the second virtual image sensor 916 of Figure 12A Although not shown in Figure 9 the virtual extensions of the previous paths of the first light and the second light towards the virtual lens and the virtual image sensor beyond the first prism 1212 and the second prism 1214, it should be understood that

[0152] Figure 12B is a conceptual diagram 1240 showing the Figure 12A redirecting element in Figure 11 which shows the elimination of light scattering from the prism edges (e.g., as shown in Figure 12BThe prism 1212 in FIG. 1 shows a reduction in light scattering, but the same light scattering reduction may occur for the second prism 1214 with respect to light reflected by the second camera lens 1208 associated with the second image sensor 1204. Since the reflected light leaves the redirecting element 1210 on the side 1220, the use of Figures 12A - 12C The redirecting element 1210 and the overlapping prisms 1212 and 1214 shown reduce or eliminate the need for Figure 11 Thus, the use of redirecting element 1210 and overlapping joined prisms 1212 and 1214 improves the image quality of both images captured using image sensors 1202 and 1204 individually and combined images generated by stitching together the images captured by image sensors 1202 and 1204. In addition, the prisms 1212 and 1214 are overlapped and joined in redirecting element 1210, which has the additional benefit of ensuring that prisms 1212 and 1214 can be accurately positioned relative to each other and are not misaligned relative to each other without requiring additional hardware to control the relative position of prisms 1212 and 1214 relative to each other.

[0153] Figure 12C It shows the viewing angle Figure 12A 1260 of a redirecting element in FIG. 1260. Light redirecting element 1210 is shown between a first camera and a second camera. The first camera includes a first lens 1206, which is hidden from view based on the perspective in conceptual illustration 1260, but is still shown using dashed lines. The second camera includes a second lens 1208, which is hidden from view based on the perspective in conceptual illustration 1260. Light redirecting element 1210 includes a first prism 1212 and a second prism 1214. The first prism 1212 and the second prism 1214 are adjacent. The edge of the first prism 1212 closest to the second prism 1214 is joined to the edge of the second prism 1214 closest to the first prism 1212. Side 1216 of the first prism 1212 includes a reflective coating. Side 1218 of the second prism 1218 includes a reflective coating. Light redirecting element 1210 includes a side 1220 that is hidden from view based on the perspective in conceptual illustration 1260, but is still pointed to using dashed lines.

[0154] In some cases, the first prism 1212 may be referred to as a first light redirecting element, and the second prism 1214 may be referred to as a second light redirecting element. In some cases, the edge of the first light redirecting element physically overlaps and engages with the edge of the second light redirecting element. In some cases, the edge of the first prism physically overlaps and engages with the edge of the second prism. In some cases, the first side 1216 (having a reflective surface) of the first prism 1212 may be referred to as a first light redirecting element, while the first side 1218 (having a reflective surface) of the second prism 1214 may be referred to as a second light redirecting element. The light redirecting element 1210 may be referred to as a single light redirecting element, where the first light redirecting element and the second light redirecting element are two different parts of the single light redirecting element.

[0155] As shown above, one or more redirecting elements can be used to direct light from a scene towards multiple cameras. The multiple cameras capture image frames to be combined to generate a wide-angle image. For example, the wide-angle image includes less distortion caused by lens curvature and can have a wider viewing angle than other individual cameras for wide-angle imaging.

[0156] Before, concurrently, simultaneously, and / or after combining the first image frame and the second image frame to generate a combined image, the device 500 can perform other processing filters on the combined image or the captured image frames. For example, the image frames can have different color temperatures or light intensities. Other example processing can include imaging processing filters performed during an image processing pipeline, such as denoising, edge enhancement, etc. After processing the image, the device 500 can store the image, output the image to another device, output the image to the display 514, and so on. In some embodiments, a series of wide-angle images can be generated when creating a wide-angle video. For example, the image sensor concurrently and / or simultaneously captures a series of image frames, and the device 500 processes the associated image frames (as described for each image frame in the series of image frames) to generate a sequence of combined images for the video. Refer to the following Figure 13A 、 Figure 13B and Figure 14 for example methods of generating a combined image. Although these methods are described as being performed by the device 500 and / or by an imaging system, any suitable device can be used to perform the operations in the examples.

[0157] Figure 13AFIG. 1300 is a flow chart illustrating an example process for generating a combined image from multiple image frames. In some examples, the operations in process 1300 may be performed by an imaging system. In some examples, the imaging system is device 500. In some examples, the imaging system includes at least one of the following: camera 112, camera 206, device 500, the imaging architectures shown in conceptual diagram 600, the imaging architectures shown in conceptual diagram 700, the imaging architectures shown in conceptual diagram 800, the imaging architectures shown in conceptual diagram 900, the imaging architectures shown in conceptual diagram 1100, the imaging architectures shown in conceptual diagram 1200, the imaging architectures shown in conceptual diagram 1240, the imaging architectures shown in conceptual diagram 1260, the imaging architectures shown in conceptual diagram 1600, at least one of the following image capture and processing systems 2000, image capture device 2005A, image processing device 2005B, image processor 2050, host processor 2052, ISP 2054, computing system 2100, one or more web servers of a cloud service, or a combination thereof.

[0158] At operation 1302, the imaging system may receive a first image frame of a scene captured by a first camera 501. For example, after the first camera 501 captures the first image frame (including a first portion of the scene), the image signal processor 512 may receive the first image frame. The first portion of the scene may be one side of the scene. At 1304, device 500 may also receive a second image frame of the scene captured by a second camera 502. For example, after the second camera 502 captures the second image frame (including a second portion of the scene), the image signal processor 512 may receive the second image frame. The second portion of the scene may be the other side of the scene.

[0159] At operation 1306, the imaging system may generate a combined image from the first image frame and the second image frame. The combined image includes a wider field of view than the field of view of the first image frame or the field of view of the second image frame. For example, the first image frame and the second image frame may be stitched together (as described above). In some embodiments, the overlap in the edges of the scene captured in the image frames is used to stitch the first image frame and the second image frame.

[0160] Based on the virtual overlap of the centers of the entrance pupils of the first camera 501 and the second camera 502 that capture the first image frame and the second image frame based on one or more redirecting elements 503 (e.g., Figure 8 , Figure 9 or Figures 12A - 12C the redirecting elements in), the combined image may have a reduced or removed parallax effect. In this way, the lenses or other components do not physically overlap, while the centers of the entrance pupils virtually overlap. In some embodiments, the image frames are captured concurrently and / or simultaneously by cameras 501 and 502 to reduce distortion caused by local or global motion.

[0161] Although Figure 13A not shown in, the imaging system can continue to process the combined image, including performing denoising, edge enhancement, or any other suitable image processing filter in the image processing pipeline. The resulting combined image can be stored in memory 506 or another suitable memory, can be provided to another device, can be displayed on display 514, or can be used in any other suitable manner.

[0162] Figure 13B is a flowchart showing an example method 1350 of digital imaging. In some examples, the operations in process 1300 can be performed by an imaging system. In some examples, the imaging system is device 500. In some examples, the imaging system includes one or more of the following: camera 112, camera 206, device 500, the imaging architectures shown in conceptual diagram 600, the imaging architectures shown in conceptual diagram 700, the imaging architectures shown in conceptual diagram 800, the imaging architectures shown in conceptual diagram 900, the imaging architectures shown in conceptual diagram 1100, the imaging architectures shown in conceptual diagram 1200, the imaging architectures shown in conceptual diagram 1240, the imaging architectures shown in conceptual diagram 1260, the imaging architectures shown in conceptual diagram 1600, at least one of the following: an image capture and processing system 2000, an image capture device 2005A, an image processing device 2005B, an image processor 2050, a host processor 2052, an ISP 2054, a computing system 2100, one or more web servers of a cloud service, or a combination thereof.

[0163] At operation 1355, the imaging system receives a first image of a scene captured by a first image sensor. A first light redirecting element redirects first light from a first path to a redirected first path toward the first image sensor. The first image sensor captures the first image based on the first light received at the first image sensor. In some examples, the imaging system includes the first image sensor and / or the first light redirecting element. In some examples, the first image sensor is part of a first camera. The first camera can also include a first lens. In some examples, the imaging system includes the first lens and / or the first camera.

[0164] Examples of the first image sensor for operation 1355 include image sensor 106, the image sensor 206 of a camera, the image sensor 501 of a first camera, the image sensor 502 of a second camera, the first image sensor 602, the second image sensor 604, image sensor 702, the first image sensor 802, the second image sensor 804, the first image sensor 902, the second image sensor 904, image sensor 1004, the first image sensor 1102, the second image sensor 1104, the first image sensor 1202, the second image sensor 1204, image sensor 2030, another image sensor described herein, or a combination thereof. Examples of the first lens for operation 1355 include lens 104, the lens 206 of a camera, the lens 501 of a first camera, the lens 502 of a second camera, the first camera lens 606, the second camera lens 608, camera lens 704, the first camera lens 806, the second camera lens 808, the first lens 906, the second lens 908, the first lens 1106, the second lens 1108, the first lens 1206, the second lens 1208, lens 1660, lens 2015, another lens described herein, or a combination thereof. Examples of the first light redirecting element for operation 1355 include light redirecting element 706, the first light redirecting element 810, the second light redirecting element 812, the first light redirecting element 910, the second light redirecting element 912, the first prism of the first light redirecting element 910, the second prism of the second light redirecting element 912, the first reflective surface on side 918 of the first light redirecting element 910, the second reflective surface 920 on side 920 of the second light redirecting element 912, the first light redirecting element 1110, the second light redirecting element 1112, the first prism of the first light redirecting element 1110, the second prism of the second light redirecting element 1112, the first reflective surface on side 1112 of the first light redirecting element 1110, the second reflective surface of the second light redirecting element 1112, light redirecting element 1210, the first prism 1212 of light redirecting element 1210, the second prism 1214 of light redirecting element 1210, the first reflective surface on side 1216 of light redirecting element 1210, the second reflective surface on side 1218 of the second light redirecting element 1212, another prism described herein, another reflective surface described herein, another light redirecting element described herein, or a combination thereof.

[0165] At operation 1360, the imaging system receives a second image of the scene captured by the second image sensor. The second light redirecting element redirects the second light from the second path to a redirected second path toward the second image sensor. The second image sensor captures the second image based on receiving the second light at the second image sensor. A virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element. In some examples, the imaging system includes the second image sensor and / or the second light redirecting element. In some examples, the second image sensor is part of a second camera. The second camera may also include a second lens. In some examples, the imaging system includes the second lens and / or the second camera.

[0166] Examples of the second image sensor for operation 1360 include image sensor 106, the image sensor 206 of a camera, the image sensor 501 of a first camera, the image sensor 502 of a second camera, the first image sensor 602, the second image sensor 604, image sensor 702, the first image sensor 802, the second image sensor 804, the first image sensor 902, the second image sensor 904, image sensor 1004, the first image sensor 1102, the second image sensor 1104, the first image sensor 1202, the second image sensor 1204, image sensor 2030, another image sensor described herein, or a combination thereof. Examples of the second lens for operation 1360 include lens 104, the lens 206 of a camera, the lens 501 of a first camera, the lens 502 of a second camera, the first camera lens 606, the second camera lens 608, camera lens 704, the first camera lens 806, the second camera lens 808, the first lens 906, the second lens 908, the first lens 1106, the second lens 1108, the first lens 1206, the second lens 1208, lens 1660, lens 2015, another lens described herein, or a combination thereof. Examples of the second light redirecting element for operation 1360 include light redirecting element 706, the first light redirecting element 810, the second light redirecting element 812, the first light redirecting element 910, the second light redirecting element 912, the first prism of the first light redirecting element 910, the second prism of the second light redirecting element 912, the first reflective surface on the side 918 of the first light redirecting element 910, the second reflective surface on the side 920 of the second light redirecting element 912, the first light redirecting element 1110, the second light redirecting element 1112, the first prism of the first light redirecting element 1110, the second prism of the second light redirecting element 1112, the first reflective surface on the side 1112 of the first light redirecting element 1110, the second reflective surface on the side 1118 of the second light redirecting element 1112, light redirecting element 1210, the first prism 1212 of light redirecting element 1210, the second prism 1214 of light redirecting element 1210, the first reflective surface on the side 1216 of light redirecting element 1210, the second reflective surface on the side 1218 of the second light redirecting element 1212, another prism described herein, another reflective surface described herein, another light redirecting element described herein, or a combination thereof.

[0167] In some examples, the first lens and the second lens are virtually overlapped. In some examples, although the first lens and the second lens are virtually overlapped, the first lens and the second lens do not physically overlap, do not spatially overlap, are physically separated and / or are spatially separated. For example, Figure 9The first lens 906 and the second lens 908 do not physically overlap, do not spatially overlap, are physically separated, and are spatially separated. Nevertheless, the first lens 906 and the second lens 908 are virtually overlapped because the first virtual lens 926 (the virtual position of the first lens 906) overlaps with the second virtual lens 928 (the virtual position of the second lens 908). Although the virtual lens positions of the first lens 1106 and the second lens 1108 are not shown in Figure 11 , the first lens 1106 and the second lens 1108 can also be virtually overlapped (e.g., the virtual lens position of the first lens 1106 can overlap with the virtual lens position of the second lens 1108). The first lens 1106 and the second lens 1108 do not physically overlap, do not spatially overlap, are physically separated, and are spatially separated. Although the virtual lens positions of the first lens 1206 and the second lens 1208 are not shown in Figures 12A - 12C , the first lens 1206 and the second lens 1208 can also be virtually overlapped (e.g., the virtual lens position of the first lens 1206 can overlap with the virtual lens position of the second lens 1208). The first lens 1206 and the second lens 1208 do not physically overlap, do not spatially overlap, are physically separated, and are spatially separated.

[0168] The first light redirecting element can include a first reflective surface. Examples of the first reflective surface can include the reflective surface of the redirecting element 706, the reflective surface 810 of the first light redirecting element, the reflective surface on the side 918 of the first light redirecting element 910, the reflective surface on the side 1112 of the first light redirecting element 1110, the reflective surface on the side 1216 of the light redirecting element 1210, another reflective surface described herein, or a combination thereof. To redirect the first light toward the first image sensor, the first light redirecting element uses the first reflective surface to reflect the first light toward the first image sensor. Similarly, the second light redirecting element can include a second reflective surface. Examples of the second reflective surface can include the reflective surface of the redirecting element 706, the reflective surface 812 of the second light redirecting element, the reflective surface on the side 920 of the second light redirecting element 912, the reflective surface on the side of the second light redirecting element 1120 closest to 1112 of the first light redirecting element 1110, the reflective surface on the side 1218 of the light redirecting element 1210, another reflective surface described herein, or a combination thereof. To redirect the second light toward the second image sensor (e.g., the second image sensor 904 / 1204), the second light redirecting element uses the second reflective surface to reflect the second light toward the second image sensor. The first reflective surface can be or can include a mirror. The second reflective surface can be or can include a mirror.

[0169] The first light redirecting element can include a first prism configured to refract the first light. The second light redirecting element can include a second prism configured to refract the second light. In some examples, the first prism and the second prism are adjacent (e.g., as Figures 12A - 12C shown). For example, the first prism and the second prism can be made of a single piece of plastic, glass, crystal, or other material. A bridge can connect a first edge of the first prism and a second edge of the second prism. For example, in Figures 12A - 12C , an edge of the first prism between side 1220 and side 1216 is connected by a bridge to an edge of the second prism between side 1220 and side 1218. The bridge can be configured to prevent light from reflecting from at least one of the first edge of the first prism and the second edge of the second prism. For example, as Figures 12A - 12C shown, a bridge connecting two prisms can prevent scattering from the Figure 11 prism angles shown and labeled in

[0170] The first prism can include at least one beveled edge. For example, in the first redirecting element 910 of Figure 9 , the edge between side 922 and side 918 can be beveled. Figure 11 The corresponding edge of the first prism in the first redirecting element 1110 of Figure 9 can be beveled. The second prism can include at least one beveled edge. For example, in the second redirecting element 912 of Figure 11 , the edge between side 924 and side 920 can be beveled. Figure 9 The corresponding edge of the second prism in the second redirecting element 1120 of Figure 11 can be beveled. The first prism can include at least one edge having a light absorbing coating. For example, in the first redirecting element 910 of Figures 12A - 12C , the edge between side 922 and side 918 can have a light absorbing coating. Figure 9 The corresponding edge of the first prism in the first redirecting element 1110 of Figure 11 can have a light absorbing coating. The corresponding edge of the first prism 1212 in the redirecting element 1210 of Figures 12A - 12C (e.g., at and / or near the bridge connecting the first prism 1212 to the second prism 1214) can have a light absorbing coating. The second prism can include at least one edge having a light absorbing coating. For example, in the second redirecting element 912 ofThe corresponding edges of the second prism 1214 in the redirecting element 1210 (e.g., at and / or near the bridge connecting the first prism 1212 and the second prism 1214) may have a light-absorbing coating. The light-absorbing coating may be paint, lacquer, a material, or another type of coating. The light-absorbing coating may be opaque. The light-absorbing coating may be reflective or non-reflective. The light-absorbing coating may be black, dark gray, dark color, dark gradient, dark pattern, or a combination thereof.

[0171] In some examples, the first path referred to in operations 1355 and 1360 refers to the path of the first light before the first light enters the first prism. Thus, the first path may be a path that has not yet been refracted by the first prism. For example, in Figure 9 the context of, the first path may refer to the path of the first light before z reaches the top side 922 of the first redirecting element 910. In Figure 11 the context of, the first path may refer to the path of the first light before reaching the corresponding top side (not labeled) of the first redirecting element 1110. In Figures 12A - 12C the context of, the first path may refer to the path of the first light before reaching the corresponding top side 1220 of the first prism 1212 of the redirecting element 1210. In some examples, the second path referred to in operations 1355 and 1360 refers to the path of the second light before the second light enters the second prism. Thus, the second path may be a path that has not yet been refracted by the second prism. For example, in Figure 9 the context of, the second path may refer to the path of the second light before reaching the top side 924 of the second redirecting element 912. In Figure 11 the context of, the second path may refer to the path of the second light before reaching the corresponding top side (not labeled) of the second redirecting element 1120. In Figures 12A - 12C the context of, the second path may refer to the path of the second light before reaching the corresponding top side 1220 of the second prism 1214 of the redirecting element 1210.

[0172] In some examples, the first prism includes a first reflective surface configured to reflect the first light. In some examples, the second prism includes a second reflective surface configured to reflect the second light. The first reflective surface may be or may include a mirror. The second reflective surface may be or may include a mirror. In some examples, the first path referred to in operations 1355 and 1360 refers to the path of the first light after the first light enters the first prism but before the first reflective surface reflects the first light. Thus, the first path may have been refracted by the first prism but not yet reflected by the first reflective surface. For example, in Figure 9In the context of, the first path can refer to the path of the first light that passes through the top side 922 of the first redirecting element 910 and enters the first redirecting element 910 but before reaching the reflective surface on the side 918 of the first redirecting element 910. In Figure 11 In the context of, the first path can refer to the path of the first light that enters the first redirecting element 1110 but before reaching the reflective surface on the side 1112 of the first redirecting element 1110. In Figures 12A - 12C In the context of, the first path can refer to the path of the first light that passes through the top side 1220 of the first prism 1212 of the redirecting element 1210 and enters the first prism 1212 of the redirecting element 1210 but before reaching the reflective surface on the side 1216 of the first prism 1212 of the redirecting element 1210. In some examples, the second path referred to in operations 1355 and 1360 refers to the path of the second light after the second light enters the second prism but before the second reflective surface reflects the second light. Thus, the second path may have been refracted by the second prism but not yet reflected by the second reflective surface. For example, in Figure 9 In the context of, the second path can refer to the path of the second light that passes through the top side 924 of the second redirecting element 912 and enters the second redirecting element 912 but before reaching the reflective surface on the side 920 of the second redirecting element 912. In Figure 11 In the context of, the second path can refer to the path of the second light after entering the second redirecting element 1120 but before reaching the reflective surface on the side of the second redirecting element 1120 that is closest to the side 1112 of the first redirecting element 1110. In Figures 12A - 12C In the context of, the second path can refer to the path of the second light that passes through the top side 1220 of the second prism 1214 of the redirecting element 1210 and enters the second prism 1214 of the redirecting element 1210 but before reaching the reflective surface on the side 1218 of the second prism 1214 of the redirecting element 1210.

[0173] In some examples, the first image and the second image are captured simultaneously, concurrently, in parallel, within a shared time window, within a threshold duration of each other, or in a combination thereof. The first light redirecting element can be fixed and / or stationary relative to the first image sensor. The second light redirecting element can be fixed and / or stationary relative to the second image sensor. The first light redirecting element can be fixed and / or stationary relative to the second light redirecting element. The first light redirecting element can be fixed and / or stationary relative to the housing of the imaging system. The second light redirecting element can be fixed and / or stationary relative to the housing of the imaging system. For example, the first image sensor, the first light redirecting element, the second image sensor, and the second light redirecting element are in Figure 8 、 Figure 9 、Figure 11 , Figures 12A - 12C The various image sensors and light redirecting elements shown, variants thereof as described herein, or combinations thereof can be arranged in a fixed and / or stationary arrangement. The first light redirecting element can be movable relative to the first image sensor and / or the second light redirecting element and / or the housing of the imaging system in some cases, for example, using a motor and / or an actuator. The second light redirecting element can be movable relative to the second image sensor and / or the first light redirecting element and / or the housing of the imaging system in some cases, for example, using a motor and / or an actuator.

[0174] The first flat surface of the first image sensor can face a first direction, while the second flat surface of the second image sensor can face a second direction. The first direction can be the optical axis of the first image sensor and / or the optical axis of the lens associated with the first image sensor and / or the optical axis of the camera associated with the first image sensor. The second direction can be the optical axis of the second image sensor and / or the optical axis of the lens associated with the second image sensor and / or the optical axis of the camera associated with the second image sensor. The first direction and the second direction can be parallel to each other. The first camera can also face the first direction. The second camera can also face the second direction. The first direction and the second direction can directly point at each other. In some examples, the first flat surface of the first image sensor can face the second flat surface of the second image sensor. In some examples, the first camera can face the second camera. For example, Figure 8 the first image sensor 802 and the second image sensor 804 face each other and are oriented in a direction parallel to each other's respective directions. Figure 9 the first image sensor 902 and the second image sensor 904 face each other and are oriented in a direction parallel to each other's respective directions. Figure 11 the first image sensor 1102 and the second image sensor 1104 face each other and are oriented in a direction parallel to each other's respective directions. Figures 12A - 12C the first image sensor 1202 and the second image sensor 1204 face each other and are oriented in a direction parallel to each other's respective directions.

[0175] At operation 1365, the imaging system uses perspective distortion correction to modify at least one of the first image and the second image. The perspective distortion correction of operation 1365 can be referred to as perspective distortion. Examples of the perspective distortion correction of operation 1365 include Figure 10A the perspective distortion correction 1022 of Figure 10B the perspective distortion correction 1022 of Figure 15 the flat perspective distortion correction 1515 of Figure 15 the curved perspective distortion correction 1525 of Figure 16 the flat perspective distortion correction 1620 ofFigure 16 Bending perspective distortion correction 1630, another type of perspective distortion correction described herein, another type of perspective distortion described herein, or a combination thereof.

[0176] In some examples, to perform the modification of operation 1365 on at least one of the first image and the second image, the imaging system uses perspective distortion correction to modify the first image from depicting a first perspective to depicting a common perspective. The imaging system uses perspective distortion correction to modify the second image from depicting a second perspective to depicting a common perspective. The common perspective can be between the first perspective and the second perspective. For example, in Figure 10B , the first image among the two images 1024 has a perspective that is tilted to the right, while the first image among the two images 1024 has a perspective that is tilted to the left. As visible in the first image portion of the combined image 1026 and the first image portion of the combined image 1026, the common perspective is straight ahead, between the left and right perspectives of the two images 1024. In Figure 16 , the perspective of the first original image plane 1614 is slightly tilted counterclockwise, while the perspective of the second original image plane 1616 is slightly tilted clockwise. The common perspective (as mapped using the flat projection transformation pixel mapping 1620) visible in the flat perspective correction image plane 1625 is completely horizontal, between the slightly counterclockwise and slightly clockwise tilted perspectives of the first original image plane 1614 and the second original image plane 1616.

[0177] In some examples, to perform the modification of operation 1365 on at least one of the first image and the second image, the imaging system identifies the depiction of one or more objects in the image data (of the first image and / or the second image). The imaging system modifies the image data by projecting the image data based on the depiction of one or more objects. In some examples, the imaging system can project the image data onto a flat perspective correction image plane (e.g., as part of the flat perspective distortion correction 1022 / 1520 / 1620 as shown in Figures 10A - 10B , Figure 15 and Figure 16 ). In some examples, the imaging system can project the image data onto a bending perspective correction image plane (e.g., as part of the bending perspective distortion correction 1530 / 1630 as shown in Figure 15 , Figure 16 , Figure 17 , Figure 18 and Figure 19 ). For example, referring to Figure 15, an imaging system (e.g., dual-camera device 1505) identifies depictions of a soda can in a first image and a second image. In the curved perspective distortion correction 1525, the imaging system (e.g., dual-camera device 1505) modifies the image data by projecting the image data based on the depiction of the soda can. Refer to Figure 16 , an imaging system (e.g., including lens 1660) identifies depictions of one or more objects along a curve in a scene 1655 in a first image and a second image. In the curved perspective distortion correction 1630, the imaging system (e.g., including lens 1660) modifies the image data by projecting the image data based on the depictions of one or more objects along the curve in the scene 1655. Refer to Figure 17 , an imaging system (not shown) identifies depictions of one or more objects (e.g., television 1740, couch 1750) in a scene 1655 in a first image and a second image. In different perspective distortion corrections of the three combined images 1710 - 1730, the imaging system can modify the image data by projecting the image data based on the depictions of one or more objects (e.g., television 1740, couch 1750).

[0178] In some examples, the imaging system uses luminance uniformity correction to modify at least one of the first image and the second image. For example, the imaging system can remove vignetting and / or other luminance non-uniformities from the first image, the second image, or both. Figure 10D The luminance uniformity correction 1062 is an example of a luminance uniformity correction that the imaging system can use to modify the luminance uniformity of the first image and / or the second image. The imaging system can also increase or decrease the overall luminance of the first image, the second image, or both, such that the overall luminance matches between the first image and the second image. The imaging system can also increase or decrease other image characteristics (e.g., contrast, color saturation, white balance, black balance, color levels, histogram, etc.) in the first image, the second image, or both, such that these image attributes match between the first image and the second image. Such adjustments of luminance and / or other image attributes can ensure that there are no visible seams in the combined image (e.g., between the portion of the combined image from the first image and the portion of the combined image from the second image). In some examples, the imaging system can perform the modifications related to luminance uniformity correction after the modifications related to perspective distortion correction in operation 1365. In some examples, the imaging system can perform the modifications related to luminance uniformity correction before the modifications related to perspective distortion correction in operation 1365. In some examples, the imaging system can perform the modifications related to luminance uniformity correction simultaneously with the modifications related to perspective distortion correction in operation 1365.

[0179] At operation 1370, the imaging system generates a combined image from the first image and the second image. The imaging system can generate the combined image from the first image and the second image in response to a modification of at least one of the first image and the second image using perspective distortion correction. The imaging system can generate the combined image from the first image and the second image in response to a modification of at least one of the first image and the second image using brightness uniformity correction. The combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image. For example, Figure 10B the combined image 1026 has a field of view that is larger and / or wider than the first field of view of the first image and the second field of view of the second image in the two images 1024. Similarly, Figure 10C the combined image has a field of view that is larger and / or wider than the first field of view of the first image captured by the first camera and the second field of view of the second image captured by the second camera.

[0180] Generating the combined image from the first image and the second image can include: aligning a first portion of the first image with a second portion of the second image. Generating the combined image from the first image and the second image can include: stitching the first image and the second image together based on the alignment of the first portion of the first image and the second portion of the second image. Figure 10C The digital alignment and stitching 1042 is an example of such alignment and stitching. The first portion of the first image and the second portion of the second image can at least partially match. For example, referring to Figure 10C , the first portion of the first image can be a portion of the first image captured by the first camera that includes Figure 10C in the middle of the scene (the letter "A" in a circle), while the second portion of the second image can be a portion of the second image captured by the second camera that includes Figure 10C in the middle of the scene (the letter "A" in a circle). The first portion of the first image and the second portion of the second image can match and can overlap for stitching. The combined image can include the first portion of the first image, the second portion of the second image, or a merged image portion that merges or combines the image data from the first portion of the first image and the image data from the second portion of the second image.

[0181] As described above, the imaging system can be the device 500. The device 500 can at least include a first camera 501 and a second camera 502 configured to capture image frames to generate a combined image. The device 500 can also include one or more redirecting elements 503.

[0182] Figure 14 is a flowchart showing an example process 1400 for capturing a plurality of image frames to be combined to generate a combined image frame.

[0183] Figure 14 The operations in can be performed by device 500 Figure 13A and / or Figure 13B Example implementations of the operations in. For example, device 500 can use Figure 8 , Figure 9 or Figures 12A - 12C the configuration of the cameras and redirecting elements (or other suitable redirecting elements) depicted in to virtually overlap the entrance pupil centers of the first camera 501 and the second camera 502 (as Figure 6 depicted in). The dashed box shows optional steps that can be performed.

[0184] At operation 1402, the first light redirecting element redirects the first light towards the first camera 501. For example, the first light redirecting element can redirect a portion of the light received through an opening in the device. In some embodiments, the first mirror of the first light redirecting element reflects the first light towards the first camera 501 (operation 1404). In Figure 8 the example of, the mirror of the first light redirecting element 810 can reflect light from the first portion of the scene to the first camera lens 806. In Figure 9 the example of, the mirror on the side 918 of the first prism 918 can reflect light from the first portion of the scene to the first camera lens 906. In Figure 12A the example of, the mirror on the side 1216 of the first prism 1212 of the redirecting element 1210 can reflect light from the first portion of the scene to the first camera lens 1206.

[0185] In some embodiments, the first prism of the first light redirecting element can also refract the first light (operation 1406). Returning to Figure 9 the example of, the redirecting element can include both a mirror and a prism. For example, one side of the prism can include a reflective coating that is used to reflect light passing through the prism. Referring back to Figure 12A the example of, the redirecting element can include multiple prisms, where one prism is used to refract the first light towards the first camera 501.

[0186] In some embodiments, the first lens guides the first light from the first light redirecting element towards the first camera 501 (operation 1408). At operation 1410, the first camera 501 captures a first image frame based on the first light. At operation 1412, the second light redirecting element redirects the second light towards the second camera 502. For example, the second light redirecting element can redirect a portion of the light received through an opening in the device. In some embodiments, the second mirror of the second light redirecting element reflects the second light towards the second camera 502 (operation 1414). In Figure 8In the example, the mirror of the second redirecting element 812 can reflect light from the second part of the scene towards the second camera lens 808. In Figure 9 In the example, the second mirror on the side 920 of the second prism of the second redirecting element 912 can reflect light from the second part of the scene to the second camera lens 908. In Figure 12A In the example, the second mirror on the side 1218 of the second prism of the redirecting element 1210 can reflect light from the second part of the scene to the second lens 1208. In some embodiments, the second prism of the second light redirecting element can also refract the second light (operation 1416). Returning to Figure 9 In the example, the second redirecting element 912 can include both a mirror and a prism. For example, one side of a triangular prism can include a reflective coating for reflecting light passing through the prism. Returning to Figure 12A In the example, the redirecting element 1210 can include a second prism and a second mirror for reflecting and refracting light towards the second camera lens 1208. Referring back to Figure 14 , in some embodiments, the first redirecting element and the second redirecting element are the same redirecting element. In certain implementations, the redirecting element includes multiple prisms and mirrors for redirecting the first light and redirecting the second light. For example, Figure 12A the redirecting element 1210 in includes two triangular prisms 1212 and 1214 (e.g., equilateral triangular prisms) with mirrors on their sides 1216 and 1218.

[0187] In some embodiments, the second lens can direct the second light from the second light redirecting element towards the image sensor of the second camera 502 (operation 1418). At operation 1420, the second camera 502 captures a second image frame based on the second light. As described above, the first light redirecting element and the second light redirecting element (which can be separate or a single redirecting element) can be placed to allow the central virtual overlap of the entrance pupils of the first camera 501 and the second camera 502. In this way, the parallax effect in the combined image can be reduced or eliminated. In certain embodiments, the second image frame is captured concurrently with and / or simultaneously with the first image frame. In this way, multiple image frames can be captured concurrently with and / or simultaneously by the first camera 501 and the second camera 502 of the device 500 to reduce the distortion in the combined image caused by global motion or local motion. The captured image frames can be provided to other components of the device 500 (such as the image processor 512) to process the image frames, including combining the image frames in operation 1422 to generate a combined (wide-angle) image, as described above.

[0188] The image frames discussed in this document may be referred to as images, image frames, video frames, or frames. The images discussed in this document may be referred to as images, image frames, video frames, or frames. The video frames discussed in this document may be referred to as images, image frames, video frames, or frames. The frames discussed in this document may be referred to as images, image frames, video frames, or frames.

[0189] Unless specifically described as implemented in a particular manner, the techniques described herein can be implemented in hardware, software, firmware, or any combination thereof. Any features described as modules or components can also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, these techniques can be at least partially implemented via a non-transitory processor-readable storage medium (e.g., Figure 5 the memory 506 in the example device 500), the non-transitory processor-readable storage medium including instructions 508 that, when executed by a processor 504 (or camera controller 510 or image signal processor 512 or other suitable component), cause the device 500 to perform one or more of the methods described above. The non-transitory processor-readable data storage medium can form part of a computer program product, which can include packaging material.

[0190] The non-transitory processor-readable storage medium can include random access memory (RAM) (such as synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, other known storage media, etc. Additionally or alternatively, these techniques can be at least partially implemented by a processor-readable communication medium that carries or transports code in the form of instructions or data structures and can be accessed, read, and / or executed by a computer or other processor.

[0191] The various illustrative logic blocks, modules, circuits, and instructions described in connection with the embodiments disclosed herein can be executed by one or more processors, such as Figure 5The processor 504 or the image signal processor 512 in the example device 500. Such processors can include, but are not limited to, one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), application-specific instruction set processors (ASIPs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. As used herein, the term "processor" can refer to any one of the foregoing structures or any other structure suitable for implementation of the techniques described herein. Additionally, in some aspects, the functions described herein can be provided within a special-purpose software module or a hardware module configured as described herein. Further, these techniques can be fully implemented in one or more circuits or logic units. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0192] Figure 15 is a conceptual diagram 1500 showing examples of planar perspective distortion correction 1515 and curved perspective distortion correction 1525. As previously mentioned, perspective distortion correction can be used to seemingly change the perspective or viewing angle of a captured scene. In Figure 10B the case of perspective distortion correction 1022, perspective distortion correction 1022 is used such that the first image and the second image seemingly share a common perspective or common viewing angle of the captured scene.

[0193] Figure 10B The perspective distortion correction 1022 shown in the conceptual diagram 1020 is an example of trapezoidal perspective distortion correction, which is an example of planar perspective distortion correction 1515. Trapezoidal perspective distortion correction maps a trapezoidal region to a rectangular region and vice versa. Planar perspective distortion correction maps a first flat (e.g., non-curved) two-dimensional region to a second flat (e.g., non-curved) two-dimensional region. The first flat (e.g., non-curved) two-dimensional region and the second flat (e.g., non-curved) two-dimensional region can have different rotational orientations (e.g., pitch, yaw, and / or roll) relative to each other. In some examples, matrix multiplication can be used to perform planar perspective distortion correction.

[0194] A device 500 having one of the dual-camera architectures discussed herein (e.g., as shown in FIGS. 900, 1100, 1200, 1240, and / or 1260) can use planar perspective distortion correction 1515 to produce high-quality combined images of many types of scenes. However, the device 500 can produce combined images of certain types of scenes when using planar perspective distortion correction 1515 that visually appear distorted and / or have visual aberrations. For such scenes, using curved perspective distortion correction 1525 can produce combined images with reduced or eliminated visual distortion compared to using planar perspective distortion correction 1515.

[0195] For example, conceptual diagram 1500 shows a scene 1510 in which five soda cans are arranged in an arc partially surrounding a dual-camera device 1505, where each of the five soda cans is approximately equidistant from the dual-camera device 1505. The dual-camera device 1505 is a device 500 having one of the dual-camera architectures discussed herein (e.g., as shown in FIG. 900, FIG. 1100, FIG. 1200, FIG. 1240, and / or FIG. 1260), and the device 500 generates a combined image of the scene 1510 from two images of the scene 1510 captured by two cameras of the dual-camera device 1505, as described herein (e.g., as shown in flowcharts 1300, 1350, or 1400).

[0196] The dual-camera device 1505 uses planar perspective distortion correction 1515 to perform perspective correction while generating a first combined image 1520. The first combined image 1520 visually appears distorted. For example, although the five soda cans in the scene 1510 are approximately equidistant from the dual-camera device 1505, the leftmost and rightmost soda cans in the first combined image 1520 appear larger than the three central soda cans in the first combined image 1520. The leftmost and rightmost soda cans themselves in the first combined image 1520 also appear distorted, where the leftmost and rightmost appear to have different heights. The leftmost and rightmost soda cans in the first combined image 1520 also appear to be farther apart from the three central soda cans in the first combined image 1520 than the three central soda cans are from each other.

[0197] Dual-camera device 1505 performs perspective correction using curvilinear transformation perspective distortion correction 1525 while generating a second combined image 1530. The second combined image 1530 reduces or removes all or most of the apparent visual distortions in the first combined image 1520. For example, the five soda cans in scene 1510 appear more similar in size to each other in the second combined image 1530 than in the first combined image 1520. The leftmost and rightmost soda cans also appear less distorted in the second combined image 1530 than in the first combined image 1520. The spacing between all five soda cans in scene 1510 appears more consistent in the second combined image 1530 than in the first combined image 1520.

[0198] In various types of scenes, curvilinear perspective distortion correction 1525 can be used more optimally than planar perspective distortion correction 1515. For example, in a panoramic scene of a distant horizon captured from a high altitude (e.g., a tall building or a mountain), using curvilinear perspective distortion correction 1525 can be superior to using planar perspective distortion correction 1515.

[0199] Figure 16 is a conceptual diagram showing the pixel mapping from the image sensor image plane to the perspective-corrected image plane in planar perspective distortion correction 1515 and curvilinear perspective distortion correction 1525. Specifically, Figure 16 includes a first diagram 1600 based on a dual-camera architecture such as shown in conceptual diagrams 900, 1100, 1200, 1240, and / or 1260. The first diagram 1600 shows virtual light beams passing through a first virtual lens 926 and reaching a first virtual image sensor 914. The first virtual image sensor 914 is also labeled as the first raw image plane 1614 because the first raw image plane 1614 represents a first image (not shown) captured by the first image sensor 902 / 1102 / 1202. The first diagram 1600 also shows virtual light beams passing through a second virtual lens 928 and reaching a second virtual image sensor 916. The second virtual image sensor 916 is also labeled as the second raw image plane 1616 because the second raw image plane 1616 represents a second image (not shown) captured by the second image sensor 904 / 1104 / 1204.

[0200] The first figure 1600 shows the dashed arrow of the projective transformation pixel mapping 1620, which performs the flat perspective distortion correction 1515. The dashed arrow of the projective transformation pixel mapping 1620 projects through each pixel of the first original image plane 1614 onto the corresponding pixel of the perspective-corrected image plane 1625, and projects through each pixel of the second original image plane 1616 onto the corresponding pixel of the perspective-corrected image plane 1625. The perspective-corrected image plane 1625 represents the combined image generated by merging the first image and the second image after performing the flat perspective distortion correction 1515.

[0201] Figure 16 The second figure 1650 in shows an example of the curved perspective distortion correction 1525. The scene 1655, which can include a flat part and a curved part, is captured using a camera with a lens 1660. The lens 1660 can be a physical lens (such as lenses 704, 806, 808, 906, 908, 1106, 1108, 1206, and / or 1208), or it can also be a virtual lens (such as virtual lenses 710, 926, and / or 928). The camera captures an image of the scene 1655, which is captured on the flat image plane 1665. In some examples, the flat image plane 1665 is the original image plane (such as in the first original image plane 1614 and / or the second original image plane 1616), which represents the image capture at a physical image sensor (such as image sensors 702, 802, 804, 902, 904, 1004, 1102, 1104, 1202, and / or 1204) and / or a virtual image sensor (such as virtual image sensors 708, 914, and / or 916). In some examples, the flat image plane 1665 is the perspective-corrected image plane 1625, as shown in the first figure 1600. The points along the flat image plane 1665 are represented by the flat x-axis. The points along the flat x-axis can be found using the formula x = f·tan(α) for a given angle α. In the second figure 1650, f is the focal length of the camera. In the second figure 1650, α is the camera's viewing angle, or an angle within the camera's viewing angle. For example, the camera's viewing angle can be 60 degrees. To perform the curved perspective distortion correction 1525, pixels are projected from the flat image plane 1665 onto the curved perspective-corrected image plane 1630. The points along the curved perspective-corrected image plane 1630 are represented by the curved x'-axis. The points along the curved x'-axis can be found using the formula x′ = f·α. Thus, regardless of the angle α, any point along the curved x'-axis is at the same distance f from the lens 1660.

[0202] When performing perspective correction on certain images, it may be useful to have more fine-grained control over the curvature of the curved perspective-corrected image plane 1630. The formula to perform a more subtle curved perspective distortion correction 1525. Here, x" represents the image plane with variable-curvature perspective correction depending on the variable P. In this formula, P is a variable that can be adjusted to adjust the curvature intensity of the image plane with variable-curvature perspective correction. For example, when P = 1, then x" = f·tan(α), making the curved perspective-corrected image plane 1630 flat and equivalent to the flat image plane 1665 (and equivalent to the flat x-axis). When P = 0, then x" is undefined, but when P approaches 0, the limit of x" is f·α. Thus, for the purpose of curved perspective distortion correction 1525, when P = 0, x" = f·α, making the image plane with variable-curvature perspective correction strongly curved and equivalent to the curved perspective-corrected image plane 1630 (and equivalent to the curved x'-axis). If P is between 0 and 1, the curvature of the image plane with variable-curvature perspective correction is less than that of the curved perspective-corrected image plane 1630 but more curved than the flat image plane 1665. Figure 17 Examples of composite images generated using the curved perspective distortion correction 1525, the image plane with variable-curvature perspective correction, and P set to different values are provided.

[0203] Figure 17 is a conceptual diagram 1700 showing three example composite images (1710, 1720, and 1730) of a scene, each composite image having the curved perspective distortion correction 1525 with a different degree of curvature applied. By using the above formula different degrees of curvature of the curved perspective distortion correction 1525 are applied by mapping to the image plane with variable-curvature perspective correction.

[0204] Specifically, the first composite image 1710 is generated by applying the curved perspective distortion correction 1525 to map image pixels onto the strongly curved perspective-corrected image plane because P = 0. The second composite image 1720 is generated by applying the curved perspective distortion correction 1525 to map image pixels onto the moderately curved perspective-corrected image plane because P = 0.8. The third composite image 1730 is generated by applying the perspective distortion correction 1515 to map image pixels onto the flat perspective-corrected image plane because P = 1.

[0205] All three composite images (1710, 1720, and 1730) depict the same scene, which shows a person sitting on a chair facing a television 1740, the chair being adjacent to a sofa 1750, and other matters. The person sitting on the chair is near the center of the captured scene, while the television 1740 is on the left side of the captured scene, and the sofa 1750 is on the right side of the captured scene. In the first composite image 1710 (where P = 0), the television 1740 and the sofa 1750 appear to be squeezed together horizontally, bent and / or tilted too much towards the camera, and thus appear unnatural. In the third composite image 1730 (where P = 1), the television 1740 and the sofa 1750 appear to be stretched out to the sides away from the sitting person, and appear unnaturally long and stretched horizontally relative to other objects in the scene. In the second composite image 1720 (where P = 0.8), the television 1740 and the sofa 1750 appear to naturally reflect the captured scene.

[0206] Figure 18 is a conceptual diagram of FIG. 1800 showing a comparison of different curvature degrees for curvilinear perspective distortion correction with respect to flat perspective distortion. By using the above formula different curvature degrees for applying curvilinear perspective distortion correction 1525 are mapped to an image plane corrected with variable curvature perspective. FIG. 1800 is based on the formula The horizontal axis of FIG. 1800 represents the normalized x with P = 1, or the mapped output of flat perspective correction using the angular range 0 <= α <= 65 degrees. The vertical axis represents x", or the mapped output of variable curvature perspective correction, which has different curvature degrees with the same scale as the horizontal axis.

[0207] FIG. 1800 shows five lines 1805, 1810, 1815, 1820, and 1825. The first line 1805 corresponds to P = 0. The second line 1810 corresponds to P = 0.4. The third line 1815 corresponds to P = 0.6. The fourth line 1820 corresponds to P = 0.8. The fifth line 1825 corresponds to P = 1.0.

[0208] Figure 19is a flowchart showing an example process for performing curved perspective distortion correction. In some examples, the operations in process 1300 can be performed by an imaging system. In some examples, the imaging system is device 500. In some examples, the imaging system includes one or more of the following: camera 112, camera 206, device 500, the imaging architecture shown in conceptual diagram 600, the imaging architecture shown in conceptual diagram 700, the imaging architecture shown in conceptual diagram 800, the imaging architecture shown in conceptual diagram 900, the imaging architecture shown in conceptual diagram 1100, the imaging architecture shown in conceptual diagram 1200, the imaging architecture shown in conceptual diagram 1240, the imaging architecture shown in conceptual diagram 1260, the imaging architecture shown in conceptual diagram 1600, at least one of the following: image capture and processing system 2000, image capture device 2005A, image processing device 2005B, image processor 2050, host processor 2052, ISP 2054, computing system 2100, one or more web servers of a cloud service, or a combination thereof.

[0209] At operation 1905, the imaging system receives a first image of a scene captured by a first image sensor of a first camera. The first image corresponds to a flat planar image plane. In some examples, the first image corresponds to a flat planar image plane because the first image sensor corresponds in shape and / or relative size to the flat planar image plane. In some examples, the first image corresponds to a flat planar image plane because the first image is projected onto the flat planar image plane using flat perspective distortion correction 1515.

[0210] At operation 1910, the imaging system identifies the curved perspective corrected image plane. In some examples, the imaging system identifies the curved perspective corrected image plane as the curved perspective corrected image plane 1630 of FIG. 1650 using the formula x′ = f·α. In some examples, the imaging system uses the formula to identify the curved perspective corrected image plane as a variably curved perspective corrected image plane.

[0211] At operation 1915, the imaging system generates a first perspective corrected image by projecting at least the image data of the first image from the flat planar image plane corresponding to the first image sensor onto the curved perspective corrected image plane.

[0212] Process 1900 can be an example of modifying the first image and / or the second image using the perspective distortion of operation 1365. In some examples, the first image received in operation 1905 can be an example of the first image received in operation 1355, and the perspective-corrected first image of operation 1915 can be an example of the first image modified using the perspective distortion of operation 1365. In some examples, the first image received in operation 1905 can be an example of the second image received in operation 1360, and the perspective-corrected first image of operation 1915 can be an example of the second image modified using the perspective distortion of operation 1365.

[0213] In some examples, P can be pre-determined. In an imaging system, user input can be received from a user via a user interface of the imaging system, and the imaging system can determine P based on the user input. In some examples, the imaging system can automatically determine P by detecting that the scene appears distorted in the first image or would likely appear distorted if the flat perspective distortion correction 1515 were applied to the first image alone. In some examples, when the imaging system determines that the scene appears distorted in the first image or would likely appear distorted if the flat perspective distortion correction 1515 were applied to the first image alone, the imaging system can automatically determine P to repair or optimize the appearance of the scene in the first image. In some examples, the imaging system can automatically determine P based on the object distance, distribution, and surface orientation of objects and / or surfaces in the scene captured in the first image. The imaging system can use the first image and / or one or more other images captured by one or more cameras of the imaging system to determine the object distance, distribution, and / or surface orientation of objects and / or surfaces in the scene based on object detection and / or recognition. For example, the imaging system can use face detection and / or face recognition to identify people in the scene, how close those people are to the camera (e.g., based on the face size determined via the distance between eyes or other measurements between facial features), the direction the people are facing, and so on. The imaging system can determine the object distance, distribution, and / or surface orientation of objects and / or surfaces in the scene based on one or more point clouds of the scene generated using one or more distance sensors of the imaging system, such as one or more light detection and ranging (LIDAR) sensors, one or more radio detection and ranging (RADAR) sensors, one or more sound navigation and ranging (SONAR) sensors, one or more sound detection and ranging (SODAR) sensors, one or more time-of-flight (TOF) sensors, one or more structured light (SL) sensors, or a combination thereof.

[0214] In some examples, the imaging system can use object detection, object recognition, face detection, or face recognition to automatically determine P to fix or optimize the appearance, face, or another specific type of object detected in the first image. For example, the imaging system can determine that the first image includes a depiction of an office building. The imaging system may expect the office building to have a rectangular prism shape (e.g., a box). The imaging system can automatically determine P to make the office building as close as possible to a rectangular prism shape in the perspective-corrected first image, and for example, to make the perspective-corrected first image remove or reduce any curvature of the edges of the office building that appear in the first image. The imaging system can determine that the first image includes a depiction of a person's face. The imaging system can recognize the person's face based on a comparison with other pre-stored images of the person's face, and can automatically determine P to make the face of the person depicted in the perspective-corrected first image look as close as possible to the pre-stored image of the person's face.

[0215] In some examples, the curved perspective distortion correction can be applied only to a portion of the first image, rather than the entire first image. For example, in the composite image 1520 depicting five soda cans, the leftmost and rightmost soda cans in the composite image 1520 appear the most distorted. In certain examples, the curved perspective distortion correction can be applied only to the region of the composite image 1520 that includes the depictions of the leftmost and rightmost soda cans.

[0216] In some examples, the curved perspective distortion correction can be applied to reduce various types of distortion, including that caused by wide-angle lenses and / or fisheye lenses.

[0217] Figure 20 is a block diagram showing the architecture of the image capture and processing system 2000. Each of the cameras, lenses, and / or image sensors discussed with respect to the previous figures can be included in the image capture and processing system 2000. For example, Figure 1 the lens 104 and the image sensor 106 of Figure 2 the camera 206 of Figure 5 the first camera 501 and the second camera 502 of Figure 6 the first camera lens 606 and the first image sensor 602 of Figure 6 can be included in one image capture and processing system 2000, while Figure 7 the second camera lens 608 and the second image sensor 604 of Figure 8The first camera lens 806 and the first image sensor 802 of Figure 8 The second camera lens 808 and the second image sensor 804 of Figure 9 The first camera lens 906 and the first image sensor 902 of Figure 9 The second camera lens 908 and the second image sensor 904 of Figure 10A The image sensor 1004 of Figure 10C The first camera and the second camera of Figure 11 The first camera lens 1106 and the first image sensor 1102 of Figure 11 The second camera lens 1108 and the second image sensor 1104 of Figures 12A - 12C The first camera lens 1206 and the first image sensor 1202 of Figures 12A - 12B The second camera lens 1208 and the second image sensor 1204 of Figure 13A The first lens and the first image sensor mentioned in the flowchart of the exemplary operation 1300 of Figure 13A The second lens and the second image sensor mentioned in the flowchart of the exemplary operation 1300 of Figure 13B The first lens and the first image sensor mentioned in the flowchart of the exemplary operation 1300 of Figure 13B The second lens and the second image sensor mentioned in the flowchart of the exemplary operation 1300 of Figure 14 The first camera mentioned in the flowchart of the exemplary operation 1400 of Figure 14 The second camera mentioned in the flowchart of the exemplary operation 1400 of

[0218] The image capture and processing system 2000 includes various components for capturing and processing images of a scene (e.g., an image of scene 2010). The image capture and processing system 2000 can capture individual images (or photos) and / or can capture video including multiple images (or video frames) in a particular sequence. The lens 2015 of the system 2000 faces the scene 2010 and receives light from the scene 2010. The lens 2015 deflects the light towards the image sensor 2030. The light received by the lens 2015 passes through an aperture controlled by one or more control mechanisms 2020 and is received by the image sensor 2030.

[0219] One or more control mechanisms 2020 can control exposure, focus, and / or zoom based on information from the image sensor 2030 and / or based on information from the image processor 2050. One or more control mechanisms 2020 can include multiple mechanisms and components; for example, the control mechanism 2020 can include one or more exposure control mechanisms 2025A, one or more focus control mechanisms 2025B, and / or one or more zoom control mechanisms 2025C. One or more control mechanisms 2020 can also include additional control mechanisms beyond the control mechanisms shown, such as control mechanisms that control analog gain, flash, HDR, depth of field, and / or other image capture attributes.

[0220] The focus control mechanism 2025B in the control mechanism 2020 can obtain a focus setting. In some examples, the focus control mechanism 2025B stores the focus setting in a memory register. Based on the focus setting, the focus control mechanism 2025B can adjust the position of the lens 2015 relative to the image sensor 2030. For example, based on the focus setting, the zoom control mechanism 2025B can move the lens 2015 closer to or farther from the image sensor 2030 by actuating a motor or servo system (or other lens mechanism) to adjust the focus. In some cases, additional lenses can be included in the system 2000, such as one or more microlenses on each photodiode of the image sensor 2030, each microlens deflecting the light received from the lens 2015 towards the corresponding photodiode before the light reaches the photodiode. The focus setting can be determined via contrast detection autofocus (CDAF), phase detection autofocus (PDAF), hybrid autofocus (HAF), or some combination thereof. The focus setting can be determined using the control mechanism 2020, the image sensor 2030, and / or the image processor 2050. The focus setting can be referred to as an image capture setting and / or an image processing setting.

[0221] The exposure control mechanism 2025A of the control mechanism 2020 can obtain an exposure setting. In some cases, the exposure control mechanism 2025A stores the exposure setting in a memory register. Based on the exposure setting, the exposure control mechanism 2025A can control the size of the aperture (e.g., aperture size or f / stop), the duration for which the aperture is open (e.g., exposure time or shutter speed), the sensitivity of the image sensor 2030 (e.g., ISO speed or film speed), the analog gain applied by the image sensor 2030, or any combination thereof. The exposure setting can be referred to as an image capture setting and / or an image processing setting.

[0222] The zoom control mechanism 2025C of the control mechanism 2020 can obtain a zoom setting. In some examples, the zoom control mechanism 2025C stores the zoom setting in a memory register. Based on the zoom setting, the zoom control mechanism 2025C can control the focal length of a lens element assembly (lens assembly) including the lens 2015 and one or more additional lenses. For example, the zoom control mechanism 2025C can control the focal length of the lens assembly by actuating one or more motors or servo systems (or other lens mechanisms) to move one or more lenses relative to each other. The zoom setting can be referred to as an image capture setting and / or an image processing setting. In some examples, the lens assembly can include a parfocal zoom lens or a varifocal zoom lens. In some examples, the lens assembly can include a focusing lens (which can be the lens 2015 in some cases) that first receives light from the scene 2010, and then the light passes through an afocal zoom system between the focusing lens (e.g., the lens 2015) and the image sensor 2030 before reaching the image sensor 2030. In some cases, the afocal zoom system can include two positive (e.g., converging, convex) lenses with equal or similar focal lengths (e.g., within a threshold difference of each other), and a negative (e.g., diverging, concave) lens between them. In some cases, the zoom control mechanism 2025C moves one or more lenses in the afocal zoom system, such as one or both of the negative lens and the positive lens.

[0223] The image sensor 2030 includes one or more arrays of photodiodes or other photosensitive elements. Each photodiode measures the amount of light that ultimately corresponds to a specific pixel in the image generated by the image sensor 2030. In some cases, different photodiodes may be covered by different color filters and can thus measure light that matches the color of the color filter covering the photodiode. For example, a Bayer filter includes a red filter, a blue filter, and a green filter, where each pixel of the image is generated based on red light data from at least one photodiode covered by a red filter, blue light data from at least one photodiode covered by a blue filter, and green light data from at least one photodiode covered by a green filter. In addition to or instead of red, blue, and / or green filters, other types of filters can use yellow, magenta, and / or cyan (also known as "emerald") filters. Some image sensors (e.g., the image sensor 2030) may have no color filter at all and may instead use different photodiodes throughout the pixel array (vertically stacked in some cases). Different photodiodes throughout the pixel array can have different spectral sensitivity curves and thus respond to different wavelengths of light. Monochromatic image sensors may also lack a color filter and thus lack color depth.

[0224] In some cases, the image sensor 2030 can alternatively or additionally include opaque and / or reflective masks that block light from reaching certain photodiodes or portions of certain photodiodes at certain times and / or from certain angles, which can be used for phase detection autofocus (PDAF). The image sensor 2030 can also include an analog gain amplifier for amplifying the analog signal output by the photodiodes and / or an analog-to-digital converter (ADC) for converting the analog signal output of the photodiodes (and / or amplified by the analog gain amplifier) to a digital signal. In some cases, certain components or functions discussed for one or more control mechanisms 2020 can alternatively or additionally be included in the image sensor 2030. The image sensor 2030 can be a charge-coupled device (CCD) sensor, an electron-multiplying CCD (EMCCD) sensor, an active pixel sensor (APS), complementary metal-oxide semiconductor (CMOS), N-type metal-oxide semiconductor (NMOS), a hybrid CCD / CMOS sensor (e.g., sCMOS), or some other combination thereof.

[0225] The image processor 2050 may include one or more processors, such as one or more image signal processors (ISPs) (including ISP 2054), one or more host processors (including host processor 2052), and / or one or more of any other type of processor 2110 discussed for the processing system 2100. The host processor 2052 may be a digital signal processor (DSP) and / or other type of processor. In some embodiments, the image processor 2050 is a single integrated circuit or chip (e.g., referred to as a system on a chip or SoC) that includes the host processor 2052 and the ISP 2054. In some cases, the chip may also include one or more input / output ports (e.g., input / output (I / O) port 2056), a central processing unit (CPU), a graphics processing unit (GPU), a broadband modem (e.g., 3G, 4G or LTE, 5G, etc.), memory, connection components (e.g., Bluetooth TM , Global Positioning System (GPS), etc.), any combination thereof, and / or other components. The I / O port 2056 may include any suitable input / output port or interface according to one or more protocols or specifications, such as an Inter-Integrated Circuit 2 (I2C) interface, an Inter-Integrated Circuit 3 (I3C) interface, a Serial Peripheral Interface (SPI) interface, a Serial General Purpose Input / Output (GPIO) interface, a Mobile Industry Processor Interface (MIPI) (e.g., an MIPI CSI-2 Physical (PHY) layer port or interface, an Advanced High-Performance Bus (AHB) bus, any combination thereof, and / or other input / output ports. In an illustrative example, the host processor 2052 may communicate with the image sensor 2030 using an I2C port, while the ISP 2054 may communicate with the image sensor 2030 using an MIPI port.

[0226] The image processor 2050 may perform many tasks, such as demosaicking, color space conversion, image frame downsampling, pixel interpolation, automatic exposure (AE) control, automatic gain control (AGC), CDAF, PDAF, PDAF, automatic white balance, merging image frames to form an HDR image, image recognition, object recognition, feature recognition, receiving inputs, managing outputs, managing memory, or some combination thereof. The image processor 2050 may store the image frames and / or the processed images in a random access memory (RAM) 2040 / 2020, a read-only memory (ROM) 2045 / 2025, a cache, a memory cell, another storage device, or some combination thereof.

[0227] Various input / output (I / O) devices 2060 may be connected to the image processor 2050. The I / O devices 2060 may include a display screen, a keyboard, buttons, a touch screen, a touchpad, a touch-sensitive surface, a printer, any other output device 2135, any other input device 2145, or some combination thereof. In some cases, caption text may be input into the image processing device 2005B via the physical keyboard or buttons of the I / O device 2060, or via the virtual keyboard or buttons of the touch screen of the I / O device 2060. The I / O 2060 may include one or more ports, jacks, or other connectors capable of establishing a wired connection between the system 2000 and one or more peripheral devices, and the system 2000 may receive data from and / or send data to one or more peripheral devices via one or more ports, jacks, or other connectors. The I / O 2060 may include one or more wireless transceivers capable of establishing a wireless connection between the system 2000 and one or more peripheral devices, and the system 2000 may receive data from and / or send data to one or more peripheral devices via these wireless transceivers. The peripheral devices may include any of the types of I / O devices 2060 discussed previously, and once they are coupled to a port, jack, wireless transceiver, or other wired and / or wireless connector, they may themselves be considered I / O devices 2060.

[0228] In some cases, the image capture and processing system 2000 may be a single device. In some cases, the image capture and processing system 2000 may be two or more separate devices, including an image capture device 2005A (e.g., a camera) and an image processing device 2005B (e.g., a computing device coupled to the camera). In some embodiments, the image capture device 2005A and the image processing device 2005B may be coupled together, for example, via one or more wires, cables, or other electrical connectors and / or wirelessly via one or more wireless transceivers. In some embodiments, the image capture device 2005A and the image processing device 2005B may be disconnected from each other.

[0229] As Figure 20 shown, the vertical dashed line will Figure 20The image capture and processing system 2000 is divided into two parts, which respectively represent the image capture device 2005A and the image processing device 2005B. The image capture device 2005A includes a lens 2015, a control mechanism 2020, and an image sensor 2030. The image processing device 2005B includes an image processor 2050 (including an ISP 2054 and a host processor 2052), a RAM 2040, a ROM 2045, and an I / O 2060. In some cases, some components shown in the image capture device 2005A, such as the ISP 2054 and / or the host processor 2052, may be included in the image capture device 2005A.

[0230] The image capture and processing system 2000 may include an electronic device, such as a mobile or fixed telephone handset (e.g., a smart phone, a cellular phone, etc.), a desktop computer, a laptop computer or a notebook computer, a tablet computer, a set-top box, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, an Internet Protocol (IP) camera, or any other suitable electronic device. In some examples, the image capture and processing system 2000 may include one or more wireless transceivers for wireless communication (e.g., cellular network communication, 802.11 wi-fi communication, wireless local area network (WLAN) communication, or some combination thereof). In some embodiments, the image capture device 2005A and the image processing device 2005B may be different devices. For example, the image capture device 2005A may include a camera device, and the image processing device 2005B may include a computing device, such as a mobile handset, a desktop computer, or other computing devices.

[0231] Although the image capture and processing system 2000 is shown as including certain components, those of ordinary skill in the art will understand that the image capture and processing system 2000 may include more components than Figure 20 shown therein. The components of the image capture and processing system 2000 may include software, hardware, or one or more combinations of software and hardware. For example, in some embodiments, the components of the image capture and processing system 2000 may include or may be implemented using an electronic circuit or other electronic hardware, which may include one or more programmable electronic circuits (e.g., a microprocessor, a GPU, a DSP, a CPU, and / or other suitable electronic circuits), and / or may include computer software, firmware, or any combination thereof and / or may be implemented using the same to perform various operations described herein. The software and / or firmware may include one or more instructions stored on a computer-readable storage medium and executable by one or more processors of the electronic device implementing the image capture and processing system 2000.

[0232] Figure 21 This is a diagram showing an example of a system for implementing certain aspects of the present technology. Specifically, Figure 21 an example of a computing system 2100 is shown, which can be any computing device, remote computing system, camera, or any component thereof that constitutes an internal computing system, where components of the system communicate with each other using connection 2105. Connection 2105 can be a physical connection using a bus, or a direct connection to the processor 2110, such as in a chipset architecture. Connection 2105 can also be a virtual connection, network connection, or logical connection.

[0233] In some embodiments, the computing system 2100 is a distributed system, where the functions described in the present disclosure can be distributed within a data center, multiple data centers, a peer-to-peer network, etc. In some embodiments, one or more of the described system components represent many such components, each component performing some or all of the functions described for the component it represents. In some embodiments, the components can be physical or virtual devices.

[0234] The example system 2100 includes at least one processing unit (CPU or processor) 2110 and a connection 2105 that couples various system components, including system memory 2115 (such as read-only memory (ROM) 2120 and random access memory (RAM) 2125), to the processor 2110. The computing system 2100 can include a cache 2112 of high-speed memory that is directly connected to, adjacent to, or integrated as part of the processor 2110.

[0235] The processor 2110 can include any general-purpose processor and hardware services or software services, such as services 2132, 2134, and 2136 stored in the storage device 2130, which are configured to control the processor 2110 and special-purpose processors in which software instructions are incorporated into the actual processor design. The processor 2110 can essentially be a completely independent computing system, containing multiple cores or processors, buses, memory controllers, caches, etc. The multi-core processor can be symmetric or asymmetric.

[0236] To enable user interaction, computing system 2100 includes an input device 2145, which can represent any number of input mechanisms, such as a microphone for voice, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, voice, and so on. Computing system 2100 may also include an output device 2135, which can be one or more of a plurality of output mechanisms. In some cases, a multimodal system can enable a user to provide multiple types of input / output to communicate with computing system 2100. Computing system 2100 may include a communication interface 2140, which can generally govern and manage user input and system output. The communication interface can use wired and / or wireless transceivers to perform or facilitate receiving and / or sending wired or wireless communications, including transceivers using the following: audio jack / plug, microphone jack / plug, universal serial bus (USB) port / plug, port / plug, Ethernet port / plug, fiber optic port / plug, proprietary wired port / plug, wireless signal transmission, low-power (BLE) wireless signal transmission, wireless signal transmission, radio frequency identification (RFID) wireless signal transmission, near field communication (NFC) wireless signal transmission, dedicated short range communication (DSRC) wireless signal transmission, 802.11 Wi-Fi wireless signal transmission, wireless local area network (WLAN) signal transmission, visible light communication (VLC), worldwide interoperability for microwave access (WiMAX), infrared (IR) communication wireless signal transmission, public switched telephone network (PSTN) signal transmission, integrated services digital network (ISDN) signal transmission, 3G / 4G / 5G / LTE cellular data network wireless signal transmission, ad-hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or some combination thereof. The communication interface 2140 may also include one or more global navigation satellite system (GNSS) receivers or transceivers for determining the location of computing system 2100 based on one or more signals received from one or more satellites associated with one or more GNSS systems. GNSS systems include, but are not limited to, the United States-based Global Positioning System (GPS), the Russian-based Global Navigation Satellite System (GLONASS), the Chinese-based BeiDou Navigation Satellite System (BDS), and the European-based Galileo GNSS. There is no limitation on the operation on any particular hardware arrangement, and thus the basic features here can be easily replaced with improved hardware or firmware arrangements (when they are in development).

[0237] The storage device 2130 can be a non-volatile and / or non-transitory and / or computer-readable memory device, and can be a hard disk or other types of computer-readable media that can store computer-accessible data, such as magnetic tape cartridges, flash memory cards, solid-state memory devices, digital versatile disks, cassette tapes, floppy disks, flexible disks, hard disks, magnetic tapes, magnetic strips / magnetic stripe tapes, any other magnetic storage media, flash memory, memristor memory, any other solid-state memory, compact disc read-only memory (CD-ROM) optical discs, rewritable compact discs (CD) optical discs, digital video discs (DVD) optical discs, Blu-ray discs (BDD) optical discs, holographic optical discs, another optical medium, secure digital (SD) cards, micro secure digital (microSD) cards, Memory cards, smart card chips, EMV chips, subscriber identity module (SIM) cards, tiny / micro / nano / pico SIM cards, another integrated circuit (IC) chip / card, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash EPROM (FLASHEPROM), caches (L1 / L2 / L3 / L4 / L5 / L#), resistive random access memory (RRAM / ReRAM), phase change memory (PCM), spin transfer torque RAM (STT-RAM), other memory chips or cartridge memories, and / or combinations thereof.

[0238] The storage device 2130 can include software services, servers, services, etc., which, when the processor 2110 executes the code defining such software, cause the system to perform a certain function. In some embodiments, the hardware service that performs a specific function can include software components stored in a computer-readable medium and connected to the necessary hardware components (such as the processor 2110, connection 2105, output device 2135, etc.) to perform that function.

[0239] As used herein, the term "computer-readable medium" includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. The computer-readable medium may include non-transitory media in which data can be stored, and excludes carrier waves and / or transient electronic signals propagated wirelessly or via a wired connection. Examples of non-transitory media may include, but are not limited to, magnetic disks or tapes, optical storage media such as compact discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or storage devices. Code and / or machine-executable instructions may be stored on the computer-readable medium, which may represent a process, function, subroutine, program, routine, module, software package, class, or any combination of instructions, data structures, or program statements. Code segments may be connected to another code segment or hardware circuit by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameter data, etc. may be passed, forwarded, or transmitted using any suitable means including memory sharing, message passing, token passing, network transmission, etc.

[0240] In some embodiments, computer-readable storage devices, media, and memories may include cables or wireless signals that contain bitstreams, etc. However, when mentioned, non-transitory computer-readable storage media explicitly exclude media such as energy, carrier signals, electromagnetic waves, and signals themselves.

[0241] Specific details are provided in the foregoing description to provide a thorough understanding of the embodiments and examples provided herein. However, those of ordinary skill in the art will appreciate that these embodiments may be practiced without these specific details. For the sake of clarity, in some instances, the present technology may be presented as including separate functional blocks, and the separate functional blocks include functional blocks that include devices, device components, steps or routines in a method embodied in software, or a combination of hardware and software. In addition to the components shown in the figures and / or described herein, other components may be used. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form to avoid obscuring the embodiments with unnecessary details. For example, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments.

[0242] Each embodiment may be described as a process or method, which is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed concurrently, concurrently, or simultaneously. In addition, the order of these operations may be rearranged. A process terminates after its operations are completed, but a process may have other steps not shown in the figure. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to the calling function or the main function.

[0243] The processes and methods according to the above examples can be implemented using computer-executable instructions stored in or obtainable from a computer-readable medium. For example, such instructions may include instructions and data that cause a general-purpose computer, a special-purpose computer, or a processing device, or otherwise configure a general-purpose computer, a special-purpose computer, or a processing device to perform a certain function or group of functions. Some parts of the computer resources used can be accessed through a network. The computer-executable instructions can be, for example, binary, intermediate format instructions such as assembly language, firmware, source code, etc. Examples of computer-readable media that can be used to store instructions, the information used, and / or the information created during the method according to the examples include magnetic disks or optical disks, flash memory, USB devices provided with non-volatile memory, network storage devices, and so on.

[0244] Devices that implement the processes and methods according to these disclosures may include hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof, and may adopt any of a variety of form factors. When implemented in software, firmware, middleware, or microcode, the program code or code segments (e.g., a computer program product) for performing the necessary tasks may be stored in a computer-readable or machine-readable medium. The processor may execute the necessary tasks. Typical examples of form factors include laptop computers, smart phones, mobile phones, tablet devices, or other small-form-factor personal computers, personal digital assistants, rack-mounted devices, stand-alone devices, and so on. The functions described herein may also be embodied in peripheral devices or add-on cards. As a further example, such functions may also be implemented on a circuit board between different chips or different processes executed in a single device.

[0245] Instructions, the medium for conveying these instructions, the computing resources for executing them, and other structures for supporting such computing resources are example units for providing the functions described in this disclosure.

[0246] In the foregoing description, various aspects of the present application have been described with reference to specific embodiments of the present application, but those skilled in the art will recognize that the present application is not limited thereto. Accordingly, although illustrative embodiments of the present application have been described in detail herein, it should be understood that these inventive concepts may be embodied and employed in other ways and that the appended claims are intended to be construed to include such variations, except as limited by the prior art. The various features and aspects of the above applications may be used alone or in combination. Additionally, embodiments may be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Accordingly, this specification and the drawings should be regarded as illustrative rather than restrictive. For purposes of illustration, methods have been described in a particular order. It should be recognized that in alternative embodiments, the methods may be performed in an order different from that described.

[0247] One of ordinary skill in the art will understand that the less than (“<”) and greater than (“>”) symbols or terms used herein may be replaced with less than or equal to (“≤”) and greater than or equal to (“≥”) symbols without departing from the scope of this specification.

[0248] In cases where a component is described as “configured to” perform certain operations, such configuration may be achieved, for example, by designing electronic circuitry or other hardware to perform the operation, by programming a programmable electronic circuit (such as a microprocessor or other suitable electronic circuit) to perform the operation, or any combination thereof.

[0249] The phrase “coupled to” refers to any component that is directly or indirectly physically connected to another component and / or any component that directly or indirectly communicates with another component (e.g., connected to another component via a wired or wireless connection and / or other suitable communication interface).

[0250] Claim language or other language that recites “at least one” of a set and / or “one or more” of a set means that one member of the set or multiple members of the set (in any combination) satisfy the claim. For example, claim language that recites “at least one of A and B” means A, B, or A and B. In another example, claim language that recites “at least one of A, B, and C” means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The language “at least one” of a set and / or “one or more” of a set does not limit the set to the items listed in the set. For example, claim language that recites “at least one of A and B” may mean A, B, or A and B, and may additionally include items not listed in the set of A and B.

[0251] Each of the illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or combinations thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

[0252] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as a general purpose computer, a wireless communication handheld device, or an integrated circuit device having multiple uses including applications included in wireless communication handheld devices and other devices. Any feature described as a module or component may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques may be implemented at least in part by a computer-readable data storage medium including program code, the program code including instructions that, when executed, perform one or more of the methods described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may include a memory or data storage medium, such as random access memory (RAM) (e.g., synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. Additionally or alternatively, the techniques may be implemented at least in part by a computer-readable communication medium that carries or transports program code in the form of instructions or data structures and that can be accessed, read, and / or executed by a computer, such as a propagated signal or waveform.

[0253] The program code can be executed by a processor, which may include one or more processors such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Such a processor can be configured to perform any of the techniques described in this disclosure. A general-purpose processor may be a microprocessor, or, alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any combination of the foregoing structures, or any other structure or device suitable for implementation of the techniques described herein. Additionally, in some aspects, the functions described herein may be provided within dedicated software and / or hardware modules configured for encoding and decoding, or incorporated into a combined video codec (CODEC).

[0254] As noted above, although the disclosure shows illustrative aspects, it should be noted that various changes and modifications may be made herein without departing from the scope of the appended claims. Additionally, unless expressly stated otherwise, the functions, steps, or acts of the method claims according to the aspects described herein need not be performed in any particular order. Further, while elements may be described or claimed in the singular, the plural is also contemplated unless expressly stated to be limited to the singular. Accordingly, the present disclosure is not limited to the illustrated examples, and any unit for performing the functions described herein is included in the aspects of the present disclosure.

[0255] Illustrative aspects of the present disclosure include:

[0256] Aspect 1: A device for digital imaging, comprising: a memory; and one or more processors configured to: receive a first image frame of a scene captured by a first camera having a first entrance pupil, wherein: a first light redirecting element redirects first light towards the first camera; and the first camera captures the first image frame based on the first light redirected towards the first camera by the first light redirecting element; receive a second image frame of the scene captured by a second camera having a second entrance pupil, wherein: a second light redirecting element redirects second light towards the second camera; the second camera captures the second image frame based on the second light redirected towards the second camera by the second light redirecting element; and a first center of the first camera lens entrance pupil and a second center of the second camera lens entrance pupil are virtually overlapped; and generate a combined image from the first image frame and the second image frame, wherein the combined image includes a first field of view wider than a field of view of the first image frame or a field of view of the second image frame.

[0257] Aspect 2: The device according to aspect 1, wherein the one or more processors are further configured to: adjust the first image frame from a first perspective to a common perspective; adjust the second image frame from a second perspective to the common perspective, wherein the common perspective is between the first perspective and the second perspective.

[0258] Aspect 3: The device according to any one of aspect 1 or aspect 2, wherein the one or more processors are configured to: align and stitch together the first image frame and the second image frame to generate a combined image, wherein aligning the image frames is based on scene matching in an overlapping portion of the scene in the first image frame and the second image frame.

[0259] Aspect 4: The device according to any one of aspects 1 to 3, further comprising: a first camera; a second camera; a first light redirecting element; and a second light redirecting element, wherein, based on positions of the first camera, the second camera, and the first light redirecting element and the second light redirecting element, a first center of the first camera lens entrance pupil and a second center of the second camera lens entrance pupil are virtually overlapped at a first position while the first camera lens and the second camera lens do not physically overlap at the first position.

[0260] Aspect 5: The device according to aspect 4, wherein: the first light redirecting element includes a first mirror configured to reflect the first light towards the first camera; and the second light redirecting element includes a second mirror configured to reflect the second light towards the second camera.

[0261] Aspect 6: The apparatus according to aspect 5, wherein: the first light redirecting element further comprises a first prism configured to refract the first light; and the second light redirecting element further comprises a second prism configured to refract the second light.

[0262] Aspect 7: The apparatus according to aspect 6, wherein: the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; one or more corners of the first prism are prevented from reflecting light from a first camera lens surface of the first camera back towards the first camera lens; and one or more corners of the second prism are prevented from reflecting light from a second camera lens surface of the second camera back towards the second camera lens.

[0263] Aspect 8: The apparatus according to aspect 7, wherein: the one or more corners of the first prism include chamfered edges, wherein a light absorbing coating is applied to the chamfered edges of the first prism; and the one or more corners of the second prism include chamfered edges, wherein a light absorbing coating is applied to the chamfered edges of the second prism.

[0264] Aspect 9: The apparatus according to any one of aspects 1 to 8, wherein the first light redirecting element and the second light redirecting element are a single redirecting element; the single redirecting element comprises the first prism and the second prism; the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; and the first prism and the second prism overlap and are joined at the first side and the second side in the single redirecting element.

[0265] Aspect 10: The apparatus according to any one of aspects 1 to 8, wherein the first image frame and the second image frame are captured concurrently.

[0266] Aspect 11: A method for digital imaging, comprising: receiving a first image frame of a scene captured by a first camera having a first entrance pupil, wherein: a first light redirecting element redirects the first light towards the first camera; and the first camera captures the first image frame based on the first light redirected towards the first camera by the first light redirecting element; receiving a second image frame of the scene captured by a second camera having a second entrance pupil, wherein: a second light redirecting element redirects the second light towards the second camera; the second camera captures the second image frame based on the second light redirected towards the second camera by the second light redirecting element; and a first center of the first camera lens entrance pupil and a second center of the second camera lens entrance pupil are virtually overlapped; and generating a combined image from the first image frame and the second image frame, wherein the combined image includes a first field of view wider than a field of view of the first image frame or a field of view of the second image frame.

[0267] Aspect 12: The method according to aspect 11, further comprising: adjusting the first image frame from a first perspective to a common perspective; and adjusting the second image frame from a second perspective to the common perspective, wherein the common perspective is between the first perspective and the second perspective.

[0268] Aspect 13: The method according to any one of aspects 11 or 12, further comprising: aligning and stitching together the first image frame and the second image frame to generate a combined image, wherein aligning the image frames is based on scene matching in an overlapping portion of the scene in the first image frame and the second image frame.

[0269] Aspect 14: The method according to any one of aspects 11 to 13, further comprising: redirecting the first light by the first light redirecting element; redirecting the second light by the second light redirecting element; capturing the first image frame by the first camera; and capturing the second image frame by the second camera, wherein, based on positions of the first camera, the second camera, and the first redirecting element and the second light redirecting element, a center of the first camera lens entrance pupil and a center of the second camera lens entrance pupil are virtually overlapped at a first position while the first camera lens and the second camera lens do not physically overlap at the first position.

[0270] Aspect 15: The method according to aspect 14, wherein: redirecting the first light by the first light redirecting element includes: reflecting the first light towards the first camera by a first mirror; and redirecting the second light by the second light redirecting element includes: reflecting the second light towards the second camera by a second mirror.

[0271] Aspect 16: The method according to aspect 15, wherein: redirecting the first light by the first light redirecting element further comprises: refracting the first light by a first prism; and redirecting the second light by the second light redirecting element further comprises: refracting the second light by a second prism.

[0272] Aspect 17: The method according to aspect 16, wherein: the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; preventing one or more corners of the first prism from reflecting light from the first camera lens surface of the first camera back towards the first camera lens; and preventing one or more corners of the second prism from reflecting light from the second camera lens surface of the second camera back towards the second camera lens.

[0273] Aspect 18: The method according to aspect 17, wherein: the one or more corners of the first prism include beveled edges, and a light absorbing coating is applied to the beveled edges of the first prism; and the one or more corners of the second prism include beveled edges, and a light absorbing coating is applied to the beveled edges of the second prism.

[0274] Aspect 19: The method according to any one of aspects 11 to 18, wherein the first light redirecting element and the second light redirecting element are a single redirecting element; the single redirecting element includes the first prism and the second prism; the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; and the first prism and the second prism overlap and are connected at the first side and the second side in the single redirecting element.

[0275] Aspect 20: The method according to any one of aspects 11 to 19, wherein the first image frame and the second image frame are captured concurrently.

[0276] Aspect 21: A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a device for digital imaging, cause the device to: receive a first image frame of a scene captured by a first camera having a first entrance pupil, where: a first light redirecting element redirects first light towards the first camera; and the first camera captures the first image frame based on the first light redirected towards the first camera by the first light redirecting element; receive a second image frame of the scene captured by a second camera having a second entrance pupil, where: a second light redirecting element redirects second light towards the second camera; the second camera captures the second image frame based on the second light redirected towards the second camera by the second light redirecting element; and a first center of the first camera lens entrance pupil and a second center of the second camera lens entrance pupil are virtually overlapped; and generate a combined image from the first image frame and the second image frame, where the combined image includes a first field of view that is wider than the field of view of the first image frame or the field of view of the second image frame.

[0277] Aspect 22: The computer-readable medium according to aspect 21, wherein the execution of the instructions further causes the device to: adjust the first image frame from a first perspective to a common perspective; and adjust the second image frame from a second perspective to the common perspective, where the common perspective is between the first perspective and the second perspective.

[0278] Aspect 23: The computer-readable medium according to any one of aspects 21 or 22, wherein the execution of the instructions further causes the device to: align and stitch together the first image frame and the second image frame to generate a combined image, where aligning the image frames is based on scene matching in an overlapping portion of the scene in the first image frame and the second image frame.

[0279] Aspect 24: The computer-readable medium according to any one of aspects 21 to 23, wherein the execution of the instructions further causes the device to: redirect the first light through the first light redirecting element; redirect the second light through the second light redirecting element; capture the first image frame with the first camera; and capture the second image frame with the second camera, where based on the positions of the first camera, the second camera, and the first light redirecting element and the second light redirecting element, the first camera lens entrance pupil center and the second camera lens entrance pupil center are virtually overlapped at a first position while the first camera lens and the second camera lens do not physically overlap at the first position.

[0280] Aspect 25: The computer-readable medium according to aspect 24, wherein: redirecting the first light by the first light redirecting element includes: reflecting the first light towards the first camera by a first mirror; redirecting the second light by the second light redirecting element includes: reflecting the second light towards the second camera by a second mirror.

[0281] Aspect 26: The computer-readable medium according to aspect 25, wherein: redirecting the first light by the first light redirecting element further includes: refracting the first light by a first prism; and redirecting the second light by the second light redirecting element further includes: refracting the second light by a second prism.

[0282] Aspect 27: The computer-readable medium according to aspect 26, wherein: the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; preventing one or more corners of the first prism from reflecting light from the surface of the first camera lens of the first camera back towards the first camera lens; and preventing one or more corners of the second prism from reflecting light from the surface of the second camera lens of the second camera back towards the second camera lens.

[0283] Aspect 28: The computer-readable medium according to aspect 27, wherein: the one or more corners of the first prism include chamfered edges, and a light-absorbing coating is applied to the chamfered edges of the first prism; and the one or more corners of the second prism include chamfered edges, and a light-absorbing coating is applied to the chamfered edges of the second prism.

[0284] Aspect 29: The computer-readable medium according to any one of aspects 21 to 28, wherein the first light redirecting element and the second light redirecting element are a single redirecting element; the single redirecting element includes the first prism and the second prism; the first mirror is located on a first side of the first prism; the second mirror is located on a second side of the second prism; and the first prism and the second prism overlap and are connected at the first side and the second side in the single redirecting element.

[0285] Aspect 30: The computer-readable medium according to any one of aspects 21 to 28, wherein the first image frame and the second image frame are captured concurrently.

[0286] Aspect 31: An apparatus for digital imaging, the apparatus comprising: a memory; and one or more processors configured to: receive a first image of a scene captured by a first image sensor, wherein a first light redirecting element is configured to redirect first light from a first path to a redirected first path towards the first image sensor, wherein the first image sensor is configured to capture the first image based on receiving the first light at the first image sensor; receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element is configured to redirect second light from a second path to a redirected second path towards the second image sensor, wherein the second image sensor is configured to capture the second image based on receiving the second light at the second image sensor; modify at least one of the first image and the second image using perspective distortion correction; and generate a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0287] Aspect 32: The apparatus according to aspect 31, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element.

[0288] Aspect 33: The apparatus according to any one of aspects 31 or 32, wherein, in order to modify at least one of the first image and the second image using the perspective distortion correction, the one or more processors are configured to: modify the first image from depicting a first perspective to depicting a common perspective using the perspective distortion correction; and modify the second image from depicting a second perspective to depicting the common perspective using the perspective distortion correction, wherein the common perspective is between the first perspective and the second perspective.

[0289] Aspect 34: The apparatus according to any one of aspects 31 to 33, wherein, in order to modify at least one of the first image and the second image using the perspective distortion correction, the one or more processors are configured to: identify a depiction of one or more objects in the image data of at least one of the first image and the second image; and modify the image data at least in part by projecting the image data based on the depiction of the one or more objects.

[0290] Aspect 35: The apparatus according to any one of aspects 31 to 34, wherein, in order to generate the combined image from the first image and the second image, the one or more processors are configured to: align a first portion of the first image with a second portion of the second image; and stitch the first image and the second image together based on the alignment of the first portion of the first image with the second portion of the second image.

[0291] Aspect 36: The apparatus according to any one of aspects 31 to 35, further comprising: the first image sensor; the second image sensor; the first light redirecting element; and the second light redirecting element.

[0292] Aspect 37: The apparatus according to any one of aspects 31 to 36, wherein: the first light redirecting element includes a first reflective surface, wherein, in order to redirect the first light towards the first image sensor, the first light redirecting element uses the first reflective surface to reflect the first light towards the first image sensor; and the second light redirecting element includes a second reflective surface, wherein, in order to redirect the second light towards the second image sensor, the second light redirecting element uses the second reflective surface to reflect the second light towards the second image sensor.

[0293] Aspect 38: The apparatus according to any one of aspects 31 to 37, wherein: the first light redirecting element includes a first prism configured to refract the first light; and the second light redirecting element includes a second prism configured to refract the second light.

[0294] Aspect 39: The apparatus according to aspect 38, wherein the first prism and the second prism are adjacent.

[0295] Aspect 40: The apparatus according to any one of aspects 38 or 39, wherein a bridge connects a first edge of the first prism and a second edge of the second prism, wherein the bridge is configured to: prevent light from reflecting from at least one of the first edge of the first prism and the second edge of the second prism.

[0296] Aspect 41: The apparatus according to any one of aspects 38 to 40, wherein the first prism includes at least one chamfered edge, and wherein the second prism includes at least one chamfered edge.

[0297] Aspect 42: The apparatus according to any one of aspects 38 to 41, wherein the first prism includes at least one edge having a light absorbing coating, and wherein the second prism includes at least one edge having a light absorbing coating.

[0298] Aspect 43: The apparatus according to any one of aspects 38 to 42, wherein the first path is the path of the first light before the first light enters the first prism, and wherein the second path is the path of the second light before the second light enters the second prism.

[0299] Aspect 44: The apparatus according to any one of aspects 38 to 43, wherein the first prism includes a first reflective surface configured to reflect the first light, and wherein the second prism includes a second reflective surface configured to reflect the second light.

[0300] Aspect 45: The apparatus according to any one of aspects 38 to 44, wherein the first path is the path of the first light after the first light enters the first prism but before the first reflective surface reflects the first light, and wherein the second path is the path of the second light after the second light enters the second prism but before the second reflective surface reflects the second light.

[0301] Aspect 46: The apparatus according to any one of aspects 31 to 45, wherein the first image and the second image are captured simultaneously.

[0302] Aspect 47: The apparatus according to any one of aspects 31 to 46, wherein the first light redirecting element is fixed relative to the first image sensor, and wherein the second light redirecting element is fixed relative to the second image sensor.

[0303] Aspect 48: The apparatus according to any one of aspects 31 to 47, wherein a first flat surface of the first image sensor faces a first direction, and wherein a second flat surface of the second image sensor faces a second direction parallel to the first direction.

[0304] Aspect 49: The apparatus according to any one of aspects 31 to 48, wherein the one or more processors are configured to: modify at least one of the first image and the second image using luminance uniformity correction.

[0305] Aspect 50: A method for digital imaging, the method comprising: receiving a first image of a scene captured by a first image sensor, wherein a first light redirecting element redirects first light from a first path to a redirected first path towards the first image sensor, wherein the first image sensor captures the first image based on the first light received at the first image sensor; receiving a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path towards the second image sensor, wherein the second image sensor captures the second image based on the second light received at the second image sensor; using perspective distortion correction to modify at least one of the first image and the second image; and generating a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0306] Aspect 51: The method according to aspect 50, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element.

[0307] Aspect 52: The method according to any one of aspects 50 or 51, wherein using the perspective distortion correction to modify at least one of the first image and the second image includes: using the perspective distortion correction to modify the first image from depicting a first perspective to depicting a common perspective; and using the perspective distortion correction to modify the second image from depicting a second perspective to depicting the common perspective, wherein the common perspective is between the first perspective and the second perspective.

[0308] Aspect 53: The method according to any one of aspects 50 to 52, wherein using the perspective distortion correction to modify at least one of the first image and the second image includes: identifying a depiction of one or more objects in the image data of at least one of the first image and the second image; and modifying the image data by projecting the image data based on the depiction of the one or more objects.

[0309] Aspect 54: The method according to any one of aspects 50 to 53, wherein: the first light redirecting element includes a first reflective surface, and wherein, in order to redirect the first light towards the first image sensor, the first light redirecting element uses the first reflective surface to reflect the first light towards the first image sensor; and the second light redirecting element includes a second reflective surface, and wherein, in order to redirect the second light towards the second image sensor, the second light redirecting element uses the second reflective surface to reflect the second light towards the second image sensor.

[0310] Aspect 55: The method according to any one of aspects 50 to 54, wherein: the first light redirecting element includes a first prism configured to refract the first light; and the second light redirecting element includes a second prism configured to refract the second light.

[0311] Aspect 56: The method according to aspect 55, wherein the first prism and the second prism are adjacent.

[0312] Aspect 57: The method according to any one of aspects 55 or 56, wherein a bridge connects a first edge of the first prism and a second edge of the second prism, and wherein the bridge is configured to: prevent light from reflecting from at least one of the first edge of the first prism and the second edge of the second prism.

[0313] Aspect 58: The method according to any one of aspects 55 to 57, wherein the first prism includes at least one beveled edge, and wherein the second prism includes at least one beveled edge.

[0314] Aspect 59: The method according to any one of aspects 55 to 58, wherein the first prism includes at least one edge having a light absorbing coating, and wherein the second prism includes at least one edge having a light absorbing coating.

[0315] Aspect 60: The method according to any one of aspects 55 to 59, wherein the first path is the path of the first light before the first light enters the first prism, and wherein the second path is the path of the second light before the second light enters the second prism.

[0316] Aspect 61: The method according to any one of aspects 55 to 60, wherein the first prism includes a first reflective surface configured to reflect the first light, and wherein the second prism includes a second reflective surface configured to reflect the second light.

[0317] Aspect 62: The method according to any one of aspects 55 to 61, wherein the first path is the path of the first light after the first light enters the first prism but before the first reflecting surface reflects the first light, and wherein the second path is the path of the second light after the second light enters the second prism but before the second reflecting surface reflects the second light.

[0318] Aspect 63: The method according to any one of aspects 50 to 62, wherein the first image and the second image are captured simultaneously.

[0319] Aspect 64: The method according to any one of aspects 50 to 63, wherein the first light redirecting element is fixed relative to the first image sensor, and wherein the second light redirecting element is fixed relative to the second image sensor.

[0320] Aspect 65: The method according to any one of aspects 50 to 64, wherein a first flat surface of the first image sensor faces a first direction, and wherein a second flat surface of the second image sensor faces a second direction parallel to the first direction.

[0321] Aspect 66: The method according to any one of aspects 50 to 64, further comprising: modifying at least one of the first image and the second image using luminance uniformity correction.

[0322] Aspect 67: An apparatus for digital imaging, the apparatus comprising units for performing operations according to any one of aspects 31 to 66.

[0323] Aspect 68: A computer-readable storage medium storing instructions which, when executed, cause one or more processors to perform operations according to any one of aspects 31 to 66.

[0324] Aspect 69: An apparatus for digital imaging, the apparatus comprising: a memory; and one or more processors configured to: receive a first image of a scene captured by a first image sensor, wherein a first light redirecting element is configured to redirect first light from a first path to a redirected first path towards the first image sensor, wherein the first image sensor is configured to capture the first image based on receiving the first light at the first image sensor; receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element is configured to redirect second light from a second path to a redirected second path towards the second image sensor, wherein the second image sensor is configured to capture the second image based on receiving the second light at the second image sensor, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; modify at least one of the first image and the second image using perspective distortion correction; and generate a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

[0325] Aspect 70: The apparatus according to aspect 69, wherein the one or more processors are configured to perform the operations according to any one of aspects 32 to 49 or aspects 51 to 66.

[0326] Aspect 71: A method for digital imaging, the method comprising: receiving a first image of a scene captured by a first image sensor, wherein a first light redirecting element is configured to redirect first light from a first path to a redirected first path towards the first image sensor, wherein the first image sensor is configured to capture the first image based on the first light received at the first image sensor; receiving a second image of the scene captured by a second image sensor, wherein a second light redirecting element is configured to redirect second light from a second path to a redirected second path towards the second image sensor, wherein the second image sensor is configured to capture the second image based on the second light received at the second image sensor, wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; using perspective distortion correction to modify at least one of the first image and the second image; and generating a combined image from the first image and the second image in response to the modification of at least one of the first image and the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is greater than at least one of a first field of view of the first image and a second field of view of the second image.

[0327] Aspect 72: The method according to aspect 71, further comprising the operations according to any one of aspects 32 to 49 or aspects 51 to 66.

Claims

1. An apparatus for digital imaging, the apparatus comprising: a memory; and one or more processors configured to: receive a first image of a scene captured by a first image sensor, wherein a first light redirecting element is configured to redirect first light from a first path to a redirected first path towards the first image sensor using at least a first prism of the first light redirecting element, wherein the first image sensor is configured to capture the first image based on the first light received at the first image sensor; receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element is configured to: redirect second light from a second path to a redirected second path towards the second image sensor using at least a second prism of the second light redirecting element, wherein the second image sensor is configured to capture the second image based on the second light received at the second image sensor, wherein a bridge connects a first edge of the first prism and a second edge of the second prism, and wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; modify at least one of the first image or the second image using perspective distortion correction; and generate a combined image from the first image and the second image in response to the modification of at least one of the first image or the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is larger than at least one of a first field of view of the first image and a second field of view of the second image.

2. The device according to claim 1, wherein To modify at least one of the first image or the second image using the perspective distortion correction, the one or more processors are configured to: modify the first image from depicting a first perspective to depicting a common perspective using the perspective distortion correction; and modify the second image from depicting a second perspective to depicting the common perspective using the perspective distortion correction, wherein the common perspective is between the first perspective and the second perspective.

3. The device according to claim 1, wherein To modify at least one of the first image or the second image using the perspective distortion correction, the one or more processors are configured to: identify a depiction of one or more objects in the image data of at least one of the first image or the second image; and modify the image data at least in part by projecting the image data based on the depiction of the one or more objects.

4. The device according to claim 1, wherein To generate the combined image from the first image and the second image, the one or more processors are configured to: align a first portion of the first image with a second portion of the second image; and stitch the first image and the second image together based on the alignment of the first portion of the first image with the second portion of the second image.

5. The apparatus according to claim 1, further comprising: the first image sensor; the second image sensor; the first light redirecting element; and the second light redirecting element.

6. The apparatus according to claim 1, wherein: the first light redirecting element includes a first reflective surface, wherein, in order to redirect the first light towards the first image sensor, the first light redirecting element uses the first reflective surface to reflect the first light towards the first image sensor; and the second light redirecting element includes a second reflective surface, wherein, in order to redirect the second light towards the second image sensor, the second light redirecting element uses the second reflective surface to reflect the second light towards the second image sensor.

7. The apparatus according to claim 1, wherein: the first prism is configured to refract the first light; and the second prism is configured to refract the second light.

8. The apparatus according to claim 1, wherein The first prism and the second prism are adjacent.

9. The device according to claim 1, wherein The bridge is configured to: prevent light from reflecting from at least one of the first edge of the first prism or the second edge of the second prism.

10. The device according to claim 1, wherein, The first prism includes at least one chamfered edge, and wherein, the second prism includes at least one chamfered edge.

11. The device according to claim 1, wherein, The first prism includes at least one edge having a light absorbing coating, wherein, the second prism includes at least one edge having the light absorbing coating.

12. The device according to claim 1, wherein, The first path is the path of the first light before the first light enters the first prism, wherein, the second path is the path of the second light before the second light enters the second prism.

13. The device according to claim 1, wherein, The first prism includes a first reflective surface configured to reflect the first light, wherein, the second prism includes a second reflective surface configured to reflect the second light.

14. The apparatus according to claim 13, wherein, The first path is the path of the first light after the first light enters the first prism but before the first reflective surface reflects the first light, wherein, the second path is the path of the second light after the second light enters the second prism but before the second reflective surface reflects the second light.

15. The device according to claim 1, wherein The first image and the second image are captured simultaneously.

16. The device according to claim 1, wherein, The first light redirecting element is fixed relative to the first image sensor, wherein, the second light redirecting element is fixed relative to the second image sensor.

17. The device according to claim 1, wherein The first flat surface of the first image sensor faces a first direction, wherein, the second flat surface of the second image sensor faces a second direction parallel to the first direction.

18. The device according to claim 1, wherein The one or more processors are configured to: modify at least one of the first image or the second image using luminance uniformity correction.

19. A method for digital imaging, the method comprising: receiving a first image of a scene captured by a first image sensor, wherein, a first light redirecting element redirects first light from a first path to a redirected first path towards the first image sensor at least using a first prism of the first light redirecting element, wherein, the first image sensor captures the first image based on receiving the first light at the first image sensor; Receive a second image of the scene captured by a second image sensor, wherein a second light redirecting element redirects second light from a second path to a redirected second path towards the second image sensor using at least a second prism of the second light redirecting element, wherein the second image sensor captures the second image based on receiving the second light at the second image sensor, wherein a bridge connects a first edge of the first prism and a second edge of the second prism, and wherein a virtual extension of the first path beyond the first light redirecting element intersects a virtual extension of the second path beyond the second light redirecting element; Modify at least one of the first image or the second image using perspective distortion correction; and Generate a combined image from the first image and the second image in response to the modification of at least one of the first image or the second image using the perspective distortion correction, wherein the combined image includes a combined image field of view that is greater than at least one of a first field of view of the first image and a second field of view of the second image.

20. The method according to claim 19, wherein, Modifying at least one of the first image or the second image using the perspective distortion correction includes: Modifying the first image from depicting a first perspective to depicting a common perspective using the perspective distortion correction; and Modifying the second image from depicting a second perspective to depicting the common perspective using the perspective distortion correction, wherein the common perspective is between the first perspective and the second perspective.

21. The method according to claim 19, wherein, Modifying at least one of the first image or the second image using the perspective distortion correction includes: Identifying a depiction of one or more objects in the image data of at least one of the first image or the second image; and Modifying the image data by projecting the image data based on the depiction of the one or more objects.

22. The method according to claim 19, wherein, The first light redirecting element includes a first reflective surface, wherein, in order to redirect the first light towards the first image sensor, the first light redirecting element reflects the first light towards the first image sensor using the first reflective surface; and The second light redirecting element includes a second reflective surface, wherein, in order to redirect the second light towards the second image sensor, the second light redirecting element reflects the second light towards the second image sensor using the second reflective surface.

23. The method according to claim 19, wherein, The first prism is configured to refract the first light; and The second prism is configured to refract the second light.

24. The method according to claim 19, wherein, The first prism and the second prism are adjacent.

25. The method according to claim 19, wherein The bridge is configured to: prevent light from reflecting from at least one of the first edge of the first prism or the second edge of the second prism.

26. The method according to claim 19, wherein, The first path is the path of the first light before the first light enters the first prism, and the second path is the path of the second light before the second light enters the second prism.

27. The method according to claim 19, wherein, The first prism includes a first reflective surface configured to reflect the first light, wherein the second prism includes a second reflective surface configured to reflect the second light.

28. The method according to claim 27, wherein The first path is the path of the first light after the first light enters the first prism but before the first reflective surface reflects the first light, wherein the second path is the path of the second light after the second light enters the second prism but before the second reflective surface reflects the second light.

29. The method according to claim 19, wherein The first image and the second image are captured simultaneously.

30. The method according to claim 19, further comprising: modifying at least one of the first image or the second image using luminance uniformity correction.

Images