Panoramic camera system for enhanced detection

Abstract

The present application generally describes imaging systems, such as multi-camera imaging systems.SOLUTION: The imaging system may include a plurality of channels, and an individual channel may include an objective lens and relay optics. The objective lens images the received light as a first image on a first imaging plane, and the relay optics images the first image as a second, magnified image on a second imaging plane. In examples, the objective lens and the relay optics constitute an optically coherent system.SELECTED DRAWING: Figure 15A

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This disclosure is incorporated herein by reference in its entirety. This disclosure is incorporated herein by reference in its entirety. This disclosure is incorporated herein by reference in its entirety. This disclosure is incorporated herein by reference in its entirety. The four listed international applications each claim priority to U.S. Provisional Patent Application No. 62 / 972,532, entitled "Opto-Mechanics of Panoramic Capture Devices with Abutting Cameras," filed December 23, 2019, and U.S. Provisional Patent Application No. 62 / 952,973, entitled "Opto-Mechanics of Panoramic Capture Devices with Abutting Cameras," filed December 23, 2019, and U.S. Provisional Patent Application No. 62 / 952,983, entitled "Multi-camera Panoramic Image Capture Devices with a Faceted Dome," filed December 23, 2019. The first three international applications listed above each also claim priority to U.S. Provisional Patent Application No. 62 / 865,741, filed June 24, 2019. The entirety of each of the eight applications listed above is incorporated herein by reference.

[0002] The present disclosure relates to a panoramic low-parallax multi-camera capture device having multiple adjacent polygonal cameras. The present disclosure also relates to an optical and optomechanical design of the cameras that captures incident light from polygonal fields of view to form polygonal images, which also provides enhanced detection that can enable improved situational awareness of an environment or scene or events occurring therein. [Background technology]

[0003] While panoramic cameras have existed for decades, the technology is evolving to meet enhanced and emerging market opportunities, including in image capture for film, virtual reality, sports and entertainment, security, mapping, and autonomous vehicle navigation. Panoramic cameras have substantial value due to their ability to simultaneously capture wide-angle images. The earliest such example is the fisheye lens, an ultra-wide-angle lens that produces strong visual distortion intended to create wide panoramic or hemispherical images. While the field of view (FOV) of a fisheye lens is typically between 100 and 180 degrees, this approach has been extended to even wider angles, including a range of 220 to 270 degrees, as provided by Y. Shimizu in U.S. Patent 3,524,697. Alternatively, there are mirror- or reflection-based cameras that capture circular panoramic images, such as the system proposed by P. Greguss in U.S. Patent 4,930,864. As another example, U.S. Patent 9,451,162 to A. Van Hoff et al. of Jaunt Inc. provides a panoramic multi-camera device in which multiple cameras are arranged around or around the circumference of a sphere in a manner that captures images with sparse but overlapping FOVs, thereby enabling the capture of a complete panoramic image as a whole.

[0004] Some panoramic multi-camera systems employ multiple cameras arranged around or around a sphere, with adjacent cameras touching along some or all of their adjacent edges. For example, U.S. Patent 7,515,177 to K. Yoshikawa and U.S. Patent 10,341,559 to Z. Niazi describe imaging devices with multiple adjacent image pickup units (cameras), whose design goal includes reducing parallax error in images captured by adjacent cameras. Parallax is the visual perception that an object's position or orientation appears different when viewed from different directions. In Yoshikawa's example, images are collected from cameras with partially overlapping fields of view to compensate for mechanical errors. The presence of parallax error significantly slows efforts to properly combine, stitch, and synthesize a larger panoramic image from images captured by adjacent cameras. However, in other systems, the presence of parallax image differences can provide useful data. For example, in U.S. Patent 6,947,059 to D. Pierce et al., multiple offset cameras capture images panoramically, with adjacent cameras capturing images with overlapping sub-images, providing stereoscopic (or depth) image capture throughput for a panoramic FOV.

[0005] Other image capture techniques are known for capturing depth or motion information from an environment, such as the relative distance or position of an object from a camera. The resulting data can then be used to enable or enhance situational awareness of an environment or scene, or events arising therefrom. The resulting optical or image data can be used by autonomous vehicles, including drones or robots, or to provide information to drivers or pilots of aircraft, cars or trucks, or air vehicles, or for numerous other purposes.

[0006] As one approach, bright-field image capture allows for the simultaneous capture of image data from multiple planes. This allows the depth or resolution of an object to then be inspected in detail. Separately, autonomous vehicle navigation is also made possible in part by advances in spatial sensing technology, particularly that for LIDAR. LIDAR, an acronym for a set of Light-Detection-and-Ranging technologies, is the term used for sensors that emit pulses of light and measure the time delay between the emission and reception of these pulses. LIDAR is a form of remote sensing that allows for the creation of three-dimensional maps of volumes or areas in proximity to the LIDAR unit or an attached object. In the field, these maps are known as point clouds, which are collections of points that represent 3D shapes or features. Each point has its own set of X, Y, and Z coordinates and, in some cases, additional attributes. While rapid development of LIDAR is currently driven by efforts to enable autonomous vehicle navigation, the technology may have broader potential applications, including robotic navigation, mapping, archaeology, and architecture. As yet another alternative, neuromorphic or event sensors, including the Oculi SPU, which are much more sensitive to signal strength or temporal changes than standard image sensors, can be used to provide optical or image data for enhanced situational awareness.

[0007] However, capturing and processing LIDAR or optical point cloud data in real time to assist vehicle navigation can be very challenging, and therefore LIDAR or event detection resolution is typically not as great as images captured by cameras. For less urgent situations or other applications, geometric calibration, combination, and comparison of images of objects in a scene with depth or event data can be beneficial. However, when using closely spaced (but not integrated) or continuously repositioned cameras and depth or event detection systems, there can be significant problems in addressing offset alignment or parallax errors. Therefore, there is an opportunity to provide an enhanced system for capturing combined image data and depth data of objects in a scene or environment, particularly for panoramic image and depth or event data capture. [Brief explanation of the drawings]

[0008] [Figure 1] 1 shows a portion of a multi-camera device capture device, and specifically two adjacent cameras thereof. [Figure 2A] 1 shows a cross section of a portion of a low parallax camera lens assembly, including lens elements and ray paths. [Figure 2B] 1 shows a cross section of a portion of a low parallax camera lens assembly, including lens elements and ray paths. [Figure 3] 1 shows a cross-sectional view of a portion of a multi-camera capture device, illustrating FOV overlap, field of view, overlap, seams, and blind areas. [Figure 4] Illustrates the general concept of laser ranging or LIDAR optical systems. [Figure 5A] 1 shows the optical geometry for the field of view for adjacent hexagonal and pentagonal lenses, as may occur in a device with a truncated icosahedral geometry. [Figure 5B] The enlarged area of ​​Figure 5A is shown in further detail. [Figure 5C]FIG. 5C shows an example of a paraxial NP point or low parallax (LP) quantity located near both the entrance pupil and the device center. [Figure 5D] The parallax difference for two adjacent cameras is shown relative to the perspective center. [Figure 5E] 1 shows the front color at the edge of the outer compressor lens element. [Figure 5F] 2C shows a graph of computational residuals versus perspective center difference for a low parallax camera or objective of the type shown in FIGS. 2A and 2B. [Figure 6] 1 shows a distortion correction curve plotted on a graph showing the percentage of distortion versus a partial field of view. [Figure 7] The fields of view for adjacent cameras are shown, including both core and extended fields of view (FOV), both of which can be useful for the design of optimized panoramic multi-camera capture devices. [Figure 8] 1 shows an improved design for a low parallax camera lens or objective lens with multiple compressor lenses. [Figure 9] An improved camera lens design is presented, which functions as an objective lens, in combination with a refractive relay optical imaging system. [Figure 10] 1 shows an electronic system diagram for a multi-camera capture device. [Figure 11] 1 shows a cross-sectional view of an improved optomechanical structure for a multi-camera capture device and a 3D view of its camera channels. [Figure 12A] 12 shows a perspective view of an alternative design to that shown in FIG. 11 for attaching the camera channels to each other and to the central support. [Figure 12B] FIG. 12B shows a perspective view of a portion of an alternative design to that shown in FIG. 12A, providing further detail regarding the interface of the secondary channel to the primary channel. [Figure 13] 10 shows a view of another alternative design approach to the center support to which the camera channel can be attached. [Figure 14]1 illustrates an alternative configuration for an improved multi-camera capture device. [Figure 15A] 1 shows a camera objective paired with relay optics including a beam splitter that directs image light to an additional optical subsystem. [Figure 15B] 10 shows an example of an alternative optical design for a camera objective paired with a relay optical system. [Figure 16A] 15A and 15B show a relay optics section of the type shown in the system of FIGS. 15A and 15B, but further including a laser ranging subsystem with MEMs mirrors. [Figure 16B] 15A-B shows a relay optics section of the type shown in the system of FIGS. 15A-B, but further including a laser ranging subsystem with an optical phased array. [Figure 16C] We present a conceptual integration of the IR depth sensing optical path with the camera objective used in low-parallax imaging. [Figure 16D] 15A-B shows a relay optics section of the type shown in the system of FIGS. 15A-B, but further includes a laser ranging subsystem having a laser array light source. [Figure 16E] 15A and 15B, but further includes a depth sensing system with an image sensor and brightfield micro-optics. DETAILED DESCRIPTION OF THE INVENTION

[0009] As commonly understood in the field of optics, a lens or lens assembly typically comprises a system or device having multiple lens elements mounted within a lens barrel or housing and working together to generate an optical image. An imaging lens captures a portion of light coming from an object or objects residing in object space at some distance or distances from the lens system. The imaging lens can then form images of these objects at an output "plane," with the images having a finite size determined by magnification, as determined by the focal length of the imaging lens and the conjugate distances to the object(s) and image plane for that focal length. The amount of image light passing through the lens, from object to image, is largely determined by the size of the imaging lens's aperture stop, which is typically quantified by one or more values ​​for numerical aperture (NA) or F-number (F# or F / #).

[0010] The image quality provided by an imaging lens is determined by many characteristics of the lens design, including the choice of optical materials used in the design, the size, shape (or curvature) and thickness of the lens elements, the relative spacing of the lens elements from one another, the spectral bandwidth, polarization, the optical load (power or flux) of the transmitted light, optical diffraction or scattering, and / or lens manufacturing tolerances or errors. Image quality is typically described or quantified in terms of lens aberrations (e.g., spherical, coma, astigmatism, or distortion) or the relative size of the resolved spot provided by the lens. The resolution provided by an imaging lens is typically quantified by its modulation transfer function (MTF).

[0011] In a typical electronic or digital camera, an image sensor is nominally located at the imaging plane. This image sensor is typically a CCD or CMOS device that is physically attached to a heat sink or other heat removal means and also includes electronics that power the sensor and readout and communication circuitry that provides image data to data storage or image processing electronics. The image sensor typically has a color filter array (CFA), such as a Bayer filter, within the device, with the color filter pixels aligned with the image pixels to provide an RGB (red, green, blue) pixel array. Alternative filter array patterns can alternatively be used, including CYGM (cyan, yellow, green, magenta) filters or RGBW filter arrays (W=white).

[0012] In typical usage, many digital cameras are used by people or remote systems in relative isolation to capture images or photographs of a scene without any dependency or interaction with any other camera devices. In some cases, such as surveillance or security, camera operation can be directed by people or algorithms based on image content seen from another camera that has already captured overlapping, adjacent, or nearby image content. In another example, people capture panoramic images of a scene, such as a landscape scene, with an extended or wide-angle FOV by sequentially capturing a series of adjacent images while manually or automatically moving or panning to assemble the adjacent images. Image processing software, such as Photoshop or Lightroom, can then be used to stitch, mosaice, or tile the adjacent images together to represent a larger, extended scene. Image stitching or photo stitching is the process of combining multiple photographic images with overlapping fields of view to generate a segmented panoramic or high-resolution image. Image quality improvements, including exposure or color correction, can also be applied either in real time, or in a post-processing or image rendering phase, or a combination thereof.

[0013] Unless objects in a scene are directionally illuminated and / or have directional optical responses (e.g., through reflection), the available light is plenoptic, meaning that there is light traveling in all directions, or approximately so, within a given space or environment. A camera can then sample a subset of this light as image light and use it to provide a generated image showing a given view or perspective of different objects in the scene at one or more times. If the camera is moved to a different nearby location and used to capture another image of that same scene portion, both the apparent perspective and relative positions of the objects change. In the latter case, one object may now partially occlude another, while a previously hidden object becomes at least partially visible. These differences in the apparent positions or orientations of objects are known as parallax. Specifically, parallax is the displacement or difference in the apparent positions of objects viewed along two different lines of sight, measured by the angle or half-angle of inclination between these two lines.

[0014] In stereoscopic image capture or projection systems, dual-view parallax is a cue, along with shading, occlusion, and perspective, that can provide the perception of depth. For example, in a stereoscopic (3D) projection system, pairs of polarized or spectrally encoded images are projected superimposed on a screen and can be viewed by audience members wearing appropriate glasses. The amount of parallax may have an optimal range, outside of which the resulting perception of depth is either too small to be practically noticed by audience members or too large to be properly fused by the human visual system.

[0015] On the other hand, in panoramic image capture applications, parallax differences can be considered an error that can complicate both the stitching and appearance of the images. In the example of manually capturing a series of individual panoramic landscape images, if objects in the scene are sufficiently far apart (e.g., optically infinity), visual differences in perspective or parallax across the images may be too small to notice. Panoramic capture devices integrated with rotating or multi-cameras have the potential to continuously capture real-time image data at high resolution without relying on the uncertainty of manual capture. However, such devices may also introduce errors, including their own binocular parallax, image artifacts, or those of parallax, perspective, and exposure. While the resulting images can often be successfully stitched together using image processing algorithms, input image errors complicate image processing, lengthening processing times, and sometimes leaving visually apparent residual errors.

[0016] To provide context, FIG. 1 illustrates an improved integrated panoramic multi-camera capture device 100 with two adjacent cameras 120 within a housing 130 designed for reduced-parallax image capture. These cameras are alternatively referred to as image pickup units, or camera channels, or objective lens systems. Each camera 120 has multiple lens elements (see FIG. 2) mounted within a lens barrel or housing 130. Adjacent outer lens elements 137 have adjacent beveled edges 132, positioning one camera channel in close proximity to another, but which may not be touching and are therefore separated by a gap or seam 160 of finite width. A portion of the available light (λ), or light rays 110, from the scene or object space 105 enters the camera 120 and becomes image light that is captured within a constrained FOV and directed to the image plane, while other light rays do not reach the camera at all. Some light rays 110 propagate into the camera and pass through the lens elements forming as field edge chief rays 170, or marginal rays, while other light rays can potentially propagate through the lens elements, creating stray or ghost light and false bright spots or images. As an example, some light rays (167) incident at a large angle on the outer surface of outer lens element 137 can travel complex paths through the camera's lens elements and create ghost images that are detectable at image plane 150.

[0017] More specifically, FIG. 2A shows a cross section of a portion of a camera 120 having a set of lens elements 135 mounted within a housing (130, not shown) within a portion of an integrated panoramic multi-camera capture device 100. A fan of light rays 110 from object space 105 diverges from on-axis to the off-axis full-field chief ray range and strikes outer lens element 137, where it is refracted and transmitted inward. Refracted and transmitted through further inner lens element 140 and aperture stop 145, this image light 115 converges to a focused image at or near image plane 150, where an image sensor (not shown) is typically located. Lens system 120 of FIG. 2A can also be defined as having a lens format consisting of outer lens element 137, or compressor lens element, and inner lens element 140, the latter of which can also be defined as consisting of a wide-angle lens group before the stop (pre-stop) and an eyepiece-like lens group after the stop (post-stop). This compressor lens element (137) directs the image light 115 sharply inward, compressing the light and allowing the overall lens assembly to provide a short focal length, while also helping to allow the camera lens housing or barrel the room necessary to provide the mechanical features necessary both to hold or mount the lens elements and to properly interact with the barrel or housing of an adjacent camera. Image light passing through the camera lens assembly from the outer lens element 137 to the image plane 150 provides an image having image quality quantifiable by image resolution, image contrast, depth of focus, and other attributes, defined by the optical aberrations (e.g., astigmatism, distortion, or spherical) and chromatic or spectral aberrations encountered by the passing light at each of the lens elements (137, 140) in the camera 120. Figure 2B shows a chief ray 170, or fan of marginal rays, incident along or near the beveled edge 132 of the outer lens element 137 of the camera optics (120) shown in Figure 2A. FIG. 2B also shows a portion of the captured polygonal or asymmetric FOV 125 extending from the optical axis 185 to a line coincident with the edge ray.

[0018] 2A , outer lens element 137 functions as a compressor lens element that redirects passing image light 115 toward second lens element 142, which is the first lens element in a group of inner lens elements 140. In this design, second lens element 142 has a very concave shape, much like the outer lens elements used in fisheye-type imaging lenses. This compressor lens element serves to sharply direct image light 115 inward, or bend the light rays, allowing the overall lens assembly to provide a short focal length, while also allowing the camera lens housing 130 or barrel the room necessary to provide the mechanical features necessary both to hold or mount lens element 135 and to properly interact with an adjacent camera barrel or housing. However, with good lens and optomechanical design, and appropriate sensor selection, camera 120 can be designed with a lens assembly that supports image resolutions of 20-30 pixels / degree, even 110 pixels / degree or more, depending on the application and device configuration.

[0019] The resulting image quality from these cameras is determined by light scattered at surfaces or within lens elements, as well as light reflected or transmitted by each lens surface. Surface transmittance and camera lens system efficiency can be improved by the use of anti-reflection (AR) coatings. Image quality can also be determined by the results of non-image light. Referring back to FIG. 1, another portion of the available light may be primarily reflected by the outer lens element 137. Still other light entering the camera 120 can be blocked or absorbed by some combination of blackened areas (not shown) provided at or near the aperture stop, the inner lens barrel surface, the edges of the lens elements, internal baffles or light-trapping features, the field stop, or other surfaces. Still other light entering the camera can also become stray or ghost light potentially visible at the image plane.

[0020] The overall image quality obtained by multiple adjacent cameras 120 in an improved integrated panoramic multi-camera capture device 100 (e.g., FIG. 1) may also depend on various other factors, including camera-to-camera variations in focal length and / or track length, as well as the magnification provided by the individual cameras. These parameters may vary depending on factors including variations in glass refractive index, variations in lens element thickness and curvature, and variations in lens element mounting. As an example, images that are tiled or mosaiced together from multiple adjacent cameras typically need to be rectified, one after the other, to correct for image size changes due to camera magnification differences (e.g., ±2%).

[0021] The images produced by the multiple cameras in the integrated panoramic multi-camera capture device 100 may vary in other respects, resulting in image quality and mosaicing or tiling of the images. Specifically, the directional pointing or collection of image light through the lens elements to the image sensor of any given camera 120 may vary depending on whether the camera is angularly distorted or asymmetric.

number

[0022] In contrast to the system of FIG. 1, in a typical commercially available panoramic camera, the seams between cameras are complete gaps that can be 30-50 mm wide or even larger. Specifically, as shown in FIG. 3, a panoramic multi-camera capture device 101 may have adjacent cameras 120 or camera channels separated by large gaps or seams 160, leaving a blind spot or area 165 between them where no camera can capture an image. The actual physical seam 160 between adjacent camera channels or outer lens elements 137 (FIGS. 1 and 3) can be measured in various ways: as the actual physical distance between adjacent lens elements or lens housings, as the angular extent of the lost FOV, or as the number of "lost" pixels. However, the optical seam can be even larger, as the distance between the outer chief ray of one camera and another, due to any gaps in light reception caused by vignetting or coating limitations. For example, anti-reflection (AR) coatings are typically not applied to the edges of the optical system, and an offset margin is provided to accommodate the coated aperture (CA).

[0023] To compensate for both camera misalignment and large seams 160 and to reduce the size of the blind area 165, a typical panoramic multi-camera capture system 101 ( FIG. 3 ) has each individual camera 120 capture image light 115 from a wide FOV 125 that provides overlap 127, thereby reducing the blind area 165 and potentially lost image content. As another example, in most commercially available multi-camera capture systems 101, the gaps are 25-50+ mm wide, and the compensated FOV overlap between cameras is similarly large; for example, the portion of the FOV 125 that overlaps and is captured by two adjacent cameras 120 may be 10-50% of the cameras' FOV. The presence of such large image overlap from the shared FOV 125 wastes potential image resolution, increases image processing and image stitching time, and introduces significant image parallax and perspective errors. These errors complicate image stitching because they must be corrected or averaged out during the stitching process. In such systems, disparity is not predictable because it varies as a function of object distance. If the object distance is known, disparity can be predicted for a given field of view and inter-camera spacing. However, because the object distance is typically unknown, disparity errors result, complicating image stitching. Optical flow and general stitching algorithms determine object depth and enable image stitching, but they incur processing and time burdens.

[0024] Similarly, in a panoramic multi-camera capture device 100 of the type shown in FIG. 1 with closely integrated cameras, the width and structure of the seam 160 can be a significant factor in the operation of the overall device. However, the seam can be smaller than in FIG. 3, with the effective optical seam between the FOV edges of two adjacent cameras determined by both optical and mechanical contributions. For example, by using standard optical engineering techniques to construct lens assemblies within a housing, the mechanical width of the seam 160 between the outer lens elements 137 of adjacent cameras can be reduced to 4-6 mm. For example, it is standard practice to mount lens elements within a lens barrel or housing with a minimum radial width of 1-1.5 mm, particularly near the outermost lens element. Then, consider standard coated apertures or coating margins, as well as possible vignetting, entrance pupil aberrations, front color, and chip edges, and attempt to mount adjacent lens assemblies or housings closely together using standard techniques. Therefore, when considering both optics and mechanics, the optical seam width between adjacent lenses can easily be 8-12 mm or more.

[0025] Wide-angle imaging can be useful for cinematic or VR image capture, sports or event imaging, mapping or photogrammetry, security or surveillance, and / or numerous other applications. Broadly speaking, imaging technologies such as that of FIG. 1 can also enable situational awareness, which is the recognition of environmental elements and events with respect to time or space, understanding their meaning, and predicting their future state. For security applications, situational awareness involves the use of perception systems to scan the environment with the intent of identifying current or anticipated future threats. Optical sensing for situational awareness can be enabled by traditional image sensors and camera systems, by ranging technologies such as LIDAR, radar, sonar, or the like, and / or by emerging technologies such as event sensors.

[0026] As an example, compared to panoramic multi-camera capture devices, which optically sample an environment by collecting and imaging a portion of plenoptic light, LIDAR systems are used to purposefully illuminate, scan, or sweep an environment or scene with laser light. This emitted laser light reflects off objects in the environment and returns to the LIDAR system's sensor. The sensor detects the returned light and generates a signal that can distinguish the returned light from ambient light. LIDAR systems also determine the position or motion rate and trajectory of detected objects within a detectable range of the LIDAR device. LIDAR is similar to laser ranging, but, as commonly understood, is extended to detect the location of objects throughout a three-dimensional environment. For example, as shown in FIG. 4, a typical LIDAR system 1000 includes a laser light source 1010 that emits laser light (λ) that is directed to illuminate objects (171-173) within an environment 1070. The illuminating laser light, whether scanned, swept, or flashed, can be modified by illumination optics 1020 to illuminate portions of environment 1070. The laser light (λ) may then scatter, reflect, or diffract from these objects, and a portion of that redirected laser light (λ') can then be collected by optics 1025 onto optical sensor 1030. The resulting signal can be examined by processing electronics 1040 to determine the relative positions of objects within the scene or environment.

[0027] Alternatively, technologies for enabling enhanced situational awareness, such as those provided by the present invention, such as large-scale, high-resolution image sensors, event sensors, LIDAR or laser ranging optics, bright field or other depth-sensing optics, can be integrated into an improved low-parallax multi-camera panoramic capture device (300) with appropriately designed optics. The improved panoramic multi-camera capture device may have multiple cameras arranged around a spherical or polyhedral shape, or around the circumference of a sphere, to capture a 360-degree panoramic FOV. A polyhedron is a three-dimensional solid composed of a collection of polygons adjoined at their edges. One polyhedral shape is that of a dodecahedron, with 12 sides, each shaped as a regular pentagon. A dodecahedron-shaped panoramic multi-camera capture device has cameras with pentagonal outer lens elements that image at a nominal full wide angle of 69.1°. Another shape is that of a truncated icosahedron, like a soccer ball, which has a combination of 12 regular pentagonal sides or faces, 20 regular hexagonal sides or faces, 60 vertices, and 90 edges. More complex shapes with more sides or facets, such as regular polyhedra, Goldberg polyhedra, or octagonal-sided shapes, or even some irregular polyhedral shapes, may be useful. Typically, a 360° polyhedral camera does not capture a full spherical FOV because at least a portion of one facet is sacrificed to allow for a support feature, such as a mounting post.

[0028] 1 and 2B, camera channel 120 resembles a truncated cone, or a portion thereof, where a truncated cone is a geometric solid (usually a cone or pyramid) that exists between one or two parallel planes that intersect it. In that context, the fan of chief rays 170 corresponding to the polygonal edges is refracted by outer compressor lens element 137 to nominally coincide with the edges of the truncated cone of the polyhedral geometry.

[0029] To help illustrate some of the issues related to camera geometry, FIG. 5A shows a cross section of a pentagonal lens 175 capturing a pentagonal FOV 177 and a hexagonal lens 180 capturing a hexagonal FOV 182, representing a pair of adjacent cameras whose outer lens elements have pentagonal and hexagonal shapes, such as might occur in a truncated icosahedron, or soccer-ball-shaped panoramic multi-camera capture device (e.g., 100, 300). The theoretical hexagonal FOV 182 spans a 20.9° half FOV along the side, or a 41.8° full FOV (θ1), with the FOV near the vertices being even larger. The pentagonal FOV 177 supports a 36.55° FOV (θ2) within a circular region, with the FOV closer to the corners or vertices being even larger. Notably, in this cross section, the pentagonal FOV 177 is asymmetric, supporting a 20 degree FOV on one side of the optical axis 185 and only a 16.5 degree FOV on the other side of the optical axis.

[0030] Optical lenses are typically designed using programs such as ZEMAX or Code V. The success of a design typically depends in part on selecting the best or most appropriate lens parameters, identified as operands, for use in a merit function. This is also true when designing lens systems for improved low-parallax multi-camera panoramic capture devices, for which there are several factors that affect performance (including, in particular, parallax) and several parameters that can be optimized individually or collectively to control it. One approach targets optimization of the "NP" point, or more significantly, a variant thereof.

[0031] By way of background, in the field of optics, there is the concept of an entrance pupil, which is the projected image of the aperture stop as seen from object space, or the virtual opening toward which image rays from object space appear to propagate before any refraction by the first lens element. By standard techniques, the location of the entrance pupil can be found by identifying the paraxial chief ray from object space 105 passing through the center of the aperture stop and projecting or extending that object space in a direction toward where it strikes the optical axis 185. In optics, an incident Gaussian or paraxial ray is understood to lie within an angular range of 10° or less from the optical axis, corresponding to a ray directed toward the center of the aperture stop, which also defines the entrance pupil location. Depending on the lens characteristics, the entrance pupil may be larger or smaller than the aperture stop and located in front of or behind the aperture stop.

[0032] In contrast, in the field of low-parallax cameras, there is the concept of a no-parallax (NP) point, or viewpoint center. Conceptually, an "NP point" is associated with a high-FOV chief ray or principal ray incident on or near the outer edge of the outermost lens element, projecting or extending that object space in a direction toward where it strikes the optical axis 185. For example, depending on the design, a camera channel in a panoramic multi-camera capture device can support a half-FOV with a non-paraxial chief ray at angles >31° for dodecahedron-type systems or >20° for truncated icosahedron-type systems. This concept of NP point projection has been applied to the design of panoramic multi-camera capture devices in conjunction with expectations for chief ray propagation and parallax control for adjacent optical systems (cameras). It can also be argued that if a camera pivots around the NP point, or multiple cameras appear to rotate around a common NP point, the parallax error is reduced and images can be aligned with little or no parallax error or perspective difference. However, in the field of low-parallax cameras, the NP point is also identified with the entrance pupil, and the axial position of the entrance pupil is estimated using the first-order optics tangent relationship between the projection of the paraxial field angle on the first lens element (see Figures 2A and 2B) and the height of the incident ray.

[0033] Thus, confusingly, in the field of low-parallax camera design, NP points have previously been associated with both the projection of the FOV chief rays' edges and the projection of chief rays that lie within the Gaussian or paraxial region. As can be seen, in practice, they both have value. Specifically, NP points associated with paraxial entrance pupils can be useful in developing initial specifications for designing and describing lenses. NP points associated with non-paraxial edge field chief rays can be useful in targeting and understanding parallax performance and in defining the conical volume or truncated cone that a lens assembly can reside within.

[0034] The projection of these non-paraxial chief rays may miss the entrance pupil defined by the paraxial chief rays due to both lens aberrations and practical geometry-related factors associated with these lens systems. Regarding the former, in well-designed lenses, image quality at the image plane is typically prioritized by limiting the impact of aberrations on resolution, telecentricity, or other attributes. Within a lens system, aberrations at intermediate surfaces, including the aperture stop, can vary widely, with emphasis on the net sum at the image plane. While aberrations at the aperture stop are often somewhat controlled to avoid vignetting, non-paraxial chief rays do not need to pass through the center of the aperture stop or the entrance pupil located along the projection paraxial axis.

[0035] To further illustrate these concepts and enable the design of improved low-parallax lens systems, note that the camera lens system 120 in FIG. 2A shows both a first NP point 190A, corresponding to the entrance pupil defined by the vector projection of a paraxial chief ray from object space 105, and an offset second NP point 190B, corresponding to the vector projection of a non-paraxial chief ray from object space. Both of these ray projections intersect the optical axis 185 at positions behind both the lens system and the image plane 150. As will be explained later, ray behavior between and within the nearby regions of projection points 190A and 190B can be complex, and neither the projection position nor the projection point has a definitive value or size. While the projection of a chief ray intersects the optical axis at a point, the projections of groups of chief rays converge toward the optical axis and intersect at different locations, which may be tightly grouped (e.g., within a few microns or tens of microns), in which case the extent or size of that “point” may depend on the set of nearby chief rays used in the analysis. On the other hand, when designing a low-parallax imaging lens for imaging a large FOV, the axial distance or difference between NP points 190A and 190B provided by the projected paraxial and non-paraxial chief rays can be significantly large (e.g., millimeters). Thus, as also explained below, the axial difference represents a useful means of parallax optimization (e.g., low-parallax budget 188) of lens systems designed for current panoramic capture devices and applications. As can also be seen, the design of the improved device (300) can be optimized to locate the device geometric center, or device center 196, outside of, but close to, this low-parallax budget 188, or alternatively, within, and preferably close to, the non-paraxial chief ray NP point.

[0036] In one embodiment, FIG. 5A shows the projection of the theoretical edge of the field of view (FOV edge 155) past the outer lens elements (lenses 175 and 180) of two adjacent cameras to provide lines pointing toward a common point (190). These lines represent the theoretical limits of complex "conical" optomechanical lens assemblies, which are typically volume-limiting pentagonal or hexagonal cones. Again, ideally, in a parallax-free multi-camera system, the entrance pupils or NP points of two adjacent cameras are co-located. However, to avoid mechanical collisions, features of a given lens assembly, including the sensor package, generally should not protrude outside the camera system's frustum cone and into the conical space of an adjacent lens assembly. However, the actual lens assemblies in a multi-camera panoramic capture device are also separated by a seam 160. Thus, the actual chief ray 170 received at the lens edge, which is inside both the mechanical seam and the physical width or aperture of the attached outer lens element (lenses 175 and 180), when projected generally toward paraxial NP point 190, may instead arrive at offset NP point 192, separated by NP point offset distance 194.

[0037] This can be better understood by considering the expanded region AA proximate to the nominal or ideal point NP 190, as shown in detail in FIG. 5B. Within the hexagonal FOV 182, rays propagating within the Gaussian or paraxial region and passing through the nominal center of the aperture stop (e.g., paraxial ray 173) can be projected to the nominal NP point 190 (corresponding to the entrance pupil) or to an offset NP point 190A at a small NP point difference or offset 193 from the nominal NP point 190. Meanwhile, the actual hexagonal lens edge chief ray 170 associated with the largest inscribed circle within the hexagon can be projected to reach a common offset NP point 192A, which may be at an even larger offset distance (194A). The two adjacent cameras in FIGS. 5A and 5B may or may not also share a co-located NP point (e.g., 190). Distance offsets can arise for a variety of reasons, including geometric considerations between the cameras (adjacent hexagonal and pentagonal cameras), geometric asymmetries within the cameras (e.g., for pentagonal cameras), or due to constraints on the practical width of the seam 160, or due to directional differences between deviating light rays.

[0038] As just mentioned, there are also potential geometric differences in the projection of incident chief rays toward a simplified nominal "NP point" (190). First, incident imaging paths from the corners or vertices or near the mid-edge (mid-chord) of a hexagonal or pentagonal lens may or may not project to a common NP point within the described range between the nominal paraxial NP point 190 and the offset NP point 192B. Also, as shown in FIG. 5B, solely due to the geometric asymmetry of the pentagonal lens, for an actual accepted FOV, associated pairs of edge chief rays 170 and 171 can project to different nominal NP points 192B that can be separated both from the paraxial NP point (190) by offset distance 194B and from each other by offset distance 194C.

[0039] Another issue is that during lens design, the best performance typically occurs on-axis or near-axis (e.g., ≤0.3 normalized field of view), near the optical axis 185. In many lenses, good imaging performance often occurs at or near the field edge by design, in which case optimization weighting is often used to enforce compliance. The worst imaging performance can then occur in the intermediate field (e.g., 0.7-0.8 of the normalized image field height). Reviewing Figures 5A and 5B again, outside the paraxial region, but not as extreme as the edge chief ray (10°<θ<20.9°), intermediate off-axis rays from the intermediate field (θ) can project toward an intermediate NP point between the nominal NP point 190 and the offset NP point 192B. However, other, more extreme, off-axis rays, specifically from the 0.7-0.8 intermediate field, which are more affected by aberrations, can project to an NP point at a location that is more or less offset from the nominal NP point 190 than the offset NP point 192B at the field edge. To accommodate differences in lens design, the non-paraxial offset "NP" point can be dropped either before (closer to the lens) the paraxial NP point (entrance pupil), as proposed in Figure 5B, or after (as shown in Figure 2A).

[0040] This is shown in more detail in Figure 5C, which essentially illustrates a more zoomed-in region AA of Figure 5B, illustrating the effects of vector projected ray paths associated with errant imaging rays converging at or near the paraxial entrance pupil (190) for an imaging lens system designed and optimized using the methods of the present approach. In Figure 5C, the projected ray paths of green errant imaging rays at multiple fields of view from the camera lens system converge within a low parallax volume 188 near one or more "NP" points. Similar illustrations of ray fans can be generated for red or blue light. Projections of paraxial rays 173 can converge at or near the nominal paraxial NP point 190, or an entrance pupil located on the nominal optical axis 185 a distance Z behind the image plane 150. Projections of field edge rays 172, including chief ray 171, converge at or near an offset NP point 192B along the optical axis 185. The NP point 192B can be quantitatively defined, for example, as the center of the plethora of full-field edge rays 172. An alternative offset NP point 192A can be identified, corresponding to the "circle of least confusion," where the paraxial, edge, and intermediate or mid-field rays converge to a minimum spot. These different "NP" points are separated from the paraxial NP point by offset distances 194A and 194B, and from each other by offset distance 194C. Thus, it can be appreciated that the convergent "NP point" for any given practical imaging lens assembly or camera lens supporting a larger paraxial FOV, or an asymmetric FOV, will typically not be a point, but instead may be an offset low-parallax (LP) smudge or quantity 188.

[0041] Within the smudge or low-parallax volume 188, various possible optimal or preferred NP points can be identified. For example, offset NP points corresponding to field edge rays 172 can be emphasized to help provide improved image tiling. Alternative intermediate-field (e.g., 0.6-0.8) NP points (not shown) can also be tracked and optimized. Additionally, the overall size and location of the “LP” smudge or volume 188, or preferred NP points therein (e.g., 192B), can be varied depending on the lens design optimization. Such parameters may also vary from lens to lens for a manufactured lens system of a given design due to manufacturing variations between lens assemblies. While FIG. 5C illustrates these alternative offset “NP points” 192A, B for non-paraxial rays as being located after the paraxial NP point 190 or further away from the lens and image plane, other lenses of this type optimized using the present approach can be provided where similar non-paraxial NP points 192A located at low-parallax volume 188 can occur between the image plane and the paraxial NP point.

[0042] FIG. 5C also shows the location relative to the center of the low-parallax multi-camera panoramic capture device, device center 196. Based on optical considerations, the improved panoramic multi-camera capture device 300 can preferably be optimized to nominally position device center 196 within low-parallax volume 188. The optimized location may include being located at or near either offset NP point 192A or 192B, or within an offset distance 194B between them, to prioritize parallax control for field edge chief rays. The actual location is then determined by parallax optimization, which can be determined by lens optimization for entrance pupil spherical aberration, direct chief ray constraints, distortion, or a combination thereof. For example, whether spherical aberration is optimized to be over-corrected or under-corrected and what weighting is used for the field operand in the merit function can affect the positioning of the non-paraxial "NP" point for peripheral or intermediate fields. "NP" point positioning may also be determined by managing manufacturing tolerances and residual variations in lens system manufacturing. The device center 196 can also be located proximate the low-parallax volume 188, but offset from the low-parallax volume 188 by a center offset distance 198. This approach also aids in tolerance control and can provide more space for cables, circuits, cooling hardware, and related structures near the device center 196. In such cases, adjacent cameras 120 may consequently have offset low-parallax volumes 188 at the "NP" point (FIG. 5D) instead of being co-located (FIGS. 5A, 5B). In this example, if the device center 196 were instead located at or proximate the paraxial entrance pupil, NP point 190, then one or more of the outer lens elements 137 of the cameras 120 would be smaller than normal and the desired full FOV would not be achievable.

[0043] Thus, the no-parallax (NP) point is a useful concept to address and can usefully inform panoramic image capture and system design, aiding in the design of low-parallax error lenses. While it is ideal, its limitations must also be understood. Given this description of NP point(s) and LP smudges, understanding ray behavior in this region is crucial in enabling improved low-parallax multi-camera panoramic capture devices to define appropriate parameters or operands to optimize and appropriate target levels of performance to strive for. In the latter case, for example, a low-parallax lens with a track length of 65-70 mm can be designed for an LP smudge as wide as 10 mm (e.g., offset distance 194A). However, alternative lens designs where this parameter is further improved may have a low-parallax volume 188 with a longitudinal LP smudge width or width along the optical axis (offset 194A) of a few millimeters or less.

[0044] The width and location of the low-parallax volume 188, as well as the vector directions of projection of the various chief rays and their NP point locations within the low-parallax volume, can be controlled during lens optimization by methods using operands associated with fans of chief rays 170 (e.g., FIGS. 2A and 2B). However, the LP smudge or LP volume 188 in FIG. 5C can also be understood as a visualization of the transverse component of the spherical aberration of the entrance pupil, and this parameter can be used in an alternative, but equivalent, design optimization method to using chief ray fans. Specifically, during lens optimization, using, for example, Code V, a lens designer can create a special user-defined function or operand for the transverse component of the spherical aberration of the entrance pupil (e.g., ray height), which can then be used in a variety of ways. For example, the operand value can be calculated as the residual sum of squares (RSS) of values ​​over the entire FOV or a local field of view, using either uniform or non-uniform weighting for the field of view operand. In the latter case of local field of view preference, a value can be calculated for a position at or near the entrance pupil, or somewhere within the low-parallax volume 188, depending on a preference for paraxial, intermediate, or peripheral vision. An equivalent operand could be the width of the circle of least confusion at a plane, such as the plane of offset NP point 192A or the plane of offset NP 192B, as shown in FIG. 5C. The optimization operand can also be calculated with weighting to reduce or limit parallax error across the field of view non-uniformly, with disproportionate weighting favoring peripheral or edge vision over intermediate vision. Alternatively, the optimization operand can be calculated using weighting to provide nominally low parallax error in a nominally uniform manner across the entire field of view (within or across the core FOV 205, as in FIG. 7). That type of optimization can be particularly useful for mapping-type applications.

[0045] Whether a low-parallax lens design and optimization method uses operands based on chief ray or entrance pupil spherical aberration, the resulting data can also be analyzed with respect to changes in imaging perspective. Specifically, parallax errors for field of view and color can also be analyzed using calculations of the center of perspective (COP), a parameter more directly related to visible image artifacts that are low-parallax quantities, and can be evaluated in terms of image pixel error or difference for imaging objects at two different distances from the camera system. Center of perspective error is essentially the change in chief ray trajectory given multiple object distances—such as an object at close range (3 ft) versus another object at “infinity.”

[0046] In drawing and architecture, perspective is the technique of depicting solid objects on a two-dimensional surface to give a correct impression of their height, width, depth, and location in relation to one another when viewed from a particular point. For example, for drawings using linear or point perspective, objects appear smaller as their distance from the observer increases. Such illustrated objects are also subject to foreshortening, meaning that an object's dimensions along the line of sight appear shorter than its dimensions across the line of sight. Perspective works by representing light passing from the scene through an imaginary rectangle (realized as the plane of the drawing) to the viewer's eye, as if the viewer were looking through a window and depicting what they see directly on the windowpane.

[0047] Perspective is related to both parallax and stereoscopic perception. In stereoscopic image capture or projection using a pair of adjacent optical systems, perspective is a visual cue, along with dual-view parallax, shadowing, and occlusion, that can provide the perception of depth. As previously mentioned, parallax is the visual perception that an object's position or orientation appears different when viewed from different positions. In the case of image capture by a pair of adjacent cameras with at least partially overlapping fields of view, parallax image differences are a cue to stereoscopic perception or an error for panoramic imaging assemblies.

[0048] To capture an image using an optical system, whether a camera or the human eye, the geometry and performance of the optical system affect the usefulness of the resulting image for low-parallax (panoramic) or high-parallax (stereo) perception. Specifically, for an ideal lens, all chief rays from object space point precisely toward the center of the entrance pupil, which coincides with the center of perspective (COP) or center of view for the resulting image. For such an ideal lens, there are no errors in perspective or parallax.

[0049] However, with respect to real lenses, there may be residual parallax error due to both physical and image quality constraints. As previously mentioned, with respect to real lenses, the projections of paraxial chief rays from the first lens element point toward a common point, the entrance pupil, whose location can be determined as an axial distance from the front surface of that first element. On the other hand, with respect to real lenses that capture a FOV large enough to include non-paraxial chief rays, the chief rays in object space may point toward a common location or a nearby amount, but typically toward a location offset from the center of the entrance pupil. While these chief rays do not inherently coincide at a single point, they can be directed through a small, low-parallax volume 188 (e.g., an LP "smudge") with appropriate lens optimization. The longitudinal or axial variation of the rays within the LP smudge can be determined from the position of the chief rays across the optical axis. Ray error can also be measured as the lateral width or axial position of the chief rays within the LP smudge.

[0050] The concept of parallax correction is illustrated in FIG. 5D with respect to the perspective center. First camera lens 120A collects light from object space 105, including light from two outer ray fans 179A and 179B, whose chief ray projections converge toward low-parallax volume 188A, and images it into at least one core FOV. These ray fans may correspond to groups 172 of field-of-view or field-edge rays near the field of view, as shown in FIG. 2B or FIG. 5C. As shown in FIG. 5C, vector projections of such rays from object space, generally directed toward image space, within LP volume 188, favor field-edge rays, so they can cross optical axis 185 beyond the image plane at or near alternative NP point 192B, which can be selected or preferred. However, as also shown in FIG. 5C, such field-edge rays 172 need not intersect optical axis 185 at exactly the same point. These differences, when transformed back into object space 105, translate into small differences in parallax or perspective for the imaging ray bundle or fan within or across the imaging FOV of the camera lens (e.g., core FOV 205, as in FIG. 7).

[0051] A second, adjacent camera lens 120B, shown in FIG. 5D , can provide similar performance, imaging a fan of chief rays 170 from within the core FOV 205 with vector projections of those chief rays converging within a corresponding low-parallax volume 188B, including ray fan 179C. LP volumes 188A and 188B can overlap, coincide, or be offset, depending on factors including the camera geometry and seams between adjacent cameras, lens system manufacturing tolerances and compensators, or whether device center 196 is offset from LP volume 188. The more these LP volumes 188 overlap or coincide, the more the perspective centers of the two lens systems overlap. Ray fan 179B of camera lens 120A and ray fan 179C of camera lens 120B are also nominally parallel to one another, e.g., there is no parallax error between them. However, even if the lens design allows for very little residual parallax error at the edges of the FOV, manufacturing variations between lens systems can magnify the difference.

[0052] Analytically, chief ray data from a real lens can also be expressed in terms of perspective error, including chromatic error, as a function of field of view. Perspective error can then be analyzed as the position error in an image between two objects positioned at different distances or orientations. Perspective error can depend on the choice of COP position, angle within the imaging FOV, and chromatic error. For example, it can be useful to prioritize the COP to minimize green perspective error. Perspective or parallax error can be reduced by optimizing the axial position (Δz) or width of a color within the LP quantity 188 relative to the perspective center for one or more field angles within the imaging FOV. The perspective center can also be graphed and analyzed as a family of curves for Z (axial) intercept position (distance in mm) versus field of view angle, for each color. Alternatively, to get a better idea of ​​what the captured image will look like, the COP can be graphed and analyzed as a family of curves for the camera system as parallax error in image pixels, for each color, versus the field of view.

[0053] During the design of a camera lens system, a goal may be to limit the parallax error to a few pixels or less for imaging within the core FOV 205 (FIG. 7). Alternatively, it may be preferable to limit the parallax error, for example, to the outer edges of the core FOV and to extended FOV regions (if provided), particularly in the peripheral field. If the residual parallax error for a camera is thus small enough, the parallax difference, seen as the perspective error between two adjacent cameras near their shared seam 160 or in seam-related regions of the extended FOV where the imaging overlaps, may similarly be limited to a few pixels or less (e.g., ≦3-4 pixels). Depending on the lens design, device design, and application, it may be possible and preferable to further reduce the parallax error for the lens system, as measured by perspective error, to ≦0.5 pixels for the entire core FOV, the peripheral field, or both. If these residual parallax errors for each of two adjacent cameras are small enough, the images can be acquired, cropped, and easily tiled while correcting or hiding image artifacts from any remaining seams 160 or blind areas 165.

[0054] Continuing with the design of the type of panoramic camera of FIG. 1, lens optimization methods and parameter selection can be important to enable an improved low-parallax multi-camera panoramic capture device (300) with multiple adjacent cameras. A camera lens 120, or system of lens elements 135, such as that of FIG. 2A, can be used as a starting point. The camera lens has compressor lens element(s) and inner lens element 140, which can also be defined as consisting of a wide-angle lens group in front of the aperture and an eyepiece-like lens group behind the aperture. In designing such a lens to reduce parallax error, it can be useful to consider how the paraxial-to-nonparaxial chief ray fan 125 (see FIG. 2A), or edge chief ray fan 170 (see FIG. 2B), or localized collection of field edge rays 172 (see FIG. 5C) or 179A, B (see FIG. 5D) are imaged by the camera lens assembly. While it is possible to optimize a lens design by using a set of merit function operands for a collection or set of chief rays (e.g., 31 defined rays), the optimization process can become cumbersome. Alternatively, in pursuit of the design of an improved low-parallax multi-camera panoramic capture device (300), it has been determined that improved performance can also be obtained by using a reduced set of ray parameters or operands that emphasize the lateral component of spherical aberration at the entrance pupil or at a similar selected surface or location (e.g., offset NP point 192A or 192B) within the LP smudge volume 188 behind the lens system. Optimization for the lateral component of spherical aberration at an alternative non-paraxial entrance pupil can be achieved using merit function weighting that emphasizes the non-paraxial chief rays.

[0055] Alternatively, in a low-parallax multi-camera panoramic capture device, the fan 170 of chief rays incident on or near the beveled edge of the outer lens element of a camera 120 (see FIG. 2B ) should be parallel to the fan 170 of chief rays incident on or near the beveled edge 132 of the outer lens element of an adjacent camera (see FIG. 1 ). Note that the “edge” of the outer lens element 137 or compressor lens is a three-dimensional structure (see FIG. 2B ), which may have a flat edge across the thickness of the glass and is subject to manufacturing tolerances of that lens element, the entire lens assembly, and the housing 130, as well as adjacent seams 160 and their structure. The location definition of where the beveled edge is cut into the outer lens element depends on factors including material properties, front color, distortion, parallax correction, tolerances, and the extent of any extra extended FOV 215. Outer lens element 137 becomes a faceted outer lens element when beveled edges 132 are cut into the lens to create a set of polygonal edges that nominally follow a polygonal pattern (eg, pentagonal or hexagonal).

[0056] A camera system 120 having an outer lens element with a polygonal shape that captures incident light from a polygonal field of view will therefore form a polygonal image at image plane 150, with the shape of the captured polygonal field nominally matching the shape of the polygonal outer lens element. Cutting these beveled edges for a given pair of adjacent cameras can affect both the imaging and optomechanical structure at or near the intervening seam 160.

[0057] Alternatively, Figure 5E illustrates "front color," which is the difference in nominal ray path through the field of view when directed toward an off-axis or edge field point. Typically, for a given field point, blue rays are offset the farthest. As shown in Figure 5E, blue ray 157, accepted on first lens element 137, is approximately 1 mm further outward (ΔX) than accepted red ray 158, also directed toward the same field point. If lens element 137 is not large enough, this blue light may be clipped or blurred, resulting in color shading artifacts at or near the edge of the field. Front color may appear in the captured image content as a rainbow-like outline of the polygonal FOV or polygon edges of an outer compressor lens element (e.g., Figure 8), which acts as a field stop for the optical system. Local color transmission differences that can cause front-color-related color mottle artifacts near the image edges can arise from differential vignetting at the beveled edges of the outer compressor lens elements 137, or from edge truncation in the compressor lens elements, or through the aperture stop 145. During lens design optimization to provide an improved camera lens (320), front color can be reduced as part of the chromatic correction of the lens design, including through glass selection within the compressor lens group or the entire lens design, or as a trade-off in lateral color correction (e.g., to Δ(BR)X≦0.5 mm width). The effect of front color on the captured image can also be optomechanically reduced by designing the improved camera lens (320) to have an extended FOV 215 ( FIG. 7 ) and by designing the optomechanics to push the straight-cut or beveled lens edge 132 at or beyond the edge of the extended FOV 215, so that any residual front color occurs outside the core FOV 220. Front color artifacts can then be removed during an image cropping step during image processing. The effects of front or lateral color can also be reduced by spatially varying color correction during image processing.Alternatively, the improved camera lens (320) can have a color-dependent aperture at or near the aperture stop that can provide a larger transmission opening (diameter) for blue light than for red or green light, for example.

[0058] Figure 5F shows the variation of the perspective center 280 as the error or difference in image pixels versus field angle and color (R, G, B) for a low-parallax lens of the type shown in Figures 2A and 2B, but with improved optical design and performance. In this example, imaging of two objects was analyzed: one at a distance of 3 feet from the improved low-parallax multi-camera panoramic capture device (300) with the improved low-parallax camera lens 320, and the other at a distance of "infinity" (∞) from the device. Figure 5F shows a parallax error of <1 pixel for all colors from on-axis to approximately the edge of the field of view (e.g., up to about 34 degrees). Parallax error can also be quantified in degrees (e.g., percentage of angle per color). The R, G, and B curves of the perspective center difference 280 have similar shapes due to parallax optimization, but there are slight offsets and slope differences between them. These differences are a representation of residual chromatic aberrations in the lens, including lateral color, axial color, and front color. Parallax errors for blue light exceed 1.5 pixels at extreme field points (e.g., vertices). However, the human visual system and most visible imaging systems, including cameras using Bayer-type color filter arrays, are less sensitive to resolution-type errors when imaging with blue light compared to imaging with red and green light. In general, providing a parallax error of ≦2 pixels from a camera within its core FOV 205 and particularly within its peripheral field, and preferably even within a modestly sized extended FOV 215, can limit residual image artifacts to an acceptable, difficult-to-detect level. It is preferable to further limit perspective or parallax errors to a sub-pixel level (e.g., ≦0.5 pixels) for imaging within these FOVs, and particularly within the peripheral field, at least for green light. If the residual parallax error between adjacent cameras is sufficiently small, captured images obtained from the core FOV can be easily and quickly cropped and tiled together. Similarly, if the residual parallax error within the extended FOV capturing content within or near the seam is similarly small enough and the two adjacent cameras are properly aligned with each other, the overlapping image content captured by the two cameras can be quickly cropped or averaged and included in the output panoramic image.

[0059] Optical performance at or near a seam can also be understood, in part, in relation to distortion ( FIG. 6 ) and a defined set of fields of view ( FIG. 7 ). Specifically, FIG. 7 illustrates a potential set of fields of view for which potential image light can be collected by two adjacent cameras. As an example, a camera with a pentagonal outer lens element, with a seam 160 separating it from the adjacent lens or camera channel, whether associated with a dodecahedron, truncated icosahedron, or other polygonal lens camera assembly, can capture an ideal FOV 200 extending to the apex (60) or polygon edge of the frustum or conical volume within which the lens resides. However, due to various physical constraints that may arise at the seam, including the finite thickness of the lens housing, the physical aspects of the beveled lens element edges, mechanical wedges, and tolerances, a smaller core FOV 205 of transmitted image light can actually be captured. The coated opening for the outer lens element 137 should encompass at least the core FOV 205 with some margin (e.g., 0.5–1.0 mm). Since lenses can be manufactured with AR coatings before beveling, the coating can extend to the seams. The core FOV 205 can be defined as the largest low-parallax field of view that a given practical camera 120 can image. Equivalently, the core FOV 205 can be defined as a sub-FOV of a camera channel whose boundaries are nominally parallel to the boundaries of its polygonal cone (see FIGS. 5A and 5B). Ideally, with small seams 160 and proper control and calibration of FOV pointing, the nominal core FOV 205 approaches or matches the ideal FOV 200 in size.

[0060] During the camera alignment and calibration process, a series of image fiducials 210 can be established along one or more edges of the core FOV 205 to assist in image processing and image tiling or mosaicing. The resulting gap between the core FOV 205 supported by a first camera and that supported by an adjacent camera can result in a blind area 165 (FIGS. 5A and 5B). To compensate for the blind area 165 and the associated loss of image content from the scene, a camera can be designed to support an extended FOV 215, which provides sufficient extra FOV to accommodate the seam width and tolerance, or offset device center 196. As shown in FIG. 7, the extended FOV 215 can be extended sufficiently to provide overlap 127 with the edge of the adjacent camera's core FOV 205, although the extended FOV 215 can be even larger. This limited image overlap may result in a small amount of image resolution loss, parallax error, and some complexity in image processing, as discussed above with respect to FIG. 3, but it may also help reduce the apparent width of the seam and blind areas. However, as provided by the present approach, if the extra overlap FOV is modest (e.g., ≦5%) and the residual parallax error therein is sufficiently small (e.g., perspective error of ≦0.75 pixels), the image processing burden may be very minimal. Image capture up to the extended FOV 215 can also be used to enable intermediate capture steps that support camera calibration and image correction during operation of the improved panoramic multi-camera capture device 300. FIG. 7 shows an inscribed circle within one of the FOV sets, corresponding to a subset of the core FOV 205, that can be captured in all directions from that camera: the common core FOV 220. The angular width of the common core FOV 220 may be useful as a quick reference to the camera's image capacity. An alternative definition of a larger common core FOV 220 to include the entire core FOV 205 may also be useful.The dashed line (225) extending from the common core FOV 220 or core FOV 205 beyond the ideal FOV 200 to nominally include the extended FOV 215 represents the region where the lens design can support careful mapping of the chief or primary rays or control of the spherical aberration of the entrance pupil to enable low parallax error imaging and easy tiling of images captured by adjacent cameras.

[0061] To reduce parallax and improve image tiling across a seam 160 spanning the distance between two adjacent usable apertures between two adjacent cameras, it may be advantageous if image light is captured substantially straight, parallel, and commonly spaced for a finite distance. The amount of FOV overlap required to provide an extended FOV and limit blind areas can be determined by controlling the relative proximity of the entrance pupil (paraxial NP point) or an alternate preferred plane (e.g., to emphasize peripheral light) within the low-parallax amount 188 relative to the device center 196 (e.g., the center of a dodecahedron shape). The amount of extended FOV 215 is preferably 5% or less (e.g., 1.8° or more additional field of view relative to a nominal core FOV of 37.5°), thereby resulting in a camera peripheral field of view of, for example, approximately 0.85-1.05°. If spacing constraints at the device center and manufacturing tolerances are well managed, the extended FOV 215 can be reduced to 1% or less additional field of view. Within the extended FOV 215, parallax should be limited to nominal system levels while both image resolution and relative illumination remain satisfactory. Parallax optimization to reduce parallax error can use either chief ray or pupil aberration constraints, targeting optimization for the high FOV region (e.g., 0.85-1.0 FOV) or beyond to include the extra camera overlap region provided by the extended FOV 215 (e.g., Figure 7, partial FOV range of approximately 0.85-1.05).

[0062] Additionally, in enabling an improved low-parallax multi-camera panoramic capture device (300) with limited parallax error and improved image tiling, it may be important to control image distortion for image light passing at or near the edges of the outer lens element's FOV, e.g., peripheral vision. In geometric optics, distortion is the deviation from the desired state in which straight lines in a scene remain straight in the image (e.g., rectilinear projection). It is a form of optical aberration and describes how light rays from a scene are mapped to an image plane. Generally, for image capture, lens assemblies used for human viewing are advantageous in limiting image distortion to a maximum of + / - 2%. In current applications, it may also be useful to have a moderate distortion of 2% or less for tiling or combining panoramic images from images captured by adjacent cameras. For reference, in cylindrical distortion, image magnification decreases with distance from the optical axis, and the apparent effect is that of an image being mapped around a spherical (or cylindrical) surface. Fisheye lenses, often used to capture hemispherical or panoramic views, typically have this type of distortion as a way of mapping an infinitely wide object plane into a finite image area. Fisheye lens distortion (251) can deviate significantly from f-theta distortion (e.g., 15% or 90° half-width (HW) for the full field of view), but only a few percent for small fields of view (e.g., ≦30° HW). As another example, in laser printing or scanning systems, f-theta imaging lenses are often used to print images with minimal banding artifacts and image processing corrections for pixel placement. Specifically, f-theta lenses are designed with a cylindrical distortion that results in a nearly constant spot or pixel size and pixel positioning that is linear with the field of view θ (h=f*θ).

[0063] Thus, an improved low-parallax camera 320 capturing a half FOV of 35-40° or less may have fisheye distortion 251 because the distortion may be sufficiently low. However, the distortion can be more advantageously optimized for the design of an improved camera lens assembly for use in the improved low-parallax multi-camera panoramic capture device (300). As a first example, it may be advantageous to provide a camera lens assembly with localized nominal f-theta distortion 250A at or near the edge of the image field, as shown in FIG. 6. In one example, image distortion 250 peaks at about 1% at about 0.75 FOV, and the lens design is not optimized to provide f-theta distortion 250A below about 0.85 FOV. However, during the lens design process, the merit function can be constrained to provide nominal f-theta-like distortion 250A or substantially flat distortion 250B for rays imaged at or near the field edge, such as for peripheral fields spanning a partial field range of about 0.9-1.0. This range of high FOV with f-theta type or flat distortion correction includes the fan of chief rays 170 or peripheral rays of FIG. 2B, including rays imaged through corners or vertices 60, such as those of a lens assembly with a hexagonal or pentagonal outer lens element 137. Additionally, due to manufacturing tolerances and dynamic effects (e.g., temperature changes) applicable to the camera 120, including both the lens element 135 and the housing 130, and to the collection of cameras 120 within a panoramic multi-camera capture device, it may be advantageous to extend the region of nominal f-theta or flattened distortion within the peripheral field beyond the nominal full FOV (e.g., 0.85 to 1.05). This is illustrated in FIG. 6, where the region of reduced or flattened distortion extends beyond the full FOV to approximately 1.05 FOV. In such a peripheral FOV range, it may be advantageous to limit the total distortion change to 0.5% or less. Peripheral field distortion is controlled to keep image "edges" straight within the adjacent pentagonal region. This may allow for more efficient use of pixels when tiling an image, and therefore faster image processing.

[0064] The preceding discussion discusses distortion in the classical sense, as image aberration at the image plane. However, in low-parallax cameras, this residual distortion is typically a trade-off or nominal cancellation of the contribution from the compressor lens element (137) for that of the collective inner lens element (140). Importantly, the ray redirection caused by the distortion contribution of the outer compressor lens element also affects both the imaging ray path and the projection chief ray path toward the low-parallax volume. This consequently means that for at least some low-parallax lens designs, distortion optimization can affect parallax or field edge NP point or perspective center optimization.

[0065] The definition of the peripheral field or subfield of view range (e.g., approximately 0.85-1.05, or including 5% or less of additional field of view), where parallax, distortion, relative illumination, resolution, and other performance factors are carefully optimized to support image tiling, may depend on the device and camera geometry. As an example, for a hexagonal lens and field of view, the lower limit of the peripheral field of view can be defined as approximately 0.83, and for a pentagonal lens, as approximately 0.8. While Figure 7 illustrates the case with two adjacent pentagonal outer lens elements and FOV sets, the approach of defining peripheral fields and extended FOVs to support small regions of overlapping image capture can be applied to multi-camera capture device designs with adjacent pentagonal and hexagonal cameras, or adjacent pentagonal cameras, or cameras with adjacent edges of other polygonal shapes or any shape or contour in general.

[0066] For the extended FOV 215 to be functionally useful, the nominal image formed on the image sensor corresponding to the core FOV 205 must underfill the used image area of ​​the image sensor at least sufficiently to allow the extended FOV 215 to also be imaged. This may be done to help accommodate actual variations in the manufactured lens assemblies from ideal, or to accommodate manufacturing variations in designs with offset device centers 196 and improved low-parallax multi-camera panoramic capture devices (300). However, as will be explained later, judicious mechanical design of the lens assemblies can help to limit mechanical displacement or wedging, affecting both the image fields of a given camera and the seams between cameras, reducing parallax error and FOV overlap or underlap. Similarly, compensators or fiducial adjustments of image FOV (core FOV 205) size and position, as well as image centroid tracking and shape tracking, can be helpful. Some combination of optimization of distortion and low- or zero-parallax imaging for the extended peripheral field of view, careful mechanical design to limit and correct for component and assembly variations, and the use of correction standards or compensators can provide a superior overall system solution. As a result, images captured from the cameras can be easily cropped to the expected nominal size and shape relative to the nominal core FOV 205, and images from multiple cameras can then be mosaiced or tiled together to form a panoramic image with reduced image post-processing effort. However, the extended FOV 215 should provide sufficient additional angular width (e.g., θ1≦5% of FOV) to match or exceed any expected wedge or tilt angle θ2 that may occur at seams (θ1≧θ2), if necessary.

[0067] In designing an improved imaging lens of the type that can be used in a low-parallax panoramic multi-camera capture device (100 or 300), several primary parameters can be calculated to inform the design effort. The key parameter is the target size of the truncated cone or conical volume based on the selected polygonal configuration (lens size (FOV) and lens shape (e.g., pentagonal)) and sensor package size. Other key parameters that can be estimated include the nominal position of the paraxial entrance pupil, the focal lengths of the compressor lens group and the wide-angle lens group, and the FOV seen by the wide-angle group.

[0068] However, design optimization for an improved camera lens (320) for use in an improved low-parallax panoramic multi-camera capture device (300) also depends on how numerous other lens attributes and performance metrics are prioritized. Specifically, relevant system parameters may include control of parallax or center of perspective (COP) error at the edge or inner field locations of the image field, or both, as optimized using spherical aberration of the chief ray fan or entrance pupil. These parameters are closely related to other important parameters, including the width and location of the "LP smudge" or volume 188, the size of any center offset distance between the entrance pupil or LP smudge and the device center 196, the target width of the gap or seam, the extent of the blind region 165, and the size of any peripheral or extended FOV to provide overlap. Relevant performance metrics may include image resolution or MTF, distortion (especially in the peripheral field and of the first compressor lens element and compressor lens group), lateral color, relative illumination, front color and color vignetting, telecentricity, and ghosting. Other relevant design variables may include mechanical and material parameters such as the number of compressor lens elements, the configuration of the compressor lens group, the wide-angle lens group, and the fisheye lens group, glass selection, the maximum allowable size of the first compressor or outer lens element, sensor package size, track length, the nominal distance from the image plane to the nearest front lens element (e.g., working distance), the nominal distance from the image plane to the entrance pupil, the nominal distance from the image plane or entrance pupil to the polygon center or device center, manufacturing tolerances and limitations, and the use of compensators.

[0069] As a second illustrative example, FIG. 8 shows an alternative, improved camera lens 320 or objective lens with lens element 335, which is an enhanced version of lens 120 of FIG. 2A that can be used in an improved low-parallax multi-camera panoramic capture device (300). FIG. 8 shows the overall lens shape on the left, as well as a partial enlargement showing inner lens element 350 in more detail. This lens, also designed for a dodecahedron system, has lens element 335, which includes both a first lens element group or compressor lens group consisting of outer lens element 345a and compressor lens elements 345b and 345c, and inner lens element 350. In this design, compressor elements 345b, c are not fully bonded, either cemented or as an air-spaced doublet. Also shown in FIG. 8, inner lens element 350 is composed of a front wide-angle lens group 365 and a rear eyepiece-like lens group 367.

[0070] In FIG. 8 , the lens system of camera 320 collects light rays 310 from object space 305 and directs them through lens element 335, consisting of outer lens element 340 and inner lens element 350, to provide image light 315 from field of view 325, providing an image at image plane 360. This lens system provides improved image quality, telecentricity, and parallax control, although these improvements are not apparent in FIG. 8 . In this example, outer lens element 340 includes a group of three compressor lens elements 345 a, 345 b, and 345 c, and the light intensity, or light bending load, is distributed among the multiple outer lens elements. Image light 310 from object space 305 is refracted and transmitted through first lens element group or compressor lens group 340, which has three lens elements, so that the chief ray at 37.377 degrees at the vertex is redirected at a steep angle of approximately 80 degrees toward optical axis 385.

[0071] This compressor lens element group is followed by a second lens element group or wide-angle lens element group 365, which consists of two lens elements between the compressor lens element group and aperture stop 355. A third lens element group or eyepiece lens group 367, having five lens elements, redirects the passing image light coming from aperture stop 355 to provide the image light telecentrically at F / 2.8 to the image sensor at image plane 360. Because this lens is designed for a dodecahedron system, first lens element 345a nominally accepts image light for an FOV width of 31.717 degrees at the middle chord. Chief ray projections converge or point toward LP smudge 392, which contains the paraxial entrance pupil.

[0072] As illustrated by FIG. 8 , this type of camera lens, or lens format, with a first lens element group or compressor lens group or lens elements (345a, b, c), a second lens element group or wide-angle lens group 365 in front of the aperture, and a third lens element group or eyepiece-like lens group 367 behind the aperture, may visually resemble a fisheye lens in whole or in part, but it is quite different. Unlike this lens design (e.g., FIG. 8A ), a fisheye lens is an ultra-wide-angle lens with severely overcorrected spherical aberration of the pupil, such that its entrance pupil is located close to the first lens element and near the front surface of the lens. This pupil aberration also causes a substantial shift and rotation of the non-paraxial entrance pupil relative to the paraxial entrance pupil. Such lenses also offer a long back focal length, reversing telephoto, and a positive value for the ratio of the entrance pupil to the image plane (EPID) divided by the lens focal length (EPID / EFL). Fisheye lenses also offer strong visual distortion, typically following a monotonic curve (e.g., H = fθ (f-theta)) and capturing images with a characteristic convex, nonlinear appearance. While a typical fisheye lens nominally captures a wide-angle full FOV of 180°, fisheye lenses capturing images with even larger FOVs (270-310°) have also been described in the literature. In contrast, the improved low-parallax wide-angle camera lens 320 of the present approach, used within the improved low-parallax multi-camera panoramic capture device (300), is intentionally designed with low distortion, particularly at or near the edges of the imaging FOV, to facilitate image cropping and tiling. Furthermore, while the present cameras are wide-angle, they typically capture image light from a significantly smaller FOV than fisheye lenses. For example, cameras for dodecahedron devices nominally capture images from a full-width FOV of ≈63-75°. On the other hand, an octahedral device can have a camera that captures image light from a nominally full-width FOV of ≈71-110°, and a truncated icosahedral device can have a camera that captures image light from a nominally full-width FOV of ≈40-45°.

[0073] The wide-angle lens group 365 before the aperture and the eyepiece lens group 367 after the aperture are not used as standalone systems for this application if the compressor lens groups 345a, b, c are removed, although these two inner groups may also work together to form an image at or near the image plane or sensor. In the optical design of the camera lens (320), these lens groups, and particularly the wide-angle lens group 365, visually resemble a door peeper lens design. However, although this combination of two groups of lens elements may again appear to resemble a fisheye or door peeper-type lens, they again do not image with fisheye-type f-theta lens distortion (e.g., H=fθ).

[0074] In contrast, the optical structure of the rear lens group (367), or sub-system, resembles an eyepiece structure similar to that used as the eyepiece of a microscope or telescope, but used in reverse, without the presence of an eye. An eyepiece is an optical system whose entrance pupil is always located outside the system. The entrance pupil of the eyepiece, where the eye may be located in visual applications, nominally overlaps with the plane where the aperture stop 355 is located. Similarly, the nominal input image plane for visual applications corresponds to the sensor plane (950) in this application. The eyepiece group (367) is not designed to interface with the eye and therefore does not meet the requirements of an actual eyepiece for pupil distance, accommodation, FOV, and pupil size. However, this eyepiece-like lens group solves similar problems and therefore has a similar shape to that of an eyepiece. Depending on the application, the optical design can provide nominal optical performance more or less similar to that of a more typical eyepiece.

[0075] This improved lens 320 of FIG. 8 is similar to the camera lens 120 of FIGS. 2A and 2B, but has been designed for a more demanding set of conditions related to parallax correction, a larger image size (4.3 mm wide), and an entrance pupil that has been moved farther away to provide more room for the use of a larger sensor substrate. Because the glass type can be varied to favor the use of both crown and flint-type glass, this type of configuration with multiple compressor lens elements can be useful for color correction. In this example, the outer lens element 345a, or first compressor lens, is a meniscus-shaped lens element of SLAH52 glass with an outer surface 338 having a radius of curvature of approximately 55.8 mm and an inner surface having a radius of curvature of approximately 74.6 mm. Thus, the overall optimized and improved multi-camera capture device 500 may have a nominal radius from the apex of the outer lens element to the nominal NP point location of approximately 65 mm. In this example, incident light 310 from object space 305, which becomes image light 315, is significantly refracted inward when it encounters outer surface 338, but not as dramatically as provided by the first surface of the FIG. 2A lens.

[0076] The requirement to use a larger sensor substrate increases the distance between the image sensor plane and the entrance pupil or low-parallax volume 392. Specifically, the focal length is larger (5.64 mm) to project the image onto the large sensor. There are several potentially useful reference planes or locations within the LP smudge or low-parallax volume 392, including the paraxial entrance pupil, or the position of the perspective center, or the position relative to the non-paraxial chief ray NP point, or the position of the circle of least confusion that has a minimum size in the plane where the LP smudge or low-parallax volume is tangent to the optical axis. The entrance pupil is a good reference because it is easily calculated from general first-order optical equations. The axial location of the perspective center is also a good reference because it is directly related to perceived image quality. While the distance from the image plane 360 ​​to any of these locations can be used as a reference, the offset distance 375 relative to the paraxial entrance pupil may be preferred. In this example (FIG. 8), for a negative ratio of entrance pupil distance to focal length, EPID / EFL=−5.3:1, the entrance pupil is placed approximately 30 mm behind the image plane 360. Depending on how it is measured, the LP smudge 392 may have an axial width of ≦2 mm.

[0077] The improved camera lens system 320 of Figure 8 provides an example of how lens formats can differ from those shown in Figures 2A and 2B. Generally, lens formats for enabling an improved low-parallax multi-camera panoramic capture device (300) share a common set of features: an initial compressor lens group that sharply bends light toward the optical axis, a physically much smaller wide-angle lens group that redirects the light into the aperture stop, and an eyepiece-like lens group that focuses the passing image light toward the image plane. The requirement to reduce parallax or perspective error while allowing multiple polygonal cameras to be adjacent to form an even larger improved low-parallax multi-camera panoramic capture device (300) results in extreme lens formats, where the lens elements in the compressor lens group can be somewhat large (e.g., 80-120 mm in diameter), and more typically, at least some of the lens elements in the wide-angle and eyepiece lens groups are simultaneously somewhat small (e.g., 5-10 mm in diameter). In these types of lens designs, the first compressor lens element or outermost lens element 345a, and adjacent outer lens elements of adjacent lens systems, may alternatively be portions of a continuous faceted dome or shell. It is also typical for some (e.g., two to four) of the lens element surfaces to have aspheric or conic contours to bend or direct light rays passing near the edge of the lens element differently than light rays passing near the center or optical axis. Typically, the wide-angle lens group 365 also has lens elements with deep concave surfaces. In some cases, during optimization, the surface may want to be overly hemispherical to improve element manufacturability, but such contours are preferably avoided. Another measure of the ultimate performance of this lens type is the offset distance of the paraxial entrance pupil (or, equivalently, LP smudge) behind or beyond the image plane. Unlike a typical lens, the entrance pupil is not in front of the image plane, but instead may be pushed far behind or beyond it. This is emphasized by the negative ratio of entrance pupil to image plane distance / focal length, EPID / EFL, which can range from -2:1 to -10:1, but is typically ≥ -4:1.

[0078] As shown in exemplary detail in FIG. 8 , optimizing the size, location, and characteristics of the LP smudge or low-parallax volume 392 affects the performance and design of the improved camera lens system 320. Low-parallax volume optimization is heavily influenced by merit function parameters and chief ray weighting for both entrance pupil spherical aberration and entrance pupil axial or longitudinal chromatic aberration. Lens element and lens barrel manufacturing tolerances can also affect the size and positioning of this volume, or equivalently, the amount of residual parallax error provided by the lens. Thus, even though these lenses may be considered to have extreme shapes, optimization can help reduce sensitivity to manufacturing errors and provide insight into how and where to provide corrective adjustments or compensators.

[0079] In designing this type of objective or camera lens system for visible applications, it can be quite useful to use high-index, low-to-medium dispersion optical materials such as Ohara S-LAH53 or SLAL-18, especially for the compressor lens element. As an alternative, the optical ceramic Alon from Surmet Corporation of Burlington, Massachusetts, USA, has a refractive index comparable to these materials but even less dispersion, which can make it very useful in these lens designs. Using optical polymers or plastics in these lens designs can be particularly useful to reduce cost and weight, but also for other reasons. Compressor lens elements, and especially the first or outermost compressor lens element 345a, can be very large and subject to complex edge beveling, making them good candidates for glass-to-polymer replacement. High-index optical polymers, such as OKP4 from Osaka Gas Chemicals or EP5000 from Mitsubishi Gas Chemical, can be particularly useful for such purposes. Similarly, using an optical polymer for a deep concave lens element (such as Zeonex E48R) immediately prior to the aperture stop 355 can be beneficial compared to fabricating a surface with a limiting hemispherical or conical profile. Unfortunately, optical polymers have a much more limited range of optical properties than optical glasses, and high-index polymers have both a lower refractive index and higher dispersion than glass, which can constrain optical design or performance. It should also be understood that camera lenses of this approach can be designed with optical elements comprised of or including refractive, gradient index, glass or optical polymer, reflective, aspherical or freeform, kinoform, Fresnel, diffractive or holographic, subwavelength, or metasurface optical properties. These lens systems can also be designed with achromatic or apochromatic color correction or thermal defocus desensitization. These alternative material or optical element technologies can also be used for the optical elements for the relay imaging system described later.

[0080] Enhanced situational awareness can be directly enabled by an improved low-parallax multi-camera panoramic capture device (300) with low-parallax camera lenses 300, such as that shown in FIG. 8, through the use of appropriate lens design and optical detectors or sensors. For example, an optical event detection sensor, such as the Oculi SPU, can be positioned at the image plane 360 ​​and use its fast response and large dynamic range to detect sudden changes in objects within the scene. Neuromorphic or event sensor technology is still relatively early in its development, and currently, these sensors tend to have lower spatial resolution compared to CCD or CMOS image sensors. Therefore, as an alternative to providing situational awareness, a high-resolution, large-pixel-count image sensor, such as the Teledyne Emerald 67M, with 67 addressable megapixels, can be placed at the image plane 360 ​​of a properly designed lens 320. However, because this sensor is large and the camera channel 320 must fit within a conical volume or frustum, the front compressor lens elements (345a, b, c) can be very large and difficult to manufacture. These issues can be addressed by reducing the sensor size (such as for the Teledyne Emerald 16M or 36M), or by reducing the FOV imaged by the camera lens, or a combination thereof. For example, if the overall polygon shape is changed from a dodecahedron to a truncated icosahedron, the image field captured by the camera lens (32) is reduced, allowing a larger sensor to be supported, improving lens image quality and resulting in improved angular resolution. FIG. 11 shows a portion of an optomechanical system for an improved multi-camera capture device 300, as shown in FIG. 8, in which the camera 300 has a sensor 270, such as an image sensor or event sensor, provided with an associated internal image plane 360.

[0081] As another approach that can enable higher-resolution imaging or dual-modality sensing, as well as various situational awareness possibilities, an improved low-parallax multi-camera panoramic capture device (300) can include a low-parallax camera lens 320, functioning as an objective lens, paired with an imaging relay optical system. FIG. 9 shows such a system with an objective or camera lens 320 including a compressor lens group 340 paired with an imaging relay 400, where the relay is a lens system with a nominal magnification of 1.5x. These lenses are nominally aligned along an optical axis 385. FIGS. 15A and 15B show additional such examples of combined objective and imaging relay systems. In FIG. 9, the example camera lens 320 is similar to that of FIG. 8, except that the front compressor lens group 340 includes a cemented doublet. In this type of system, the original image plane 360 ​​corresponds to the actual aerial image, which is an intermediate image relative to a second image plane 410 at the far end of the imaging relay. A large, high-resolution image sensor, such as a Teledyne 67M, can then be provided at this second image plane 410. The optical system would be appropriately designed so that the optical resolution and the sensor resolution are nearly matched. The aperture stop 355 of the objective lens (320) is nominally reimaged in the relay optics to a secondary aperture stop 455. The optical relay design 400 also includes a gap or clearance 420 between the outer surface of the last field lens element 430 and the subsequent lens element. FIG. 14 shows a portion of an example optomechanical system for an improved multi-camera capture device 300 having a camera 320 paired with an imaging relay and sensor, in which an image sensor or event sensor is provided at the offset or secondary image plane 410. The system of FIG. 14 can include a nexus-type internal frame (e.g., FIG. 13) that provides a hollow center or open space through which multiple imaging beams of image light from multiple camera channels can intersect with each other. As will be explained later, the relay optics may also include beam splitting optics, mirrors, or other components to allow for multiple sensing modalities or other functions per camera channel.

[0082] The optics for the improved low-parallax multi-camera panoramic capture device (300), enabling enhanced imaging or situational awareness and using low-parallax camera lenses 320 directly (e.g., FIG. 8) or with accompanying relay optics (e.g., FIGS. 9 and 15A, B), must also be designed to accommodate reality, including optomechanics, sensors, electronics, and cooling or thermal control support. In this example, a dodecahedron-type device has eleven cameras 320 and an electromechanical interface at the twelfth camera position. Image data is collected from each of the eleven cameras and can be routed through an interface input-output module and through a cable or cable bundle to a portable computer capable of providing image processing, including live image cropping and stitching or tiling, and camera and device control. Output image data can be directed to an image display, a VR headset, or even a locally or remotely located computer. Power and cooling can also be provided as needed.

[0083] To help reduce thermal gradients between the sensors and their electronics and the optical system, microheat pipes or Peltier devices can be used to cool the sensors and redirect heat. Heat can be removed from the entire device by either active or passive cooling provided through the electromechanical interface at the 12th camera position, as shown in Figure 10. This cooling can be provided by convection or conduction (including liquid cooling), or a combination thereof. External ambient or environmental factors can also affect the performance of a multi-camera capture device. These factors can include the effects of illuminating ambient light or thermal extremes or changes in the environment. For example, because sunlight is typically highly directional, some outdoor image capture scenarios can result in cameras on one side of the device being brightly illuminated while other cameras are seeing plenoptic illumination from the scene or even being in shadow. In such cases, captured images may exhibit dramatic exposure changes, which can then be corrected by exposure compensation, which can be provided locally by various means. For example, exposure correction can be enabled by embedding photodetectors within the seams 160 or at the vertices, between the outer lens elements 137 (e.g., FIG. 1). These abrupt exposure differences can also cause spatial and temporal differences in the thermal load of some image sensors compared to others within the multi-camera capture device 300. Therefore, sensor cooling, whether enabled by heat pipes, heat sinks, liquid cooling, or other means, can be designed to absorb such differences. Performance can be verified by finite element analysis (FEA).

[0084] As part of addressing such issues, an improved multi-camera capture device 300, as shown in FIG. 11, may include features for providing kinematic mounting of the individual cameras 320 or objective lenses. Specifically, FIG. 11 shows two views of a dodecahedral multi-camera capture device 300, including a partial cross-sectional view, in which eleven pentagonal cameras 320 are mounted on a central support 525 that occupies the nominal position of a twelfth potential camera channel. Each camera 320 has a separate base lens assembly or housing 630, consisting of a lens mount that mounts a compressor lens (637) as well as an inner lens element 640 that together form the base lens assembly. The compressor lenses 637 can function as field stops for their respective camera channels, or a baffle (not shown) can be provided within the lens housing 630 adjacent to the associated compressor lens to function as a field stop. For each camera 320, the lens elements and housing 630 fit within a nominal conical space or volume, but they need not nominally fill that space. In practice, the sharp ray bending introduced by the compressor lens elements may mean that the inner lens elements 640 and their housings or barrels underfill the available space, and the entire lens housing 630 may taper further inward, potentially leaving an open interior volume 590 between adjacent lens assemblies. Adjacent lens housings 630 are separated by seams 600, which can be completely or partially filled with adhesive.

[0085] The housing 630, or base lens assembly, in FIG. 10 also includes a turned section that can be machined on a CNC multi-axis (5-axis) machine and engages with the tripod-like channel centering hub 530. The lens housing 630 can be manufactured from materials such as stainless steel or Invar. The channel centering hub 530 can be machined entirely on a lathe except for the pentagonal flange, which is completed in a finishing operation after lathe. Machining on a lathe means exceptional concentricity and runout can be achieved, aiding in ultimate channel alignment. The housing 630 engages with the inner diameter of the channel centering hub 530, a key part of the center-mounted mechanical assembly, which is designed to fit with it, ranging from a slip fit to a slight interference fit to ensure axial alignment without significant variation due to clearance tolerances. This same fit reduces perpendicularity error with respect to the channel axis.

[0086] The tripod or channel centering hub 530 also includes a swivel or ball pivot 540 that engages with a socket 545 of a spherical socket array 546 provided on the central support 525. In this system, the camera 320, located at the polar position opposite the central support 525, is the precisely located reference channel. The central support 525 consists of a cylindrical post with a ball mounted on it. The geometry of the central support 525 and tripod or centering hub 530 can be designed to provide more space for the sensor 270, power and communication cables, cooling tubes, and a mechanism for securing the lens housing 630 or cables. In this example, the ball includes sockets 545, each of which can accept a ball pivot 540. The ball pivot 540 is at the end of an extension pin or ball pivot arm 542. This ball and socket portion of the central support 525 can be expensive to machine, but given the expected precision regarding socket location and depth, the advantage is that centerline pointing is controlled, while there is only one part per device 300 that requires exceptional precision. On the other hand, each of the camera channels 320 can be machined with less precision, which reduces both manufacturing and replacement costs.

[0087] Each camera lens housing 630 in FIG. 11 features an external, or outer, inter-channel datum 535 located midway along a pentagonal side or face 537. These inter-channel datums 535 can include two parallel, convex, slightly curved, protruding rods separated by a groove between them. Both datums are designed to provide single-point or localized kinematic contact or interaction between the lens housings, whereby the datum features intertwine in such a way that only one part or housing dominates from a tolerance perspective. Because they intertwine, only one part variation affects the distance between each camera channel and, therefore, the angle between the channels. Specifically, if one datum 535 is large, the other will dominate because they do not touch. Thus, only one tolerance contributes to the two parts. The intertwining of the inter-channel datums 535 from one camera 320 to another allows limited angular movement of the lens housing 630 while also limiting lateral movement between the mating (pentagonal) faces or sides.

[0088] In this system, it may be useful to designate a camera channel as the primary channel and mount it precisely, but in a set manner that can serve as a datum to which other camera channels are directly or indirectly aligned. As an example, to take advantage of symmetry, the designated primary channel 610 can be one on the opposite side of the support post 525. Individually and collectively, the interaction between the camera lens housings 630 or base lens assemblies limits mechanical displacement and wedge or channel pointing error (roll, pitch, and yaw) between the cameras due to both the ball and socket arrangement and the datum feature (535). Each camera channel assembly works with its neighboring assemblies to limit channel pointing error. The outer lens element 637 or the portion of the base lens assembly (630) that holds the compressor lens also has an internal functional datum that allows the compressor lens to be positioned perpendicular to the optical or mechanical channel axis, and it has an additional internal datum feature that limits axial misalignment.

[0089] The use of alignment features shown in FIG. 11 , specifically the ball pivot and socket datums (550 and 556) and inter-channel datum feature 535, reduces the risk of rotation, pivoting, or splaying from one camera channel (320) to another. These features also therefore help allow the joints 600 between cameras 320 to have a more consistent thickness relative to the design values ​​than might otherwise occur. The use of internal features within the lens housings (e.g., compensators, adjustment screws, and shims) and external features between the lens housings (e.g., inter-channel datums, ball and socket datums, and channel loading supports) helps control the core FOV or extended FOV aiming so that one camera channel can align with another adjacent channel. The device (300) may also have channel loading supports (not shown) that can help deflect secondary or tertiary camera channels relative to the primary channel. The combined use of inter-channel datums, ball and socket datums (FIG. 11), channel loading supports, and adhesive within seam 600 can also help reduce the device's sensitivity to mechanical or thermal loads while controlling or limiting the occurrence of mechanical over- or under-constraint between adjacent camera channels (320).

[0090] The lens elements, including outer lens element 637, can be attached to housing 630 with a compliant adhesive. Along edge seam 600, the outermost edges of adjacent outer lens elements 637 flare out, and these lens elements can be nearly adjacent, separated by a gap that can be only 0.5 mm wide or less. In practice, optimizing the seam width can depend on how brittle or compliant the lens material is (glass or polymer), the flexibility of the adhesive filling the seam, the use of other protective measures, and the application.

[0091] FIG. 12A shows an alternative optomechanical design to that of FIG. 11 for a camera lens housing that generally fits within the confined volume of a pentagonal or hexagonal cone to position and support adjacent low-parallax camera channels. Specifically, FIG. 12A shows a portion of five adjacent imaging lens or camera channels 700, including an upper primary channel 710 and four secondary channels 715 separated by narrow seams 705. Each camera channel 700 includes a lens housing 730, a polygonal outer lens element 738, and a channel centering hub or tripod 740 that attaches to and interfaces with a central hub 750. A portion of at least one tertiary camera channel 720 is also shown. FIG. 12B then shows an exploded perspective view of a portion of the design of FIG. 12A, providing further detail of the interface of the secondary channels 715 to one primary channel 710.

[0092] 12A and 12B , unlike the above example that uses inter-channel datum 335, the inter-channel datum includes kinematic ball, flat, and V-shaped features. Specifically, primary channel 710 can include multiple balls 760 (or otherwise partial spherical or arcuate surfaces) protruding from a face or side 735 of lens housing 730. During assembly of primary camera channel 710, a pair of balls 760 can be aligned using a mounting system (not shown) so that they protrude from their respective side or face 735 of housing 730 by a specified amount, within tolerance. Lens housing 730 of secondary channel 715 can then be manufactured with corresponding V-shaped slots 762 and flats 765. During assembly of the multi-camera capture device 300, the primary channel 710 can be aligned to the central hub 750 using a pin (not shown) and attached using a tensioned cable or bolt to pull the channel centering hub or tripod 740 into contact with the central hub 750. The secondary channel 715 can then be similarly attached to the central hub 750 using a second spring-tensioned cable 755. As the assembly process occurs, across the seam 705, a first alignment ball 760 pops out to contact a corresponding secondary V-slot 762, and a second alignment ball 760 pops out to contact a corresponding one of the flats 765. The interaction of the first kinematic ball 760 with the kinematic V-slot 762 prevents movement in both directions, accurately and precisely positioning the two camera channels relative to each other. Similarly, a second precision ball 760 on the primary channel can interact with a corresponding flat 762 on the opposing secondary channel 715 to stop rotation in a third direction, providing stable and repeatable relative positioning of the two camera channels with respect to one another. By way of example, ball 760 can be a precision stainless steel ball with a nominal 0.188 inch diameter, and precision V-slot 762 can have a nominal 110 degree angle.

[0093] The optomechanical interface of the multi-camera capture device 300 shown in FIGS. 12A and 12B also includes magnets 770 on the camera lens housing 730. For example, one magnet 770 can be provided near each of the balls 760 on the side 735 of the primary channel 710. For example, two magnets 770 can be mounted on the primary channel side with their north poles facing outward. Each magnet can be set at the correct height and glued into a machined pocket while the magnets do not interfere with magnets in adjacent assemblies and are oriented so that the poles attract. The lens housing 730 is preferably made from stainless steel, such as Alloy 416, which has magnetic properties that help collapse inward-facing magnetic fields and strengthen outward-facing magnetic fields (e.g., toward the secondary channels).

[0094] On adjacent secondary channels 715, two magnets 770 can be provided with their south poles facing outward, adjacent to the V-shaped slots 762 and flats 765. When the two camera channels are brought close together, the effect of magnetic attraction between the north and south poles extends across the seam 705. This magnetic attraction can ensure that the two camera channels are attracted to each other, so that the V-shaped slots 762 and flats 765 contact their respective precision balls 760, thus providing kinematic alignment for the two channels and preventing separation or rotation within the inter-channel constraints. As an example, a rare earth permanent magnet, 3 / 16" diameter x 1 / 8" thick, with a pull force of 1-2 lbs, part number D32SH from K&J Magnetics, Pipersville, Pennsylvania, USA, can be used. For a suitable gap across the seam between magnets 770 of 0.75 mm, and for the magnets to be mounted within a lens housing 730 made from stainless steel, the attraction strength between the two magnets can be approximately 0.5 lbs.

[0095] The use of magnets 770 attached to the side 735 of the camera channel lens housing 730 can be configured in various ways. For example, pairs of magnets on the side 735 of the primary channel can be oriented with respect to their magnetic direction as an N-N pair, an S-S pair, or an N-S pair. In FIGS. 12A and 12B, some magnets 770 are marked with an "N" to indicate that their N poles point outward. The magnetic orientation can vary depending on the side 735 of the primary channel 710. In alternative configurations, a primary channel surface 735 may have only one magnet or none, while other primary channel surfaces 735 are equipped with a different number. Magnets can be positioned not only adjacent to the precision ball 760, V-shaped slot 762, or flat 765, but also in other locations on the side. The nominal magnetic strength or pulling force need not be the same for magnets on one side or from side to side of the camera channel (whether primary or otherwise).

[0096] 12A and 12B, magnets 770 are provided between the primary channel 710 and the secondary channel 715, and on all sides 735 of the secondary channel 715. The use of magnets between the secondary channel 715 and the tertiary channel 720 is similarly anticipated. However, the use of magnets 770 between the secondary channels 715 or the tertiary channels 720 may be optional by design. Similarly, the nominal strength of magnets 770 used between the secondary channels 715 and the tertiary channels 720, or between the secondary channels 715 or the tertiary channels 720, need not be the same as that used between the primary channel 710 and the secondary channel 715. In particular, those secondary magnets located on the primary channel 710 or in locations other than around the periphery of the primary channel 710 can be selected to have a lower magnetic strength.

[0097] In the nominal system shown in FIGS. 12A and 12B , the primary channel 710 can be aligned with a central hub 750 using pins and attached to the hub with bolts. The central hub 750 can be made of stainless steel (e.g., alloy 440). The secondary channel 715 can be attached by being pulled toward the hub 750 by a cable 755, while simultaneously being pulled toward the primary channel by a magnet 770, causing the V-shaped groove 762 and flats 765 to kinematically contact precision balls 760 on the sides 735 of the primary channel 710. The tertiary channel 720 can also have two balls, although in some instances, the balls 720 can be in different positions. For example, the balls on the tertiary channel can be positioned one on each side and the other in the corners. The tertiary channel 720 can be pulled toward the hub by a cable and toward the secondary channel by a magnet, simultaneously kinematically aligned by the precision balls with the “V” created by the intersection of two adjacent secondary channels. Tertiary channel rotation is prevented by contact with the flats on the secondary channel of other balls.

[0098] In other examples, the mounting plate, channel loading support, or channel nesting plate (not shown) can also include magnets 770, springs, flexures, vlier pins, or other devices to provide underlying support to the tertiary channels. While magnets 770 are shown in FIG. 12A as being used in the multi-camera capture device 300 with balls 760, V-shapes 762, and flats 765, magnets can also be used in the previous device of FIG. 11, where the lens housing has a low-profile inter-channel datum 335. Magnets can also be used on the lens housing for devices (300) with other optomechanical configurations, including the example of FIG. 13 with nexus internal frame 800. Moreover, while the examples of FIGS. 12A and 12B show the use of both magnets 770 and ball-based alignment features, in other cases, the magnets can be omitted. Furthermore, magnets 770 can be used in the absence of ball-based mounting features. Other combinations of inter-channel interface or datum selection or design can also be provided. Without limitation, a first interface between a primary channel and a first secondary channel can use ball-based and magnetic alignment features, while a second interface between a primary channel and a second secondary channel can use only ball-based features or only magnets. Similarly, as a design alternative, ball datum features can be provided on the secondary camera channel, while V-shaped and flat datum features are provided on the primary and tertiary camera channels. As another alternative, ball and V-shaped or flat datum features can be intermixed across or between camera channels, whether primary, secondary, or tertiary. For example, the sidewall of a primary channel can have ball and V-shaped datum features, while the sidewall of an adjacent secondary camera channel has corresponding flat and ball datum features. Furthermore, other alignment and / or positioning techniques described herein can be used in place of or in conjunction with the features illustrated in FIGS. 12A and 12B.

[0099] Improved low-parallax camera lenses 320, such as those in Figures 2A, 2B, and 8, can be designed to include a beam splitter and a second optical sensor therein. However, as exemplified by both the lens shapes shown in Figures 2A, 2B, and 8 and the exemplary optomechanical designs shown in Figures 11, 12A, and 12B, there is limited space for adding these components due to the need for a confining conical volume or frustum and other hardware. As one approach, if the improved low-parallax multi-camera panoramic capture device (300) provides more camera channels than a device having a regular dodecahedron shape, and instead has, for example, a truncated icosahedron or truncated rhombic triacontahedron (also known as a chamfered dodecahedron) shape, the FOV captured by any given camera channel (320) in this alternative device may be smaller. Easing the FOV per camera channel and, consequently, helping to alleviate the extreme shape and space constraints of both lens geometry (e.g., FIGS. 2A, 2B, and 8) and optomechanics (e.g., FIGS. 11 and 12A and 12B), thus enabling the use of a larger optical sensor at the image plane or easier inclusion of a beamsplitter and second optical sensor. Going all the way to a chamfered dodecahedron incorporates a mix of regular and irregular hexagons, compared to an icosahedron, which has only regularly shaped hexagons. Essentially, the polyhedron shape of the overall improved multi-camera panoramic image capture device 300 is selected to aid both the optical or lens geometry and the optomechanical design, compared to the selected polyhedron type (e.g., Goldberg polyhedron type). In optical terms, the optical invariants or Lagrangians of the individual camera channels are selected to optimize the optical design, sensor selection, and imaging performance of the objective lens (320), as in Figure 8, or the combination of the objective lens and relay optics (e.g., Figure 9 and Figures 15A and 15B). However, as the number of camera channels in a device increases, the number of seams and vertices also increases, which can result in increased overall mechanical complexity.This can complicate the use of optomechanical design approaches for devices with a "solid" central hub (e.g., Figure 11 and Figures 12A and 12B). In the case of systems without relay optics where at least some camera channels also provide secondary sensors (e.g., Figure 8 and Figures 12A and 12B), space is available to allow the outer ring camera channels to have optomechanics that protrude away from the device center outside their limiting polygonal conical volume.

[0100] As an alternative that can relax mechanical constraints, FIG. 13 provides an example of an alternative mechanical configuration with a nexus internal frame 800, which has multiple pentagonal faces 810 arranged in a dodecahedron pattern with a hollow center. Generally, the internal frame 800 is a polygonal frame with an array of adjacent polygonal mechanical faces, equipped with mounting and alignment features. The internal frame 800 can be designed as a mounting mechanical assembly for an eleven-camera system, with a support strut attached at the twelfth position (similar to FIGS. 11 and 12A-B). Polygonal internal frames, or half or partial internal frames, can also be used in partial or hemispherical systems, where a camera assembly, including an image sensor, is directly or indirectly attached to the frame. Connections, cables, and wiring for data transfer and cooling can then be directed out through the open polygonal portion 830 of one face 810, into the hollow center of the internal frame 800, and out through the open polygonal portion 830 of another face 810. Alternatively, a hemispherical system with an internal mounting frame 800 (see, e.g., FIG. 14 ) can provide a hollow center or open space (e.g., a nexus) to allow the image light beam to intersect through opposing pairs of open polygonal portions 830 of the faces 810 to pass through the subsequent relay optics (400) and reach a distant optical sensor at the secondary image plane 410. Positionally, the width of the gap or clearance 420 within the relay optics (see FIG. 9 ) between the outer surface of the last field lens element 430 and the nearest subsequent lens element 435 nominally matches the width of the central hollow volume between the opposing faces 810 provided by the nexus internal frame 800. For example, the clearance 420 may be 75 mm wide. However, it should be noted that the field lens elements 430 and their housings may protrude moderately through the open polygonal portions 830 of the faces 810 into the central hollow center volume, as long as they do not block the imaging light of the adjacent objective lens 320. In such a case, the clearance between the lens elements will be less than the width of the hollow center of the inner frame 800. For example, the width of the clearance 420 may be 10 mm less than the center width.

[0101] As shown in FIG. 13 , the nexus internal frame 800 may have a pentagonal face (810A) that may have three adjusters 820, such as set screws or flexures, oriented nominally 120° apart, that can be used to interact with mounting and alignment features on the camera housing and thus assist in aligning a given camera channel. For an improved multi-camera panoramic image capture device 300 constructed with a dodecahedron pattern, the internal frame may also be a dodecahedron with pentagonal faces, oriented with the internal pentagonal faces nominally aligned with the external pentagonal geometry. The internal frame approach can be used with other polygonal device structures, such as for an octahedron, icosahedron, or chamfered dodecahedron. In such cases, at least some of the polygonal faces 810 would have other polygonal shapes, such as a hexagon.

[0102] The internal frame 800 can be machined separately and assembled from two or more pieces, or it can be made as a single-piece structure by casting or 3D printing. While manufacturing a single-piece frame is more complex, the resulting structure is stiffer, stronger, and can support tighter mechanical tolerances. For example, a dodecahedral frame (800) with a hollow center can be cast from stainless steel and then selectively post-cast machined on faces 810 to provide precision datum features, including flats, V-shaped slots, or ball mounting features (e.g., similar to FIGS. 12A and 12B). Specifically, one or more pentagonal faces 810A, 810B, or 810C can include one or more adjusters 820 that can be used to gently press the respective camera channel against precision V-groove structures (not shown). These V-groove structures can be fabricated into or protrude from the inner edges of the pentagonal vertices 60 of the pentagonal faces. The alignment balls can be attached to surface 810 or to the mating adjacent lens housing, or a combination thereof. This internal frame can then be provided with flexures or adjusters on all or most of the pentagonal surfaces to provide kinematic type adjustments to reduce or avoid over-constraints during device assembly and use.

[0103] As previously mentioned, the attachments and adjustments for the secondary channels may have a different design or configuration than those for the primary channel. In these improved devices (300), springs, flexures, magnets, or adhesives may be used on or within the internal frame 800 to provide low-stress mechanical bonds or connections between the lens housings of adjacent camera channels, and between the camera channels and the Nexus internal frame 800, or even between different portions of the internal frame, thereby helping to limit under- or over-constraint between assemblies or lens housings. Alternatively, the internal frame may be made, at least in part, from a more compliant material, such as brass or Invar.

[0104] As mentioned above, a potential problem with the improved multi-camera capture device 300 is the multiple camera channels and their respective image sensors, which are confined to a nominally spherical interior, leaving little room for the inclusion of other components or functions. Therefore, in some applications, a device with a roughly hemispherical configuration, with potential room underneath for other hardware, may be beneficial. However, because the outer lens elements and cameras are typically polygonal, hemispherical devices may have jagged or irregular peripheries. Also, in such systems, one or more of the cameras can be designed with folds (e.g., using mirrors or prisms) so that the optical path extends through the irregular periphery surface underneath. This configuration can provide additional room for the use of modular sensor units that can be swapped in and out.

[0105] However, in a "hemispherical" version of a truncated icosahedron, with six camera channels having pentagonal faces and ten having hexagonal faces, as an example, it may be difficult to provide space for the optomechanics to fold so many optical paths. FIG. 14 shows an alternative version of the improved multi-camera capture device 300 in which image light collected by each camera objective lens system 920 is directed along a nominally straight optical path through a nominal image plane and then through image relay optics (925) to a more distant image sensor located within a sensor housing 930. The original image plane provided by the objective or camera lens system is essentially an actual intermediate image plane within the larger optical system. It can be reimaged at a magnification (e.g., 1:1 or 2.75:1) to a subsequent image plane (not shown) where the image sensor is located. Thus, advantageously, the image sensor may be larger and provide a higher pixel count, and the relay lens system 925 may refocus the image provided by the objective lens (920) at the appropriate magnification to nominally fill the more distant sensor with the projected image. Image light from each camera 920 traverses a central volume or nexus 910 as it passes through its respective relay lens system 925 (e.g., gap 420 in FIG. 9 ). The relay lens system 925 may include a field lens (e.g., field lens 430 in FIG. 9 ) behind the imaging plane of the camera lens, which may be mounted within the near side of the nexus internal frame. The light may then traverse the hollow central volume of the nexus internal frame to interact with subsequent lens elements. The improved multi-camera capture device 300 also includes a support structure 940, a support post 950, and a cable 960 for providing power and extracting signal (image) data. The support structure 940 can provide more substantial support for the sensor housing 930 than is illustrated in Figure 14, including through the addition of a space frame or latticework of support members that interconnect the relay system 925, the sensor housing 930, and other associated hardware. The system can also include mechanical design features to improve sturdiness or reduce vibration sensitivity.

[0106] The basic pairing of an objective lens, a low-parallax camera lens 320, with a refractive or lens-based imaging relay optics 400 is shown in FIG. 9. Simplistically, the imaging relay 400 images or magnifies an image plane 360 ​​onto a secondary image plane 410 with sufficient image quality relative thereto. However, this becomes more challenging the more extreme the lens shape is for the objective lens (320). As previously suggested, shifting from a regular dodecahedron shape to another polygonal shape, such as that of a truncated icosahedron or truncated rhombic triacontahedron, for the design of an improved low-parallax panoramic multi-camera capture device 300 can reduce the Lagrangian or optical invariants supported by the camera channel. In this case, the design of both the objective lens and the relay optics can be relaxed to improve performance or manufacturability.

[0107] In one embodiment, in a system intended to optimize imaging performance at the secondary image plane 410, to roughly balance the effects of aberration and diffraction, it may be advantageous to design the objective lens 320 to have an F-number of approximately F / 2.5 to F / 3, and the relay to have a magnification in the range of 1.5x to 2.5x, so that the speed or F-number of the image light 425 at the secondary image plane 410 is preferably in the range of F / 4 to F / 7. Higher speed performance (e.g., F / 4) will likely be required for systems or channels with higher resolution detection or smaller pixel size at the secondary image plane 410. In some systems, the preferred relay magnification may be approximately 1.5x to 2.5x, e.g., as little as 1x or as much as 5x to 10x.

[0108] Furthermore, when the improved low-parallax objective or camera lens example 320 of FIG. 8 was designed, the target height and width of the image plane 360, or the size of the conical volume or frustum of the entire camera channel, was determined in part by the size of the image sensor, rather than the size of the optically active or pixelated area. Specifically, additional space is required to clear the entire sensor package, including the frame and electrical and cooling interconnects or hardware. Because the difference between the active area and the overall sensor package size can be significant, the additional strain on the lens design to provide that greater clearance can be substantial. However, in a system with an objective lens and imaging relay, as in FIG. 9, the larger volume required for the sensor, sensor package, and other support hardware is shifted to a more accessible location away from the image plane 360. Therefore, the target for the size of the image plane 360 ​​can be further driven by optical considerations, such as Lagrangian, objective lens performance, or imaging performance onto the intermediate image plane (360) or subsequent image plane 410. Specifically, optimizing the size of the image plane 360 ​​for optical considerations instead of mechanical considerations can allow the low parallax amount 188 or NP point 190 to be shifted closer to the image plane 360, thereby helping to relax lens performance to provide target parallax performance.

[0109] Advantageously, the objective lens 320 and imaging relay 400 are coherently optimized or designed to sacrifice imaging performance or increase aberrations for the objective lens in order to share the burden with the imaging relay, thereby improving overall imaging performance at the secondary image plane 410. In contrast, the imaging performance provided by the first imaging lens is typically not corrected by the second imaging lens (e.g., relay) and is measured by MTF, the result being the product of the MTFs of the individual lens systems. However, current approaches to developing optically coherent designs in which a low-parallax objective lens is paired with an imaging relay correct aberrations to selectively track best. For example, referring again to FIG. 8, the front portion of the camera lens 320, including the compressor lens group 340 and wide-angle lens group 365, is specifically designed to provide both the range of the low-parallax volume 188 or the desired parallax performance by limiting the positioning of the offset NP point 192 (see FIG. 5C) therein. For example, as noted above, depending on the lens design, device design, and application, the residual parallax error for a lens system, as measured by perspective error, can be reduced to ≦0.5 pixels for the entire core FOV, within the peripheral field, or both. Similarly, the extent of LP smudge 188 can be optimized, again limited to ≦2 mm, by limiting spherical aberration at the entrance pupil or controlling the propagation of the outer chief ray fan, as an example. In designing an optically coherent system having a low-parallax objective lens 320 in combination with an imaging relay 400, the objective lens, and specifically the first two lens groups (compressor lens group 340 and wide-angle lens group 365), can be designed to provide target parallax performance, and then inadvertent degradation of parallax performance must be avoided during subsequent design of the overall system. Although the design of the third or eyepiece-like lens group 367 and imaging relay 400 does not affect the entrance pupil aberrations and therefore does not affect the actual chief ray propagating through the first two lens groups, they must change the downstream chief ray trajectories that reach the first and second image planes, respectively.The third lens element group attempts to balance all aberrations occurring in the first two groups to form an image at the first image plane. The relay can magnify the first image to further attempt to balance all aberrations at the secondary image plane. In a coherent optical design, aberrations at the first image plane can be sacrificed to benefit aberrations at the secondary image plane.

[0110] Moreover, while pursuing an optically coherent lens design for objective lens 320 with imaging relay 400 (e.g., FIG. 9 and FIGS. 15A and 15B), the design of eyepiece-like rear lens group 367 can be modified or relaxed to selectively sacrifice imaging performance for local image plane 360 ​​in order to relax the relay optical design, improve imaging performance for secondary image plane 410, or both. As an example of such a trade-off, relative illumination (RI), telecentricity, lateral color, distortion, image size, and aberration control or MTF requirements can be relaxed to various different degrees for the intermediate image (e.g., image plane 360), while these same attributes can be simultaneously optimized at secondary image plane 410. At the same time, similar trade-offs can be applied to other optical aberrations (e.g., spherical, coma, astigmatism, field curvature, and longitudinal or axial color) to help provide a well-resolvable image or MTF at secondary image plane 410. As an example, preferably the optical image resolution or MTF at the secondary image plane is nominally equal to the sensor pixel size, or a multiple thereof, when a Bayer color filter or equivalent is used.

[0111] While many aberrations, such as spherical aberration, are often defined by their third-order mathematical equations, more complex behavior often occurs in real lenses, which is sometimes accounted for by incorporating higher-order terms (e.g., fifth- and seventh-order) in models or design optimization. As shown in Figure 5E, front color is a simple BR color position difference. However, as shown in Figure 5F, color parallax exhibits more complex differences across the image field, which can be attributed to chromatic spherical aberration. Therefore, during coherent lens optimization, it can be more widely useful to both optimize or control the chromatic spherical aberration (spherochromatism) of the objective lens at the entrance pupil and to sacrifice lateral color at the first image plane to limit front color (Figure 5E), or chromatic parallax difference (Figure 5F), or color difference in the width or position of the low-parallax volume 188 (Figure 5C). As mentioned above, the sacrificed lateral color performance can then be corrected by a relay lens when designing for the final image plane. If properly controlled, the chromatic spherical aberration (spherochromatism) at the entrance pupil of the objective can be a few millimeters or less (eg, ≦2 mm).

[0112] However, when designing an optically coherent objective and imaging relay combination, special care may be required to limit front color (Figure 5E), as front color can result in residual visible color variations and chromatic perspective artifacts in the overlapping FOV near the seam during image tiling (see, e.g., Figure 5F). During lens design, reducing front color (e.g., to ≤0.2) can help bring the RGB curves closer together at or near the edges and apexes of the outer compressor lens elements, thus reducing subtle differences in color parallax correction. Reducing front color also reduces the size of the blue clear aperture (CA), which can aid in sizing both the extended FOV 215 and the width of the seam 600 and can also help manage lens element and lens housing manufacturing tolerances. The amount of front color is determined by both the design of the objective's first and second lens groups and the balance sought in controlling the lateral color contribution from the entire objective. Reduction in front color can be enabled during coherent lens design efforts for the objective lens and imaging relay combination by relaxing the lateral color target for the intermediate image (e.g., image plane 360), while simultaneously correcting for the lateral chromatic aberration increased during the design of imaging relay 400, so that the final lateral color for secondary image plane 410 is also sufficient. Note that in imaging systems, lateral color can often be reduced to a width of a few microns at the image plane (e.g., ≦10 mm for visible light). Alternatively, lateral color is typically reduced to ≦1.5 pixel widths, preferably ≦1 pixel width. For a system with a monochrome image sensor, such as the Teledyne 67M, the target width would be ≦2.5 microns (pixel width), while for the same system with a Bayer-filtered sensor, the target would be ≦5 microns. Optically coherent optimization of telecentricity can be similarly beneficial in relaxing performance for intermediate image plane 360 ​​while meeting more demanding specifications for secondary image plane 410 .

[0113] As yet another aspect to the design of an optically coherent objective and relay combination, the design of the first field lens element 430 in the relay system can advantageously be targeted to maintain both a reduced imaging beam size to fit within the apertures on both sides of the hollow center of the nexus inner frame 800. The use of the field lens element 430 can also improve image quality to the secondary image plane 410, helping to reduce the complexity of the overall relay optical design. Additionally, as part of an approach to optically coherent lens design, the lens elements of the third eyepiece-like lens element group 367 of the objective lens 320 and the field lens element 430 can be designed in combination to provide an improved image of the first aperture stop 355 to the secondary aperture stop 455. Because these are aperture stop planes rather than image planes, performance can be benchmarked in aberration waves. In most systems, the quality of aberrations at the aperture stop is much less of a concern than aberrations at the image plane. However, in some applications, such as those in which active wavefront modulators are used, such as for atmospheric turbulence correction, it may be beneficial to limit aberrations at or near an internal stop plane (e.g., secondary aperture stop 455). As an example, a useful target may be to limit wavefront aberrations near the aperture stop to a few waves or less (e.g., ≦8 waves).

[0114] FIG. 15A shows a variation of the optical system of FIG. 9 , further including a beam splitter 460 for splitting the incident light 425 into separate imaging paths 465, each with focusing optics 470 that serve to enable optical imaging to the respective image planes (410 and 415). The beam splitter 460 can be an optical plate (as shown) or a prism-type component, including an x-prism that can simultaneously split the light into three optical paths 465, thus enabling the simultaneous use of three different sensor types. The imaging relay system can also support more than one beam splitter within it. Depending on the system specifications and the sensor provided at the image plane, the beam splitter 460 can split the light based on proportional intensity, polarization, wavelength or spectrum (e.g., a dichroic prism), spatial or angular filtering, or a combination thereof. Similarly, depending on the differences between the optical sensor at secondary image plane 410 versus that at tertiary image plane 415, each focusing optic (470A or 470B) may have a different optical design. For example, if the optical sensor at tertiary image plane 415 has a different pixel size or resolution, or overall sensor size, than the sensor at secondary image plane 410, each focusing optic 470A or 470B may have a different design to help meet different performance goals, including matching the two image fields so that they are mutually aligned with the optical axis of the system and nominally the same size. Focusing optic 470 in optical path 465 may also provide zoom or focus correction functionality, so that optical magnification and focus adjustment can be dynamically changed. These different needs may arise for a variety of reasons. For example, both optical sensors may be CMOS image sensors, but one may support higher optical resolution than the other, or one may detect color signals (e.g., with a Bayer filter) and the other monochrome signals. Spatiotemporal dithering of the image sensor laterally within the imaging plane can also be used to increase the effective sensor resolution.As another example, objective lens 320 and imaging relay 400 can be designed to support expanded spectrum imaging, such as including both visible and infrared (IR) light, such that beam splitter 410 can separate the visible and IR light from each other and then pass them through different imaging paths 465A or 465B. Each image sensor in these optical paths 465A, 465B can then support a different dedicated spectrum with different performance specifications, such as sensitivity, temporal response, and resolution. For example, an IR sensor such as a microbolometer can be used.

[0115] The general configuration of FIG. 15A provides the advantage that each camera channel 320 in the improved low-parallax panoramic multi-camera capture device 300 can be arranged to coaxially detect image light for at least two different detection modalities. When this channel architecture is applied to multiple camera channels 320, the device 300 can be used to provide multi-modality detection over a wide angle to improve situational awareness of objects or events in the environment. For example, when the device is being used to detect moving objects, whether ground-based or airborne, it can be advantageous to have the camera channels capture both visible and IR light simultaneously. The IR light image can be used to detect and locate the heat signature of a vehicle or another object in the environment, while the visible light image can be used to assist in identifying the vehicle or object. To enable traffic monitoring, the improved low-parallax panoramic multi-camera capture device 300 can be mounted on a vehicle, in the environment, or both.

[0116] As a specific example, one of the optical sensors located at or near the imaging plane (410 or 415) can be an event sensor or neuromorphic sensor, including devices available from Oculi-ai (Fris Inc., Columbia, Maryland, USA) or Prophesee (Paris, France). These devices are much faster (e.g., 10,000 fps) and more sensitive (e.g., 120 dB) than standard CMOS or CCD image sensors, making them useful in detecting sudden events or fast-moving objects within a scene or environment. These devices can also be IR-sensing, which is useful for detecting moving objects such as vehicles, aircraft, drones, missiles, and other objects in the air or on the ground, or other concerns related to improved situational awareness. However, currently, these sensors have large pixels (e.g., 5-20 microns) and low resolution (e.g., ≦1 MP) when compared to standard image sensors. It may therefore be advantageous to provide an improved apparatus 300 with multiple camera channels equipped with imaging relays and beam splitters (e.g., FIG. 15A) to provide mutually aligned WFOV sensing of the environment, with dual imaging and event sensors. These sensors can operate simultaneously to correlate, compare, and contrast the image data they collect. The image sensors can operate at lower power or performance until the event sensors detect a trigger event and higher resolution imaging is required.

[0117] Optical configurations such as that of FIG. 15A may also include a second beam splitter (not shown) at, near, or before the secondary aperture stop 455 to direct image light to the optical sensor. Early interception of the passing image light in the relay optical path may make it easier to provide customized beam shaping or focusing optics (470) to the second image sensor rather than waiting later in the optical path (as shown). Alternatively, beam splitting near the secondary aperture stop 455 may enable the inclusion of an optical wavefront sensor. Similarly, the beam shaping or focusing optics (470) for the secondary optical sensor may be more complex than shown and may provide a further relayed image of the aperture stop 355 (e.g., a tertiary aperture stop (not shown)). As previously suggested, the relay or beam shaping optics for the second optical sensor may include an optical zoom. The optical zoom may include an axial step zoom (e.g., dual view) to shift the optics between at least two fixed zoom settings, a continuous axial optical zoom, a variable focus or focus compensation zoom, or a tumbler-type zoom that uses a mechanism to insert one or more alternative sets of optics into the relay or beamsplitting optical path 465 in place of the previous set. When zoom optics are used, it may also be desirable to be able to move the small zoom FOV back and forth within the larger FOV imaged by the camera objective 320. Beam steering optics, such as a pair of crossed galvanometers (e.g., galvo scanners) or a two-axis scanning mirror device (e.g., a 2D MEMs device, such as from Fraunhofer IPMS), can be included in the beam shaping optics for this purpose. As another variation, for applications involving imaging through significant distances (e.g., several miles) of the Earth's atmosphere, atmospheric turbulence may degrade image quality. To correct for this, the beam shaping optics for the sensor may further include adaptive optics, such as a wavefront modulator or an atmospheric turbulence correction device (eg, a deformable mirror device from Alpao, Montbonneau, France).

[0118] The relay optics can also be designed to collect image light from the intermediate image plane either telecentrically or non-telecentrically and similarly present image light to the distant image sensor either telecentrically or non-telecentrically. The relay can also be double-telecentric, meaning that it is simultaneously telecentric with respect to both the intermediate image plane and the distant sensor plane. Either the first aperture stop 355 or the secondary aperture stop 455 can be the limiting stop for the system. As an example, the first aperture stop can be the limiting stop, and the secondary aperture stop can be slightly oversized but still contribute to vignetting of stray light and improved detection contrast.

[0119] As yet another aspect, the design of the imaging relay optics, whether or not a beam splitter 460, optical zoom, or other devices are included, can complicate the optomechanical or mechanical design supporting the relay optics, beam shaping or zoom optics, sensor, and other hardware (see, e.g., FIG. 14 ). To aid optical performance, the optomechanics may include a zoom mechanism, an active focus shift mechanism or alternative focus correction design, or a sensor dithering mechanism, or other devices. Compared to the example of FIG. 14 , which has the relay system 925, sensor, and their housings 930 widened, it may be mechanically advantageous to have the relay imaging paths bent parallel to one another. The imaging relay optics design can be lengthened to facilitate the addition of mirrors or prisms to redirect the relay or beamsplitting optical paths. Alternatively, for mechanical or packaging reasons, it may be desirable to shorten the imaging relay optical path(s). The relay optics design can be optimized to enable this, including by using an inverse telephoto design approach.

[0120] FIG. 15B shows another example of an optical design in which the objective lens or camera 320 designed for the improved low-parallax multi-camera panoramic capture device 300 is optically coherently paired with a relay lens system 400 along the optical axis 385. In this example, the objective lens 320 is sized to result in a hexagonal camera channel with a 20.9-degree half-FOV at the midchord and a 23.8-degree half-FOV at the apex, as required for the improved low-parallax panoramic multi-camera capture device 300 based on a truncated icosahedron. This objective lens has an axial length of 83.5 mm with a front lens diameter of 87 mm, and it operates at f / 2.8 for a 10 mm wide-angle image at the intermediate image plane 360. Performance-wise, it provides 0.5 mm front color, + / - 3.5 microns lateral color (±1.5 pixels), <2% distortion, and MTF >60% at 140 cy / mm. Relay optics 400 operates at 2x magnification, allowing a Teledyne 67M sensor to be used at secondary image plane 410. This combination of objective and relay provides <2% distortion, RI>70%, total lateral color of approximately 2.5 microns or approximately ±0.5 pixels, and a polychromatic (relative to a Bayer-filtered sensor) MTF at 100 cy / mm of approximately 50%. Objective 320 provides 5 degrees of telecentricity at intermediate image plane 360, but the combination of the two lenses provides only 1.4 degrees at secondary image plane 410. Parallax for this example objective is reduced to ≦0.4 arc minutes in green across the image field.

[0121] 9 and 15A, the objective 320 of FIG. 15B, operating with a smaller FOV (e.g., 23.8° instead of 37.4°), provides better imaging performance for the larger intermediate image plane 360, which in turn better supports a larger sensor and higher resolution at the secondary image plane 410. Additionally, for the coherent optics optimization, both lateral color and telecentricity at the intermediate image plane 360 ​​are increased or sacrificed somewhat to benefit both front color at the outer lens elements and lateral color and telecentricity at the secondary image plane 410. This design trade-off can be further made to bring front color (e.g., ≦0.1 mm) or entrance pupil chromatic spherical aberration (e.g., ≦0.5 mm) under tighter control, while at the same time allowing for further image quality degradation for the intermediate image plane 360 ​​that is improved for the secondary image plane 410 by the coherent optics design of the objective and relay. Reducing the front collar helps to enable a reduction in the width of the gap or seam 600 between the imaging channels.

[0122] As previously mentioned, FIG. 4 shows a basic configuration of a LIDAR system 1000, in which a laser source 1010 emits laser light (λ). This light can be configured for optical scanning 1015 by scanning optics or laser source configurations, or a combination thereof. Whether scanned, swept, or flash illuminated, the laser light emerging from the first optics can be modified by additional illumination optics 1020 and then illuminate a portion of an environment 1070, which may have a set of objects (1071-1073) therein. The laser light (λ) can then be scattered, reflected, or diffracted from these objects, and the redirected laser light (λ) can then be collected into a second optical path, including optics 1025 and optical sensor 1030. The resulting signal can be examined by processing electronics 1040 to detect object size, depth, or location, and to determine and track the relative positions of such objects within a scene or environment, thus providing situational awareness of the environment. As an example, the HDL-64E1 is a 3D scanning LIDAR from Velodyne (Morgan Hill, California, USA) that uses 64 separate lasers arranged vertically to cover -24.8° to +2° in approximately 0.4° increments, scanning a 360° horizontal field of view (FOV) with 0.09° incremental postings as the laser array rotates. Alternative LIDAR technologies are being developed that are more compact and more easily integrated into other types of optical systems, and therefore may have the potential to provide enhanced situational awareness for a wider range of applications. These technologies include optical phased arrays (OPAs), flash VCSEL laser devices and arrays, and scanning systems using microelectromechanical systems (MEMS).

[0123] Compared to neuromorphic or event detection techniques that passively detect light coming from objects in the environment (e.g., ambient scattered light or radiated (thermal) light), LIDAR systems actively emit light into the environment to be detected and then detect the returning light. Many LIDAR devices detect the time-of-flight (TOF) of the returning light to determine the relative position of the object, while others use frequency-modulated continuous wave (FMCW) frequency modulation techniques to detect the beat frequency or Doppler shift between the emitted and returning light. The latter approach is more accurate but also more difficult to implement. On the other hand, in flash LIDAR, a scene can be illuminated with a single flash, and as the light bounces off from different directions, a two-dimensional array of tiny sensors detects it, allowing the time delay for the entire pixel matrix to be measured simultaneously. While flash LIDAR systems can operate without scanning, hybrid scanning-flash systems exist. Flash LIDAR may best enable sensing for automotive and other time-dynamic applications compared to scanning techniques that require the use of complex mechanical or optoelectronic (e.g., optical phased array (OPA)) devices operating at lower effective frame rates. However, regardless of whether one or more flash LIDAR sources are used, the instantaneous laser power requirements are higher, and eye laser safety concerns for pulsed laser exposure may be more challenging.

[0124] In scanning systems, beam steering is typically achieved with mechanical systems such as rotating mirrors, which can make the systems large, expensive, and unstable. Recently, microelectromechanical systems (MEMS) solid-state mirrors have been adopted to reduce size and cost, but there can be trade-offs between size, beam divergence (or resolution), and speed. Therefore, completely non-mechanical (solid-state) devices have been sought, and optical phased arrays (OPAs) fabricated using silicon (Si) photonics, complementary metal-oxide semiconductor (CMOS) processes, have been extensively developed for this purpose. However, many challenges remain for OPAs in large-scale integration of optical antennas, complex and power-consuming optical phase control, and the trade-off between steering range, resolution, and efficiency. Because OPA devices can have light emission and detection integrated on a single device, they can be used in systems with shared optics for both illumination and light collection.

[0125] As previously mentioned, a low-parallax camera or objective 320 of the type of Figure 2A or 8 can be used in the improved low-parallax panoramic multi-camera capture device (300) to provide enhanced situational awareness, including through the use of multiple sensor modalities, including LIDAR or laser ranging techniques. There are several possible configurations of laser ranging or LIDAR optics that can be combined to be coaxial with the imaging lens, including the camera objective for the improved low-parallax panoramic multi-camera capture device (300).

[0126] As a first example, FIG. 16A shows a relay optics section of the type shown in FIG. 15A, but it also includes a laser ranging subsystem with a MEMs mirror. In this example, light from a laser source 1100 (e.g., at 905 nm), which can be directly or indirectly modulated to provide light pulses for time-of-flight distance sensing, is directed onto a MEMs mirror device 1110. Candidate MEMs mirror devices can be single- or dual-axis scanning devices offered by vendors such as Preciseley Microtechnology Corp. (Edmonton, Alberta, Canada) or Fraunhofer IPMS (Dresden, Germany). Preferably, dual-axis MEMs mirrors are used to scan in both θx and θy, although a pair of offset single-axis devices can also be used. The MEMs mirror device can also have multiple micromirrors and can even be a linear or area device, including DLP or DMD-type devices from Texas Instruments. The ranging laser light can then be directed through beam-shaping optics 470 and into a common optical path shared with the image light heading toward image sensor 475A. This light can then pass through relay optics 400 and be directed out of the camera objective lens (not shown) into the environment, where it can illuminate objects within the FOV or scene. Portions of this illuminating laser light that are back-reflected, back-scattered, or diffracted from these objects can then be collected by the objective lens and then passed through relay optics 400 into a secondary optical path and incident on laser ranging or LIDAR sensor 458B. Depending on the system configuration, the sensor can be a single-photon avalanche diode (SPAD) array, or a detector or sensor array including a SPAD array, a Geiger-mode avalanche photodiode (GmAPD), or a multi-pixel photon counter (MPPC). Such devices are available from suppliers including Fraunhofer, Excelitas, or Hamamatsu. The sensor can also be an event or neuromorphic sensor from Oculi-ai or Prophesee.The resulting laser ranging data is then compiled into a point cloud to create an IR image and distance map of the locations of objects in the scene. This data can then be correlated or compared with the visible image data captured by the image sensor 475B, thus allowing objects to be easily located and identified. Because the LIDAR system and the visible optical image capture system can operate nominally coaxially simultaneously, the LIDAR and visible image data can be reliably aligned and calibrated alternately.

[0127] The MEMs mirror approach of Figure 16A has the advantage of providing two-axis scanning while using a single laser source operating within a narrow wavelength range (e.g., Δλ≦6 nm). The scanning resolution, limited by the laser pulse rate and the MEMs mirror scan rate and duty cycle, can be high (e.g., approaching low- to medium-resolution camera levels). However, the MEMs mirror device can sense external vibrations.

[0128] As a second example, we show part of an alternative system in which a laser ranging or LIDAR device is combined with relay optics and camera objectives to enable an improved low-parallax panoramic multi-camera capture device (e.g., device 300). In this case, light from a laser (e.g., 1450–1600 nm) is coupled into an optical phased array (OPA) 1120, which then provides a scanning laser beam. OPA-based LIDAR scanning technology and systems have been developed by companies including Quanergy (Sunnyvale, CA), Analog Photonics (Boston, MA), and Voyant Photonics (New York, NY). Typically, input laser light from a fiber-coupled source laser is fed into an input waveguide. This light is then split into a series of secondary waveguides, where it encounters a series of phase adjusters. The phase of the light passing through each individual waveguide is then changed by a modulating microheater, which locally modulates the refractive index within a few microns of the waveguide. These waveguides run parallel for a moderate distance before terminating in a low, rough, plasma-etched surface from which light is emitted.

[0129] The optical phased array 1120 includes multiple optical antenna elements that are fed with coherent signals of equal strength. A phased array is an array of transmitters that can change the direction of an electromagnetic beam by adjusting the relative phase of the signal from one transmitter to the next. Because of this variable phase control, when the transmitters all emit electromagnetic waves in sync, the beam generates a far-field radiation pattern that can be directed in a desired direction. For example, the beam can be sent straight ahead—that is, perpendicular to the array.

[0130] To steer a beam to the left, the transmitter tilts the phase of the signal sent by each antenna so that the signal from the left transmitter lags that of the right transmitter. To steer a beam to the right, the array does the opposite, shifting the phase of the leftmost elements to be ahead of those further to the right. Two-dimensional aiming can be achieved by varying the frequency (or wavelength) of the laser light and then passing it through a grating array, which—like a prism—directs the light in slightly different directions depending on its "color." For example, an IR laser can be tuned in use to emit laser light between approximately 1475 nm and approximately 1640 nm. Depending on the technology and company, beam steering has been demonstrated with beam divergences of only 0.02–0.08° within a 40–50° FOV.

[0131] The goal is to provide a clean output beam that illuminates the scene in the intended direction, without crosstalk (cross illumination). However, the emitted beam may have side lobes if the antennas (emitter + waveguide) are spaced apart by more than half a wavelength. The presence of residual side lobes depends on the waveguide spacing and tolerances, as well as the wavelength of the transmitted light.

[0132] As before, the scanning laser light is directed to the relay optics 400 and low-parallax camera objective (not shown), through which it may illuminate one or more objects in the environment. A portion of the reflected light from the object space may then travel the same path it traveled after emission—in reverse. As shown in FIG. 16B, the system may have a non-monostatic configuration, in which the reflected light is directed to a nearby detector or detector array 475B, such as a GmAPD, MPPC, or event sensor. Another possible system configuration includes two separate OPA switched tree array distribution networks, one for signal transmission (transmission) and one for reception. Yet another configuration has the reflected light directed back through the switched tree array to the 2×2 switch (redirector) to travel the reverse path to be collected by the same OPA device 1120 from which the light was emitted, and then back through the switched tree array to the 2×2 switch, where it is directed to a coherent detector for detection. This is called a monostatic or bidirectional configuration. In either case, once a signal is detected, a point cloud data set can be collected to determine the presence and location of objects within the surrounding environment, which can be used in conjunction with image data collected by sensor 457A.

[0133] Compared to the MEMs approach, the OPA approach can be completely solid-state, with no mechanical moving parts. It also uses FMCW or Doppler ranging, allowing for more accurate determination of object location and even the velocity and acceleration of moving objects. However, the OPA approach is more complex, requiring the use of fiber-coupled lasers and more complex drive and processing electronics. Because the OPA approach also captures range data at a low angular duty cycle, angular resolution can be poor due to diffraction.

[0134] As conceptually illustrated in Figures 15A and 16A and 16B, a MEMs or OPA LIDAR scanning system can be designed as conceptually shown in Figure 16C. The LIDAR laser 1100 can provide light through beam-shaping optics 470B and mirror 480 to focus the laser light near the objective lens aperture stop 355 in conjunction with the relay optics, so that a nominally parallel light beam can then emerge from the objective lens 320 with the beam waist positioned at, near, or somewhat beyond the outer surface of the objective lens 320's outer compressor lens element. As a result, the LIDAR subsystem scans the environment through the objective lens, so that a single pulse represents a single chief ray. The aperture stop 355 of the objective lens 320 or a mask in the relay optics (secondary aperture stop 455) can also be "color" dependent, using spatially distinct filters to provide a different aperture diameter for IR light than for visible light.

[0135] As a third example, Figure 16D shows part of an alternative system in which a flash laser ranging or LIDAR device is combined with relay optics 400 and camera objectives (not shown) to enable an improved low-parallax panoramic multi-camera capture device (300). Solid-state flash LIDAR is made possible by vertical cavity surface-emitting laser (VCSEL) technology (1100). For example, IR VCSEL devices are available from companies including Trilumina (Albuquerque, New Mexico, USA) and Finisar (Sunnyvale, California, USA). Two-dimensional VCSEL laser arrays can have 5,000 or more addressable laser emitters that provide distinct laser beams, where the returning light can then be collected on a detector such as a single-photon avalanche diode (SPAD) array (475B). However, while flash LIDAR is simple to develop, it can waste laser power by sending light to locations not seen by the detector. On the other hand, "multibeam" flash LIDAR can be selectively operated to provide laser light (e.g., 850 nm or 940 nm) in the direction the detector is looking. The laser light emitted by the VCSEL can be directionally controlled by various means to fill the scanning FOV, including using beam-shaping optics such as lenslets or micro-optic arrays, or by providing spatially distinct epitaxial growth on top of the laser cavity. Multibeam flash devices can also function as hybrid LIDAR cameras, capturing low- to medium-resolution infrared images of the environment. VCSEL lasers are also much cheaper than the 1550 nm fiber lasers used in many pulsed LIDAR (e.g., OPA) systems.

[0136] While LIDAR can rapidly provide position and velocity information for objects in an environment, it can be confounded by complex configurations or objects, including windows or mirrors. Confusion from light redirection from windows or mirrors is a recognized problem for LIDAR systems, whether the LIDAR is used for autonomous vehicle navigation, mapping or photogrammetry, or non-vehicle robotic navigation. Specifically, light reflection or refraction at these surfaces can confuse LIDAR-based detection systems by causing spurious noise (e.g., from light scattering at contaminated optical surfaces), providing an overly strong return signal, or refracting light in an expected direction. In the latter case, the mirror or window may be invisible to the LIDAR system and therefore be missing from the resulting point cloud. Light redirection can also cause other objects behind or near the mirror or window to be hidden or detected in the wrong position.

[0137] Published literature in the LIDAR field suggests that confusing reflections caused by windows and mirrors can be corrected by detecting the presence or location of these objects by detecting differences in their optical direction and intensity compared to reflections from other objects. Mirror or window reflections can be compared to reflections from diffusely or specularly reflective objects to detect "jump edges" or frames, or to look for reflection symmetry or asymmetry. These mirror and window detection methods rely on novel algorithms for processing the incoming sensed environmental data. There are also approaches to reducing the effects of window and mirror reflections that rely on double detection, in which laser ranging or LIDAR systems are used in combination with another detection technology (e.g., the use of cameras, sonar, or ultrasound). Generally, the use of double detection often solves the object confusion problem, whether for mirrors or windows or other complex objects, but it can be more expensive and time-consuming, does not necessarily resolve object ambiguity, or may not be acceptable for all position or depth sensing applications. Furthermore, the double detection approach imposes the additional burden of accurately overlapping, comparing, and prioritizing data from the two modalities.

[0138] In the aforementioned example, a channel of the improved low-parallax panoramic multi-camera capture device (300) includes beam-splitting optics (and possibly relay optics 400), an image sensor, and a laser ranging or LIDAR system and sensor, which nominally function through a partially common optical system (e.g., at least a portion of the camera objective 320). While the camera objective is typically designed to image visible light with low parallax or perspective error and high image quality, the ranging system generally uses IR laser light, which then emerges from the objective and requires appropriate beam control to illuminate an appropriate field of view. Nominally, the FOV scanned by the LIDAR system matches or is slightly larger than the FOV imaged by the camera objective. Similarly, to aid in angular resolution, the beam waist for the scanning laser beam is nominally positioned at or near the exit face of the outermost compressor lens element of the objective 320, if not several feet beyond, within the surrounding scanning environment. To aid in overall system performance, the camera objective lens 320 can be designed to image both visible and IR light. However, the beam-shaping optics 470 in the secondary optical path for IR depth sensing can be designed to correct or compensate for IR-specific chromatic aberrations of the objective lens 320. Depending on the type of LIDAR system, whether MEMs-, VCSEL-flash-, or OPA-based, the design configuration of the beam-splitting optics will differ with respect to FOV mapping, beam waist control, and chromatic correction of the IR light passing through the objective lens (and relay optics).

[0139] The present approach to an improved low-parallax multi-camera panoramic capture device 300 can coaxially support dual sensing modalities for viewing the surrounding environment, with or without relay optics. Indeed, designs that combine a low-parallax objective lens 320 paired with relay optics (e.g., FIG. 15 ) can more easily include various or multiple sensing modalities, combining LIDAR technology (e.g., FIGS. 16A-16D ) with standard visible or IR image sensors. This allows co-registered LIDAR-enabled depth point cloud data to be used in combination with low-parallax, real-time image data from multiple camera channels 320 viewing a wide FOV. The image sensor, whether a standard CMOS or CCD device, an IR-sensing array device, or a neuromorphic or event sensor device, or a combination thereof, can act as a triggering device to passively detect objects or events in the environment, which are further evaluated using the LIDAR-captured point cloud data. The LIDAR system can be in a standby state or low-power operating mode until a triggering event occurs. For directionally controllable LIDAR, such as some flash VCSEL systems, real-time directional control can be informed by image data collected by other sensors. Pairing with event or neuromorphic sensors can be particularly advantageous, as these devices are much faster and more photosensitive than standard image sensors. This approach to pairing coaxial dual or multi-sensing modalities with LIDAR can also help address the problems previously mentioned for LIDAR in dealing with confusing light reflections from windows and mirrors.

[0140] To improve the signal-to-noise ratio for depth sensing, the overall optical system must also be designed to suppress color crosstalk. This means that the visible optical path to the image sensor 410 may have additional filtering beyond the beamsplitter 460 and a regular IR-cut filter to block IR light. Similarly, the secondary IR optical path may have additional filtering beyond just the beamsplitter 460 to block visible light. These additional filters may be light-absorbing or dichroic filters. Note also that the AR coatings on the objective lens 320 and relay optics 400 must be designed to effectively transmit both visible light and IR depth-sensing light. In the latter case, this improves efficiency and helps prevent spurious back reflections from being interpreted as a signal. Pulse timing filtering may also be useful, since back reflections from within the optical system occur much faster than from objects in the environment.

[0141] As shown in FIG. 16E, the improved low-parallax panoramic multi-camera capture device (300) can also have bright-field-based depth-sensing optics. Bright-field detection technology has been developed by companies such as Lytro and Raytrix. Specifically, in this approach, bright-field detection optics can be provided by relay optics and beam splitters. The bright-field sensor is essentially a standard image sensor (e.g., 100 px / degree) amplified by bright-field optics (e.g., a microlenslet array or pinhole cavity) so that visible image light is directed to one or more sensing pixels within a subarray of sensing pixels. From the resulting pixel data, information regarding the local directional orientation of the visible image light can be determined, providing an indication of the depth or location of objects within the scene. As with the laser ranging or LIDAR example, the regular visible 2D image data collected by the image sensor and the bright-field data can be aligned and compared to improve bright-field image data processing and associate location data with specific objects.

[0142] In an example system of this approach, in which a low-parallax camera or objective lens 320 is combined with imaging relay optics 400, the relay optics is shown as a lens system consisting of multiple lens elements (see, for example, FIG. 15A ). However, the relay optics can also be designed as a reflective system, using multiple curved and flat mirrors with metal or dielectric coatings. The relay optics can also be catadioptric, consisting of a combination of refractive lenses and curved mirrors. Alternatively, the relay optics can consist of or include a coherent optical fiber bundle to transmit image light to one or more optical sensors. As a specific example, a device design with a shortened image relay can have several image light beams pass through a shared hollow space within the nexus internal frame 800, but then focus or image that light onto the input face of an optical fiber array. A coherent optical fiber array or bundle, in which the relative 2D arrangement of the optical fibers is maintained at the input and output faces, can then transmit the image light to a remote image sensor for detection. Alternatively, fiber optic bundles may be used in the systems of Figures 11 and 12A and 12B to convey image light to a remote sensor without the use of relay optics.

[0143] As another variation, the improved low-parallax panoramic multi-camera capture device 300 may have a combination of camera channels that include imaging relay optics (e.g., FIG. 15A ) and mechanisms (e.g., FIG. 13 ) and others that do not (e.g., FIG. 8 and FIGS. 12A and 12B ). For example, the device 300 may have a primary camera channel and secondary rings of camera channels, each with an imaging relay, single or multiple sensor modalities, and optics appropriately optimized to support imaging or focusing light onto different optical sensors. At the same time, the outer or tertiary ring(s) of camera channels 320 can image light directly onto optical sensors located at their internal imaging planes 360. As another variation, the camera channels on one side of the device (e.g., the left side) may include both an objective lens (320) and an imaging relay 400, while the camera channels on the other side of the device (e.g., the right side) have camera channels 320 that image light directly onto optical sensors located at their internal imaging planes 360. While we have focused on devices 300 with generally spherical (e.g., Figures 12A and 12B) or hemispherical (e.g., Figure 14) configurations, this approach can be extended to other device configurations, such as those with annular arrangements of camera channels. Additionally, note that this approach can be applied to single imaging channel systems with a single objective lens 320 paired with imaging relay optics 400 and one or more sensors at the secondary imaging plane. For example, an event sensor or LIDAR system could be along one optical path off the beam splitter, while a high-resolution image sensor is off the second optical path. In this case, the dual sensors could be coaxially aligned to image from the environment through the objective lens, while the objective lens is designed to control perspective errors (e.g., Figure 5F). For single-lens systems, the outer compressor lens element of the objective lens need not have a polygonal shape; instead, it could be circular or have a freeform contour.

[0144] Environmental effects may also heat or cool the multi-camera capture device asymmetrically. The previously described kinematic mounting or coupling of adjacent camera housings for improved multi-camera capture device 300 (see, e.g., FIG. 11 and FIGS. 12A and 12B ) can help reduce this effect by deflecting and attempting to average out the effects of mechanical stresses. For example, it may be further beneficial to provide channels or materials to transfer or shift asymmetric thermal loads so that they are more evenly shared between or by cameras 700 and their housings 730. With respect to FIGS. 12A and 12B , this means that the space around lens housing 730 and channel centering hub 750 is provided with a compliant, yet highly thermally contacting, thermally conductive material (e.g., Sil-Pad or CoolTherm) to help spatially average out asymmetric thermal loads or differences. At the same time, however, some of the effects of thermal changes on the imaging performance of the camera lenses can be mitigated by judicious selection of both the optical glass and the non-thermal mounting of the optical elements within lens housing 730. In combination, an effective design approach may allow for heat transfer or crosstalk between the camera lenses and their housing 730 to affect the environment, while at the same time isolating the lenses and housing from the sensors and their electronics.

[0145] The improved multi-camera capture device 300, and the cameras 320 therein, may also be protected by an optical dome or shell (not shown) with nominally concentric inner and outer spherical surfaces through which the device can image. The protective optical dome may be faceted to provide multiple outer compressor lens elements, one for each camera channel. The dome may also be a hybrid design, with some faceted to provide outer compressor lens elements and other portions having just concentric inner and outer spherical surfaces. The addition of an outer dome can be used in conjunction with the approximately spherical device of FIGS. 12A and 12B or the approximately hemispherical device of FIG. 14, or to encompass alternative geometries or total FOVs for the device. The dome may consist of a pair of interlocking hemispherical or approximately hemispherical domes joined at a joint, or it may be a single approximately hemispherical shell (e.g., for FIG. 14). The optical dome or shell material may be glass, plastic, or polymer, a hybrid or reinforced polymer material, or a robust optical material such as ceramic, sapphire, or Alon. The optically clean dome or shell helps eliminate environmental contamination and can also function as a FRU and be replaced if damaged. Replacing an FRU dome can be easier than replacing the entire camera 320 or an FRU-type outer lens element or outer lens element assembly. The dome or shell can also be enhanced with an AR, oleophobic, or hydrophobic coating on the outer surface and an AR coating on the inner surface. The use of a dome or shell can also reduce the need or burden of using a carrying case or shipping container, although such an enclosure can still be useful.

[0146] Additionally, imaging systems of the type of FIGS. 12A and 12B including multiple low-parallax imaging lenses of the type of FIG. 8 without an imaging relay, as well as imaging systems of the type of FIG. 14 that may have the nexus internal frame of FIG. 13 with the optical system of the type of FIG. 9, can also be used for display applications. Specifically, instead of placing an image sensor at the objective lens image plane 360 ​​or at the secondary image plane 410 provided by the imaging relay, a display device is placed at either of these locations to create a multi-objective lens projection display system (e.g., as in device 300, but a display system rather than a camera system), which can be used, for example, in movie theaters, dome theaters, simulators, or planetariums. In each imaging channel, the objective lens and imaging relay system work together to image the display device onto a portion of the display screen. The image display device can be an array device with directly addressed pixelated light emitters that directly emit light using LEDs (e.g., microLED arrays), lasers, superluminescent diodes (SLEDs), or quantum dot (Q-Dot) devices. The image display device can also be a pixelated light modulator, such as a reflective liquid crystal on silicon (LCOS) device or a deformable micromirror device (DMD), which modulates passing light provided by separate light sources. The array display device, whether a light-emitting or light-modulating device, can provide multicolor or monochromatic dimming. Because these light-emitting or light-modulating devices are easily larger than the image sensor array, they may be difficult to fit into the embedded imaging surface 355 of the objective lens 320 (e.g., Figures 8 and 12A, B) in the multi-lens device (300). However, a system with an imaging relay 400 can be advantageous due to the additional space. When three light-emitting or light-modulating array devices, one color (RGB) per imaging channel (320), are used, the image light is typically combined into a common optical path by an RGB beam combiner. Therefore, the imaging relay approach (e.g., Figure 15A) is even more beneficial to this architecture due to the additional space it can provide for optomechanics.Additionally, display array devices have demanding cooling needs and therefore may require additional mechanical space that an imaging relay approach can provide. Using such a system, images can be conveniently presented on a wide-angle FOV screen with minimal distortion or other image artifacts within the small image overlap areas corresponding to seams. As an example, for a theater with a dome configuration, this type of projection device may be positioned at or near the theater's hemispherical center to project image content onto a perimeter screen. The projection display device (300) may also be a "quarter-sphere" type system that uses a low-parallax multi-lens imaging device to project onto a combined spherical screen portion nominally 180 degrees wide horizontally and 90 degrees high vertically. Some display pixels of the array display device used in the peripheral display channels of the display device (300) may be removed to provide smooth edges along the outer contours of the entire projected image.

[0147] While this description has emphasized the design of improved multi-camera image capture devices 300 for use in broadband visible or human-detectable applications, these devices can also be designed for narrowband visible applications (modified using spectral filters), ultraviolet (UV), or infrared (IR) optical imaging applications. An improved low-parallax panoramic multi-camera capture device (300) for infrared imaging can support multispectral or hyperspectral imaging, which may also include near-IR (NIR) or short-wave IR (SWIR), mid-wave IR (MWIR), or long-wave IR (LWIR) imaging, and visible imaging. Polarizers or polarizer arrays may also be used. Additionally, while the imaging camera 320 has been described as using an all-refractive design, the optical design can also be catadioptric, using a combination of refractive and reflective elements.

Claims

1. 1. An imaging system for use in a low parallax multi-lens imaging device, the imaging system comprising: an objective lens system including a first lens element group having an outer lens element, a second lens element group in front of an aperture stop, and a third lens element group behind the aperture stop, the outer lens element being a polygonal lens having multiple edges, and the first lens element group, the second lens element group, and the third lens element group direct incident light within a field of view as an image to a first image plane; a relay optical system configured to magnify the image onto a secondary image plane as a magnified image; Equipped with the objective lens system is configured to direct incident light entering the outer lens elements of the first lens element group such that projections of chief rays contained in the incident light and incident proximate to the polygonal lens are focused towards a low-parallax volume disposed behind the first image plane; 1. An imaging system, wherein the objective lens system is configured to correct parallax by limiting a lateral component of spherical aberration at a plane by applying a first weighting to image light from a peripheral field of view and a second weighting to image light from an intermediate field of view, the first weighting being greater than the second weighting.

2. The imaging system according to claim 1 , wherein the parallax is corrected by limiting a vertical width of the low parallax amount.

3. 2. The imaging system of claim 1, wherein the objective lens system is configured to produce a front color artifact and a first lateral color artifact at the first image plane, and the relay optical system is configured to reduce the first lateral color artifact so that the magnified image has a second lateral color artifact that is lower than the first lateral color artifact.

4. The imaging system of claim 1 , wherein the field of view of the objective lens system and the magnification of the relay optics provide a target optical resolution at the secondary image plane.

5. 2. The imaging system of claim 1, wherein the objective lens system and the relay optics are configured to sacrifice one or more optical performances, including spherical, coma, astigmatism, field curvature, distortion, chromatic aberration, and telecentricity, at the first image plane in favor of performance at the secondary image plane.

6. 2. The imaging system of claim 1, wherein the relay optics further includes a beam splitter configured to split incident light into a plurality of optical paths and a plurality of optical sensors, each of the optical sensors being associated with one of the plurality of optical paths.

7. The imaging system of claim 6 , wherein the relay optics further comprises one or more of a zoom optics, a focusing optics, a galvo scanner, a wavefront modulator, and an optical filter.

8. The imaging system of claim 6 , wherein the plurality of optical sensors comprises at least one of a visible image sensor, an infrared image sensor, an event sensor, a neuromorphic sensor, and a bright field sensor.

9. The imaging system of claim 6 , wherein a field of view for one of the plurality of optical sensors substantially coincides with a field of view captured by the objective lens system.

10. 7. The imaging system of claim 6, wherein the relay optics further includes depth sensing optics including a laser ranging system, the laser ranging system including a laser light source, one of the plurality of optical sensors, and beam shaping optics.

11. The imaging system of claim 10 , wherein the laser light source comprises a directionally controlled flash laser light source.

12. The imaging system of claim 10 , wherein the depth sensing system includes a MEMs mirror device that provides directional scanning of laser light from the laser light source.

13. The imaging system of claim 10 , wherein the depth sensing system includes an optical phased array for directionally scanning laser light from the laser light source in at least one scan direction.

14. The imaging system of claim 10 , wherein the beam shaping optics directs the depth-sensing laser light to a focal point at or near an aperture stop of the objective lens system.

15. 11. The imaging system of claim 10, wherein the objective lens system is designed to image visible light, the depth sensing system is designed to emit and detect infrared light, and the depth sensing beam shaping optics provides optical correction / correction for chromatic aberrations encountered with the infrared light.

16. The imaging system of claim 1 , further comprising an outer dome having concentric spherical surfaces through which light enters the objective lens system.

17. the objective lens system is a first objective lens system, the relay optical system is a first relay optical system, the first objective lens system and the first relay optical system include a first image channel, the first image channel further includes a first housing coupled to the first objective lens system and the first relay optical system, and the imaging system further includes: a second housing including a second image channel adjacent to the first image channel and coupled to a second objective lens system and a second relay optical system; Including, the first housing and the second housing are separated by a seam width; The imaging system according to claim 1 .

18. The imaging system of claim 1 , wherein the aperture stop of the objective lens system is nominally imaged onto the aperture stop of the relay optical system.

19. 10. The imaging system of claim 1, further comprising a display device proximate said secondary image surface for displaying said magnified image as a projection display.

20. The imaging system of claim 1 , wherein the objective lens system is further configured to reduce a center of perspective error to two pixels or less.

21. the objective lens system is a first objective lens system, the outer lens element is a first outer lens element, and the imaging system comprises: a polygonal frame having a hollow center; a second objective lens system having a second outer lens element, the second outer lens element being a polygonal lens element including a plurality of edges; Including, the polygonal frame is configured to secure the first objective lens system adjacent to the second objective lens system such that an edge of the first outer lens element is aligned with an edge of the second outer lens element; The imaging system of claim 1 , wherein the imaging light from the first objective lens system and the imaging light from the second objective lens system intersect each other.

Images