Visor-type camera array system

Abstract

The multi-camera imaging system includes a cylindrical frame and a plurality of cameras kinematically mounted on the cylindrical frame. The cameras include outer optical elements cut to have a pair of substantially parallel edges and are configured to capture a field of view with angled edges. The cameras are arranged so that the fields of view of adjacent cameras overlap in an overlapping region along the optical gap between the parallel edges.

Description

Technical Field

[0001] The present disclosure relates to a panoramic low-parallax multi-camera capture device that arranges a plurality of adjacent polygonal cameras in a visor or halo shape and captures an image of an arc-shaped array. The present disclosure mainly relates to its optical and mechanical configurations.

Background Art

[0002] The present disclosure claims the benefit of priority of U.S. Provisional Patent Application No. 65 / 513,721, titled "Visor-Type Camera Array System," and U.S. Provisional Patent Application No. 65 / 513,707, titled "Image Synthesis Using Adjacent Low-Parallax Cameras," both filed on July 14, 2023, and the entire text of each application is incorporated herein by reference.

[0003] This invention was made under government support, based on Grant No. 2136737 awarded by the National Science Foundation of the United States. The government has certain rights in this invention.

[0004] In imaging applications that require real-time situation awareness regarding activities, events, or objects in a wide-angle or panoramic environment, imaging a scene with a gimbal-mounted camera is a common solution. For example, U.S. Patent No. 7,136,726, titled "Aerial Reconnaissance System," describes a system having both internal and external gimbals, each gimbal having at least two degrees of freedom, servo means for orienting the gimbals, and a partial selection unit for sequentially selecting another area portion from the viewing area of the region of interest. Despite being widely adopted (including military drones such as the MQ-9), gimbal-mounted camera systems are limited to a scanning field of view that provides real-time situation awareness only in the direction the camera happens to be facing. Gimbal-mounted camera systems are sensitive to both vibrations from the internal servo mechanism and impacts and vibrations from external factors such as drones or ships on which the system is mounted, leading to a degradation in image quality.

[0005] Alternatively, panoramic imaging can also be provided using a camera with a fisheye lens (e.g., U.S. Patent No. 4,412,726), a fisheye lens with an extended field of view (e.g., >180°, U.S. Patent No. 3,737,214), or two fisheye lenses positioned back-to-back (e.g., U.S. Patent No. 9,019,342). However, fisheye lenses have low resolution and significant distortion, limiting their usefulness in applications where real-time situational awareness of activities occurring in vast environments is required.

[0006] There are also panoramic multi-camera devices in which multiple cameras are arranged on a sphere or the circumference of a sphere, and adjacent cameras are configured to engage along some or all of the adjacent edges. The joint U.S. Patent Application No. 10341559 describes the design of a low parallax imaging lens that can be arranged in a dodecahedron shape to capture panoramic image content within a nearly spherical field of view for capturing cinematic or virtual reality (VR) type image content. The joint patent application publication U.S.20220357645 describes a method for mounting multiple cameras on an optically integrated dodecahedron unit or system. However, this camera system may not be optimized to provide panoramic situational awareness of events occurring at long distances. [Brief explanation of the drawing]

[0007] [Figure 1] This is a perspective view illustrating the use of a multi-camera system in a detection and avoidance (DAA) scenario. [Figure 2] This is a perspective view showing a portion of a single-row visor type multi-camera system using low parallax cameras. [Figure 3A] Figure 2 is a top view showing a portion of a single-row visor type multi-camera system using low parallax cameras. [Figure 3B] Figure 3A is a cross-sectional view showing in detail an exemplary lens design of the type used in one of the cameras. [Figure 3C]This diagram shows the field of view captured by adjacent cameras in a multi-camera system using low-parallax cameras. [Figure 3D] This diagram shows the field of view captured by adjacent cameras in a multi-camera system using low-parallax cameras. [Figure 3E] This diagram illustrates the concept of parallax jump between two adjacent camera channels. [Figure 4A] This is an exploded perspective view showing part of a single-row visor type multi-camera system, illustrating the mounting of the low-parallax camera channel onto a cylindrical frame. [Figure 4B] This is an exploded perspective view showing part of a single-row visor type multi-camera system, illustrating the mounting of the low-parallax camera channel onto a cylindrical frame. [Figure 4C] This is an exploded perspective view showing part of a single-row visor type multi-camera system, illustrating an alternative mounting of the low-parallax camera channel onto a cylindrical frame. [Figure 5] These are side and exploded views of a camera channel assembly showing the sensor mounting design, including thermal insulation. [Figure 6] This is a plan view of an alternative example of a single-row visor type multi-camera system, with the seam between adjacent pairs of camera channels magnified. [Figure 7] This is a perspective view showing a part of a two-row visor type multi-camera system using low parallax cameras. [Figure 8] This is a perspective view showing a portion of a two-row visor type multi-camera system using conventional cameras. [Figure 9] This is a schematic diagram showing imaging of a subject approaching the boundary between two channels. [Figure 10] This flowchart shows how to combine images from adjacent cameras. [Figure 11] This is a cross-sectional view of the arc-shaped arrangement of cameras usable in the system shown in Figure 8. [Modes for carrying out the invention]

[0008] While numerous applications can benefit from improved real-time situational awareness of events and objects within wide-field-of-view (WFOV) or panoramic scenes and environments, aspects of this disclosure relate to enabling improved air traffic safety. In particular, the need for onboard sensor devices to assist in collision avoidance is increasing as autonomous drones (or other unmanned aerial vehicles "UAVs") and eVTOL (electric vertical take-off and landing) aircraft (e.g., flying cars) are developed and put into use. For example, drones and VTOLs can be equipped with acoustic, optical, radar sensors, GPS detectors, and / or ADS-B transponders. However, each of these device types has its drawbacks, and multiple types are often needed in combination to ensure redundancy. As the diversity and density of air traffic increase, the potential problems are likely to become even more serious.

[0009] This disclosure provides an improved optical sensing solution using a visor-configured arc-shaped camera array (e.g., six cameras). This array can be mounted on an unmanned aerial vehicle (drone) or eVTOL and is used for detecting potential collision risks or environmental monitoring during flight. Figure 1 shows an example of such a system, in which an aircraft 150 has an arc-shaped multi-camera system 100 mounted on the nose to detect potential collision risks, including other aircraft or bogie aircraft 160 within a field of view 105. Figure 2 shows details of the visor-type multi-camera system 100, in which adjacent cameras 110 are mounted on a frame 130 having an offset gap or seam 120, and the outer cut lens elements are protected by a protruding hood 115. The mechanical gap or seam 120 spans the distance between the lens housings, while the optical gap or seam is larger and spans the distance from the coated clear aperture (CA) of one camera to the CA of the adjacent camera. For example, a multi-camera system 100, one embodiment of the present invention, simultaneously monitors a field of view (FOV) or sight (FOR) with a horizontal FOV of approximately ±100° and a vertical FOV of approximately ±20°. While aircraft typically fly horizontally, the direction of the FOV can be defined relative to the aircraft 150 rather than the environment. The pitch angle or tilt angle of the aircraft 150 (especially a multi-rotor UAV) can change during operation depending on the plan (speed, etc.) and wind conditions. The multi-camera system 100 can be tilt-adjusted to compensate for this change.

[0010] This system 100 enables fixed-mode detection across the entire field of view, and the gimbal camera system relies on fine-scanning camera operation. The visor-type multi-camera system 100, as shown in Figure 1 and detailed in Figure 2, can image in visible light, infrared (IR) light, or a combination thereof. For visible light imaging, an image sensor with a monochrome or color filter (e.g., a Bayer filter) can be used. The acquired image data is analyzed for collision avoidance to realize detection and avoidance (DAA) functions. Image data from the image sensors is output to an image processor including a GPU, FPGA, or SOC, which is used to algorithmically analyze the airspace sampled by the FOV captured from each camera to detect one or more bogie aircraft 160 or other objects. If a bogie aircraft 160, such as a Cessna 172, is detected, the DAA software tracks it within the captured FOV. This data can be output to another processor to assess the current collision risk and determine appropriate avoidance actions. This data can be transmitted to the autopilot, pilot, or remote operator.

[0011] The DAA bogey detection software can use iterative windowing to support the entire FOR of the camera system 100, or the entire or a portion of the FOV of a specific camera. Real-time detection of uncooperative aircraft or bogey planes 160 flying in the airspace is a challenging task and can be computationally intensive; therefore, windowing, which scans the entire camera FOV to search for new objects at a low frame rate (e.g., 1-10 fps), is useful. Once a potential bogey is detected, it can be adaptively tracked at an increased frame rate (e.g., 30-60 fps) using a lightweight, non-advanced program to detect changes in illumination, position, attitude, and / or direction over time. The software can also simultaneously track multiple objects within the FOV of a single camera 110 or camera channel, or within the FOV of multiple cameras (Figures 1 and 2). The DAA software can include algorithms for recognizing or classifying objects, prioritizing the tracking of the fastest or closest bogey. Bogey distance estimation is then possible through bogey recognition, stereo camera detection, LiDAR scanning, or radar. After detection, Bogey 160 can be tracked using a tracking window, region of interest (ROI), or instantaneous field of view (IFOV), which are areas slightly larger than the Bogey's captured image but much smaller than the total FOV of the camera channel. An open-source example of DAA Bogey detection and tracking software is currently available from Purdue University.

[0012] This disclosure relates to alternative and improved lens system designs and configurations for realizing a visor-type camera system 100 capable of providing improved detection and avoidance, sensing and tracking, search and tracking, navigation, and / or other functions. The camera 110 includes lenses generally designed to optically and photomechanically limit parallax and perspective errors, as described in or improved thereto in the joint application for lens design-related patents (U.S. Patent Application Publication No. 20220252848 and International Application No. 2022173515). These camera lenses are used in the camera 110 of a multi-camera system 100, in which the outer lens elements are typically cut along a polygonal edge, allowing the camera channels 110 to be mounted in close proximity with a narrow gap or seam 120 in between, and the lens design technique controls the behavior of image rays (e.g., parallax and perspective) relative to principal rays along the polygonal lens edge. In particular, these prior joint disclosures describe lens design methods and exemplary lens designs in which the axial entrance pupil and its non-axial variation are located behind or ahead of the image plane.

[0013] As background, in the field of optics, lenses can be characterized by many specifications, including focal length, field of view (FOV), image quality (e.g., MTF), and bandwidth (e.g., for visible light). One of the technical terms in this field is "entry pupil." In context, the position of the entrance pupil is determined by identifying the quasi-axial principal ray incident from object space and passing through the center of the aperture diaphragm, and projecting or extending its vector direction in object space forward to a position where it intersects the optical axis of the lens system. The on-axial principal ray has optical directivity into the lens, compared to the off-axial principal ray which is slightly inclined from the optical axis (e.g., ≤7-10°). On the other hand, the off-axial principal ray usually incident at a much larger angle (e.g., 20°, 40°, or 90°) to the first lens element. In standard lenses such as double Gauss and Cooke triplets, the entrance pupil is located in the front third of the lens system, and its position, as well as the difference in the point where the projections of paraxial and non-paraxial rays intersect the optical axis, are substantially irrelevant to lens design and are therefore not explicitly defined or analyzed. Fisheye lens systems are a partial exception, and can be designed to control distortion at large FOVs (e.g., ±90°) without directly controlling the entrance pupil aberration, although there are also fisheye lens design techniques that directly optimize pupil aberration. In either case, it is generally recognized that in fisheye lenses, the position of the "entrance pupil" varies considerably near the front of the lens system with respect to the principal ray angle and FOV.

[0014] Most optical lenses are designed with little to no consideration of the position or diameter of the entrance pupil. For example, in Rik Littlefield's paper, "Theory of the “no-parallax” point in panorama photography" (Pano Post 7 (2006)), pp. 1-19, it is stated that the perspective of an image is determined by the rays that form it, and since the aperture selects those rays, its position determines the perspective. This paper states that the center of perspective and the no-parallax point are identical and are located at the center of the entrance pupil. However, in some lens designs, such as fisheye lenses, this general theory can break down significantly when considering the difference between the quasi-axial and non-quasi-axial directions of the principal ray. In comparison, these generally assigned patent applications and this application identify and address significant differences in the non-quasi-axial and quasi-axial entrance pupil positions, or the related differences and optimizations of parallax and perspective centers, that were not foreseen in this document.

[0015] In contrast, in the aforementioned co-pending patent application, the optimization of the entrance pupil behind the lens, particularly behind the image plane or beyond the image plane, is an intentional goal, thereby controlling the parallax or perspective difference with respect to the chief ray angle across the FOV. A lens design for parallax control can be achieved by optimizing the lens system using a chief ray constraint, pupil spherical aberration (PSA or PSA_sum), or spherical chromatic aberration pupil aberration (SCPA) terms in an evaluation function within a lens design program (e.g., Code V). The optimization priority is typically directed towards the range of off-axis chief rays passing over or near the cut polygonal lens edge of the outermost lens element (front lens or compression lens). As an example, in a dodecahedron-shaped multi-camera system, the chief rays along the cut polygonal lens edge extend over a range of 31.7 - 37.4°. Considering various system geometries and applications, a typical low-parallax lens design optimizes the parallax control of the maximum FOV angle in the angular range from ±20° to ±40°, although designs at larger or smaller angles are also possible. The residual parallax or perspective error can be tracked in various ways, such as as the angular difference from the nominal geometric angle or as the ratio difference of image pixels.

[0016] These low parallax lenses, including those described herein, can also be designed to control front color, which is the residual color shift of the principal ray at a given principal ray field of view angle. A variable color cropping or vignette effect on the principal ray at the edges of the field of view passing through the lens system can cause iridescent artifacts at the edges of the image projected onto the image plane or image sensor. In such lens systems, the truncated outer lens elements also function as a "fuzzy" field stop, causing the image within the core FOV to underfill the image sensor. In contrast, in typical optical imaging systems, the field stop is located on the image plane or on the image on the image plane, whereas the front lens of these cameras 110 is neither. The magnitude of this fuzzyness depends on the entrance pupil diameter and the degree of overlap of the beam footprints between adjacent fields of view. This "fuzzyness" is also affected by residual front color (color-dependent overlap) and the truncation of the lens edges, which are not as sharp as edges that can be defined by a typical black metal mask used on or near the image plane.

[0017] When applying this low parallax lens design technique, it is useful to track the nominal distance of the on-axial entrance pupil behind the image plane, and the nominal distance or offset of this entrance pupil relative to the device center. Ideally, to optimally control or limit residual parallax, it is desirable that the entrance pupil coincides with the device center. In this case, the mechanical gap or seam between adjacent camera channels is virtually zero. This can be difficult to achieve, especially in multi-camera systems, where the lens elements of each camera channel are mounted in lens housings with finite thickness, resulting in actual gaps or seams between adjacent cameras and consequently an actual offset between the entrance pupil and the device center. For example, in multi-camera systems designed for content capture of nominally close-range environments or activities, such as in film or VR, the actual offset between the entrance pupil and the device center may be relatively small (e.g., 2-5 mm).

[0018] A system (see co-pending U.S. Patent Application Publication No. 2022 / 0357646) composed of a faceted dome or faceted arc of an integrated front lens element provides one approach to reducing both the gap between adjacent camera channels and the offset distance from the entrance pupil to the device center. Alternatively or additionally, the gaps or seams between adjacent camera channels can be optically masked by designing the lens system with a slightly surplus or extended FOV (e.g., XFOV of about 0.3 - 1.0°) for each side or gap. The addition of XFOV contributes to correcting the placement tolerances of camera channels, assisting in camera calibration and image stitching / tiling processing, and reducing dead zones in front of the camera system. However, both the addition or increase of the allowable XFOV and the enlargement of the gap between channels increase the distance from the device center to the center of perspective (COP).

[0019] The aforementioned co-pending lens design application also details design differences that can occur at or near the projected entrance pupil position relative to the nominal on-axis ray entrance pupil position for off-axis chief rays. To explain or contextualize these differences, a number of terms are used, including the no-parallax point (NP point), center of perspective (COP), and low-parallax smidge (LP smidge). In particular, in many practical lens designs, for chief rays passing along the polygonal lens edge of the outer or front lens element, the residual low-parallax smidge when projected from its front lens surface can be locally offset by 1 - 2 mm from the on-axis entrance pupil position. Thus, as one way to refine the lens design, the position or offset distance of the LP smidge with respect to chief rays along the polygonal front lens edge can be optimized with respect to the image plane or device center instead of using the on-axis entrance pupil position in such metrics.

[0020] The "LP" smudge measures the positional variation between the quasiaxial entrance pupil position and the pupil position relative to one or more non-quasiaxial principal rays. This can be measured as a vertical distance difference or length 275 (see Figure 3E) along the optical axis 230, or as an area or volume encompassing the range where the principal rays deviate out of the plane. The relationship between parallax error and field of view and color can also be analyzed using the calculation of the center of view (COP), a parameter more directly related to visible image artifacts than low parallax volume, and can be evaluated as the image pixel error or difference of subjects at different distances from the camera system. The center of view error is essentially the change in principal ray trajectory at multiple object distances, such as objects at close range (3 feet) and objects at "infinity". Within the lens, the COP can be estimated as the position within the LP smudge. Distance differences in COP position or COP jumps between adjacent cameras capturing overlapping FOVs can also be analyzed to evaluate the visual differences due to parallax between cameras.

[0021] The offset of the on-axis entrance pupil or off-axis LP smudge from the device center can also be application-dependent. In applications such as filming movies or virtual reality (VR) content where close-ups are desired, the camera system may be expected to provide focused images of objects at a distance of only 3-4 feet (approximately 0.9-1.2 m), while the maximum focal distance may be around 500 feet (approximately 150 m). In such systems, the gaps and seams between camera channels, as well as the nominal offset of the entrance pupil and off-axis LP smudge from the device center, tend to be modest (e.g., about 6 mm) to keep the blind area in front of the camera system below the minimum imaging distance.

[0022] In summary, for close-range imaging applications, it is desirable for the focal plane (COP) to be far from the front vertex and located near or behind the image sensor, as this offers several advantages, including the following: • Compact mechanical packaging with small optical and mechanical gaps. • COP can be brought significantly closer to the device center. This means that the COP of one camera channel can be brought much closer to the COP of an adjacent camera channel. This means that the parallax difference between channels will decrease, and the parallax error in areas where the FOVs overlap will also decrease.

[0023] In contrast, Figure 2 illustrating an aspect of the present invention shows a multi-camera system 100 having seven low-parallax cameras 110 arranged in an arc to form a visor. For example, this type of system enables improved situational awareness and safety for aircraft and ground vehicles performing "long-range" imaging over a range of several miles. As shown herein, the camera channels 110 are independently mounted on a cylindrical frame 130, with portions of the cameras 110 inserted into the frame 130 and engaging with the outer surface of the cylinder. The primary function of the cylindrical frame is to precisely fix the position of each camera channel, but it also functions as a housing for electronic equipment and, in some cases, as a heat sink for cooling the electronic equipment. Alternatively, the system shown in Figure 2 can employ similar embodiments consisting of a multi-camera capture device in which multiple cameras are arranged in a circular or polygonal configuration. The multi-camera system 100 may also include covers or lids (not shown) on the top and bottom, which can enhance sealing to prevent contamination inside the system and further support external mounting capabilities and internal thermal control (e.g., heat sinks).

[0024] Camera 110 can image a nominally polygonal (e.g., rectangular or square) conical FOV cross-section of the collected and imaged incident light. In the example in Figure 2, the corresponding polygonal outer lens elements are cut only on two opposing sides. As shown in Figure 2, each illustrated camera 110 has an effective aperture of approximately 50 mm in width, images a rectangular field of view, and has a seam or gap of approximately 15 mm in width between them. The cylindrical frame 139 has a diameter of approximately 200 mm. By tapering the lens elements and supporting lens housings inward into a conical or frustoconical shape, the camera channels can be densely arranged with narrow gaps or seams 120 in between. This allows the entire system to have a high packing ratio, for example, the ratio of the total area of ​​the camera apertures (cut size of the outer lens elements) to the arc-shaped system is 85% or more. Each camera channel is provided with a protruding hood 115 to protect it from direct sunlight, internal ghosting, and contact with external foreign objects. The camera channel may also include a transparent protective shield or window (not shown).

[0025] The multi-camera system 100 shown in Figures 2 and 3A can be used for applications other than DAA and sensing / avoidance. For example, the output image data can be used for navigation and inspection purposes. In these and similar applications, it may be advantageous to pivot or tilt the multi-camera system 100 downwards. Alternatively, the system may include an arc or one or more cameras that are pointed or imaged in a downward-tilted position.

[0026] Although the gap 120 may appear mechanically wide, these adjacent camera channels 110 are optically and mechanically mounted in close proximity to the frame 130 to maintain parallelism of the nominal FOV across the intervening seam 120, thus preserving the optical advantage of low parallax control between adjacent cameras 110. In bogie aircraft detection, these cameras can photograph targets several miles away and simultaneously support shooting at minimum focal distances of just 50–100 feet (approximately 15–30 meters). For example, individual cameras use Teledyne 36M image sensors and can provide a resolution of 2–3 feet (approximately 0.6–0.9 meters) per image pixel at distances of 3–5 miles (approximately 4.8–8 kilometers). This is sufficient performance to detect a Cessna aircraft. Image processing software sets a digital ROI around the detected enemy aircraft, enabling tracking over time while improving relative resolution and data / frame rate compared to the surrounding bogie-free areas. The camera can provide an extended FOV (e.g., XFOV ≤ 1°), including an FOV angle overlap 107 that spans both the mechanical seam 120 and the larger optical gap or seam 118 between the lens clear aperture. This XFOV or limited FOV margin keeps the blind spot area below the minimum in-focus imaging distance. This limited XFOV provides tolerance for camera alignment while simultaneously enabling camera calibration and smooth image tiling or stitching. However, because the parallax difference between images captured by adjacent cameras 110 is minimal, images can be tiled in real time without the computational load or image artifacts that occur in typical image stitching processes. By applying low-intensive image stitching software processing during image tiling, calibration differences that could cause image artifacts at the boundaries or gaps of adjacent lenses can be smoothed out. This stitching can be applied dynamically or selectively, such as when a tracked bogey (or its associated ROI) moves from the FOV of one camera 110 to the FOV of an adjacent camera 110.

[0027] Figure 3A shows a cross-sectional view of a visor-type multi-camera imaging system 100 of the type shown in Figure 2, with more optical details but fewer mechanical details. The camera 110 typically includes a lens 140 that causes incident light to form an image on an image sensor (not shown). The lens 140 includes a front compression lens group 142, which may have 1 to 3 lens elements, including at least an outer lens element cut into a polygon. The lens system further includes a multi-lens element wide-angle group 144 positioned before and after the aperture diaphragm and provided in front of the image plane 146 and the associated image sensor. The lens elements of these two lens groups combine to form a polygonal image on the image sensor, and the shape of this polygonal image corresponds in principle to the cut polygonal lens shape of the first lens element.

[0028] Figure 3A also shows the nominal conical or frustoconical shape of the camera 110 and lens 140, but the surrounding lens housing is not shown. Typically, depending on the lens and system design, the taper angle 135 is in the range of 13-18 degrees. This tapered shape of the lens housing allows the camera channels to be placed close together with a narrow intermediate seam 120 in between. Figure 2 shows the camera 110 with outer lens elements that are horizontally trimmed while maintaining vertical rounding to achieve a narrow seam. However, polygonal outer lenses and cameras can also be vertically trimmed to form camera channels with a roughly square or rectangular cross-section. Figure 3A also shows adjacent lens pairs receiving nominally parallel principal incident rays 145. The optical gap 118 can be measured as the angular or distance difference between the principal rays of adjacent channels, or as the nominally small distance between the coated effective apertures of both lenses.

[0029] Furthermore, Figure 3A shows the axial or ΔZ offset distance 122 between the nominal device center 126 of the camera channel 110 and the quasiaxial entrance pupil 124. In applications involving imaging of more distant objects (e.g., detection of a bogie machine several miles away (see Figure 1)), the imaging lens is designed with a longer focal length and higher magnification than for close-range imaging applications (e.g., cinema or VR). As the minimum imaging distance without refocusing increases, the blind spots, gaps, seams 120 between adjacent camera channels, and the offset distance 122 of the quasiaxial / non-quasiaxial "entry pupil" 124 relative to the device center 126 can also be set larger, while maintaining a limited XFOV of approximately 0.5–1.0° per camera channel. For example, in such a system, the nominal axial offset distance 122 between the device center 126 and the on-axial entrance pupil 124 can be 30-70 mm, and the axial offset between the on-axial entrance pupil and the off-axial LP smid or COP can be kept small (e.g., ≤2 mm) through design optimization. The offset distance 122 can also be measured as the distance ΔZ between the device center 126 and the COP of the camera channel.

[0030] As a result of this increase in offset distance or difference ΔZ, the lens 140 in the camera channel 120 can be designed differently compared to lenses for close-range imaging applications with small offset distances (e.g., ≤5mm). In particular, for longer-range imaging applications, parallax and perspective can be sufficiently optimized even when the entrance pupil or nearby LP smear (or NP point or COP) is located on or near the image plane 146, or slightly in front of the image plane (see Figure 3B). As an example, a low-parallax lens system with a focal length of approximately 8mm and a track length of ~60mm from the front lens vertex to the image plane was designed, in which the NP point was optimized using the PSA sum method to a position of approximately 10% of the lens length of 60mm from the image plane, keeping it within acceptable limits.

[0031] Figure 3B is a cross-sectional view showing an example of a lens 240 having 11 optical elements designed for long-range imaging, which can be used in a camera 140. The lens 240 includes a compression lens group 242 and a wide-angle lens group 244 that image light (e.g., a ray beam 212) onto the image plane 246. This exemplary lens includes 10 lens elements and a window that serves as a UV or IR cut filter substrate. The projection of the principal rays in the ray beam 212 is projected toward the entrance pupil 224, and the actual projection position differs between the on-axis principal rays and the off-axis principal rays (e.g., LP smids). The entrance pupil 224 is located in front of the image plane 246 by an image offset distance 228, with the offset 222 extending from the entrance pupil 224 to the device center 226. At least some of the lens elements within the lens 240 may be made of optical glass or optical plastic, or may have an aspherical profile surface, or may be metaoptics with a subwavelength surface structure, or may be a combination thereof.

[0032] Figure 3C shows the two-dimensional core FOVs 262 corresponding to two adjacent channels as the forward region of each camera 240. These each form an image on the image sensor. Three distances from the camera are shown, and as the distance from the multi-camera system increases, the image-forming region of the scene expands. The optical gap 130 is kept constant because the principal rays are parallel at the boundary. The regions are aligned with the optical axis of the left channel. These imaging regions are not vignetted. A wider peripheral region with vignetting exists and is typically cropped from the captured data.

[0033] Figure 3D shows the two-dimensional imaging fields of view of two adjacent cameras 240, which are larger because they include XFOV264. The imaging field of view is shown as the forward region of each camera, which is projected onto the sensor. Three distances from the camera are shown, with the imaging area expanding as the distance increases. The imaging area is aligned with the optical axis of the left channel. The cameras are designed to prevent vignetting within XFOV264.

[0034] In some cases of this type of multi-camera system 100, the edges of the FOV 260 nominally coincide with both the edges of the cutting lens and the effective pixel edges of the image sensor array, and the parallax is optimized for nominal principal ray parallelism along the edges of the cutting lens. Two adjacent cameras can be mounted on a support frame such that these edge principal rays are nominally parallel to each other. This configuration extends the blind area virtually infinitely. However, if, for example, there is a mechanical seam or optical gap, or the blind area is 15-25 mm wide on the outer surface of the frame or on the outer lens 243, the relative resolution loss when photographing a target aircraft several miles away is negligible, given the pixel resolution of 1-3 feet wide at that distance.

[0035] FOV260 can correspond to core FOV262 and can be defined as the largest low-parallax field of view that a given real camera lens 240 can capture. Similarly, core FOV262 can be defined as a sub-FOV of a camera channel whose boundary is nominally parallel to the boundary of the polygon cone (see Figures 4A and 4B). Ideally, by minimizing the seam 160 and with proper control and calibration of FOV orientation, the nominal core FOV262 approaches or matches the ideal FOV, and adjacent core FOVs touch with only a small gap in between. However, in practice, considering the variability in alignment tolerances between cameras, some extended FOV is necessary so that images can be acquired even if the cameras are not perfectly aligned. Additional extended FOV may be necessary to enable geometric camera channel calibration (e.g., intrinsic and extrinsic parameters). Extrinsic parameters represent the position of each camera in the 3D scene. Intrinsic parameters represent the optical center and focal length for each individual camera.

[0036] The inclusion of XFOV264 can be achieved by assigning the core FOV to an area of ​​the image sensor's effective pixel region that does not completely fill the sensor, while reserving pixels in the outer boundary region for the extended FOV. For example, if the image sensor may have 4096 x 5120 effective pixels, a smaller area such as 3800 x 4800 pixels can be assigned to the core FOV while leaving an outer boundary of approximately 150 pixels wide on all four sides. Figure 3D shows a cross-sectional view of the polygonal field of view of two adjacent camera channels, with the fields of view partially overlapping. Each camera channel has a core FOV262 corresponding to a parallax-optimized field of view, and this pair of core FOVs remains parallel to each other even when projected onto object space (environment, scene, etc.). Each camera channel also supports a wider extended FOV264, which overlap to limit the blind spot in front of the camera and provide tolerances for misalignment (offset, tilt, etc.) and camera calibration. Within the overlapping FOV 266 between adjacent camera lenses 240, the associated XFOV 264 completely or partially overlap. While the optimization of the parallax lens design focuses on principal ray alignment to the defined boundary of the core FOV 262, the residual parallax or difference in viewpoint center within the XFOV 264 of the lens 240 is usually still small (e.g., ≤1 pixel).

[0037] Figure 3E illustrates the concept of a parallax jump between two adjacent camera channels 240. Here, the aforementioned LP smid is conceptually represented as an ellipsoid with an LP smid length 275, within which projections of both on-axis and off-axis principal rays cross the optical axis 230. A COP is described as a typical position within the LP smid where parallax within a single camera channel 240 is minimized. The image plane is offset by a distance 277 from the device center 226, and the COP is located within the LP smid length 275 and can be offset by a distance 229 from the image plane. The two COPs are separated by a distance 270 (e.g., causing a parallax jump). Lens design techniques are described that allow for control or optimization of the size and position of the LP smid for monoscopic or single-channel parallax.

[0038] However, these cameras are designed or optimized for use in multi-channel camera systems (100, 300). Each camera channel 240 has its own LP smid and its own COP. However, in an integrated multi-camera system, adjacent cameras 240 are offset by a finite-width seam, and a moderate FOV overlap (Figure 3D) may be included to reduce the extent of the blind spot area corresponding to the seam, thereby creating a moderate parallax difference in the overlapping area of ​​adjacent channels. As shown in Figure 3E, the COPs of adjacent channels are separated from each other by a COP separation distance 270, and this distance or range may be affected by system design constraints such as the size of the image sensor package 247. These system constraints are generally "fixed" by application requirements.

[0039] More specifically, in lens design, the position of the entrance pupil 224 or the position and width of the LP smudge are controllable parameters for altering the COP separation 270. In typical imaging systems, the entrance pupil and COP may not be in close proximity. However, in well-performed PSA lenses, it is possible to optimize the design to intentionally bring the entrance pupil and COP closer together to control parallax within the lens. In many applications, it is desirable to position the entrance pupil behind the image plane and near the device center 226, as this reduces the physical separation distance 270 between adjacent COPs, thereby limiting the amount of parallax occurring at the two-channel boundary. However, this lens design can impose burdens of increased lens length, diameter, weight, and cost. For each application, appropriate trade-offs must be made to determine how to balance these factors.

[0040] In applications or systems with close imaging distances to the camera (e.g., ≤500 feet), low parallax lens design allows for preferential optimization of the entrance pupil 224 position or LP smudge position relative to the device center, and similarly, the LP smudge size. When the distance between cameras, especially the optical spacing, is sufficiently large (around a few millimeters in systems imaging tens of feet away), blind areas due to image loss may occur. However, increasing the distance from the device center to the entrance pupil can provide a reasonable COP separation and FOV expansion to cover a wide field of view and reduce blind areas. This may not be a problem if the "important feature size" is larger than the camera channel spacing. If the design includes a small XFOV 264, the principal rays within that XFOV converge at a finite distance in front of the camera. This is the maximum distance at which information can be obscured (blinded) from the camera. In these systems, it may be desirable to have a small XFOV or overlap to provide optical-mechanical tolerance, camera calibration, and FOV margin for image synthesis.

[0041] In such close-range imaging systems and applications, the distance between principal rays is primarily controlled by optimizing the entrance pupil (LP smear) with respect to the position and range on the camera FOV, including both the core FOV 262 and XFOV 264. As background, in cinema-type multi-camera systems with low parallax camera lenses, close-range imaging is performed, and the ratio of distance from the device center to the entrance pupil / entrance pupil smear length can be made small (e.g., approximately 2:1). In such systems, the influence of lens parallax due to direct parallax optimization (principal rays, PSA) and the influence of focal plane separation 270 can be comparable.

[0042] On the other hand, in systems optimized for longer-range imaging applications, such as enabling DAA sensing for collision avoidance, parallax optimization within the lens can be relaxed, allowing for a larger distance between the device center and the on-axis or off-axis entrance pupil, or between the device center and the viewpoint center 226. For example, as shown in Figure 3A, the offset distance 122 can be several tens of millimeters, and the entrance pupil can be positioned near or in front of the image plane. Figure 3E further illustrates an example where the COPs of two cameras are located further away from the device center. Since the LP smid is still kept small, the ratio of the distance from the device center to the EP / LP smid length can be set relatively large (e.g., about 20:1).

[0043] In such lenses, the entrance pupil can be positioned near or in front of the image plane, which allows for an increase in COP separation of 270 during lens design. Furthermore, by reducing the relative weight of the PSA sum in the lens performance function, image aberration correction can be proportionally prioritized over PA sum reduction for parallax limiting. In this type of application (where the camera system is mounted on an aircraft), positioning the entrance pupil or COP near or in front of the image plane or image sensor can result in a shorter overall length and lighter weight of the lens and lens housing compared to similar lenses where the entrance pupil is located closer to the center of the device.

[0044] Figure 4A shows an external exploded view of a single camera channel 340 separated from the cylindrical frame 330, and Figure 4B shows an internal exploded view, illustrating examples of how the channel and frame can interface. Each camera 340 may include a low parallax lens system 240 of the type shown in Figure 3B. In particular, these figures show parts of a multi-camera system 300 to illustrate an embodiment in which individual low parallax camera channels 340 are kinematically mounted around the arcuate portion of the cylindrical frame 330. Similar to Figure 2, Figures 4A and 4B show cameras 340 having outer lens elements that are horizontally truncated to allow for a narrow seam, but remain rounded vertically. Similar to system 100 in Figure 2, system 300 may have a lid or cover (not shown) with mounting and thermal control functions (e.g., fins). Also, although the frame 330 is cylindrical as shown, it could instead have a polygonal cylindrical shape (e.g., octagonal) and congruent rectangular surfaces on its sides, to which cameras could be kinematically mounted. In this example, the upper and lower covers or lids provide, in principle, two polygonal surfaces that are parallel to each other and have corresponding polygonal shapes.

[0045] The entire camera channel 340 and its lens barrel or housing 345 are housed within a conical or frustoconical space, although the inner portion of the camera housing 345 is locally square rather than tapered. This inner square portion of the housing 345 nominally accommodates the lens elements of the wide-angle lens group and has a mounting portion for connecting to the image sensor substrate 347. In this example, the camera housing 345 is inserted into a roughly square opening or slot 335 of the frame 330. A pair of molded V-pins on the bottom surface of the housing 345 are used to form a V-block 350 that aligns with a ball-shaped portion 354 when the housing 345 is inserted into the slot 335. A V-shaped portion 352 is provided on the inner surface of the housing 345, which is positioned relative to the outer surface of the frame 330. For example, a pair of spring pins 360 are mounted inside the frame 330 to position the lens housing 345 in the slot 335 in the vertical (Z) direction. As another example, a spring 356 attached to the upper and lower surfaces of the slot 335 with shouldered screws 358 positions the housing 345 in the slot 335 in the X and θ-Y directions.

[0046] Figure 4C shows an exaggerated 3D view of an alternative structure for a single-row visor type multi-camera system, showing how the channel 340 is mounted on a cylindrical frame 330. In this example, the outer portion of the camera channel 340 (including the outer lens elements and the outer portion of the lens housing) is truncated in both the horizontal and vertical directions. The inner portion of the housing 345, on the other hand, has a nominally circular cross-section that interfaces with a nominally circular slot 335 on the frame 330.

[0047] In each example, a V-shaped projection incorporated into the channel housing 345 engages with the outer diameter of the frame 330, fixing all degrees of freedom except two (e.g., translation around the z axis and rotation around the x, y, and z axes). These remaining degrees of freedom (e.g., translation around the y and z axes) are eliminated by docking a small V-block 350, located on the bottom of the housing 346, with a ball-shaped component 354 mounted on the frame 330. Mounting hardware (screws) and a compression spring 356 provide the necessary vertical and horizontal engagement forces. The exemplary system 300 shown in Figures 4A-C employs kinematic mounting elements, which are components that make up a simple device providing a connection between two objects, typically corresponding to six local contact areas (i.e., strictly constrained). These contact areas are usually composed of a combination of classical kinematic or strictly constrained mechanical elements such as balls, cylinders, V-blocks, tetrahedrons, cones, and planes. The accompanying mating or retaining force is supplied by a spring or spring pin, but various mechanisms such as springs, spring pins or flyer pins, flexures, magnets, elastic bodies, and adhesives can be used to provide dynamic load forces for mounting and aligning the camera to the cylindrical frame. The frame 330 may also include auxiliary functions (not shown) to maintain the frame's rigidity, shape, and structural integrity to withstand external loads, vibrations, and shocks.

[0048] The positioning tolerance between the camera channel 340 and the frame 330 should be minimized to ensure a sufficiently wide field of view for camera calibration and image synthesis or tiling operations. In addition to manufacturing tolerances, mounting stresses due to changes in environmental conditions such as temperature, shock, and vibration can affect the positioning and directional accuracy of the channel. To achieve the required accuracy, it is essential to employ precision restraint methods using kinematic components such as V-grooves or ball functions to accurately position the channel. Kinematic mounts not only enable repeatable mounting but also minimize stresses due to thermal expansion and vibration.

[0049] In the design example shown in Figures 4A-C, a strict constraint or kinematic mounting method was selected to allow the camera channels 340 to withstand and compensate for external loads and to return to a consistent position while nominally maintaining the previously described mounting or assembly precision, even when the system 300 is subjected to temperature changes. For example, by using average milling and commercially available kinematic components, it is possible to limit the positional variation of each camera channel 340 to less than 75 μm and the rotational variation to less than 0.07 degrees. The camera channels 340 are fixed with spring-loaded hardware, and the springs 356 have sufficient strength to reliably guide and engage the kinematic components. These have the rigidity to hold the camera channels 340 even under shock and vibration. For example, the mounting mechanism can be designed to withstand residual vibrations from a small multi-engine fixed-wing aircraft (peak-to-peak amplitude of up to 0.1 inches in the 5-62 Hz band). At the same time, the springs 356 or spring pins 360 allow the channels to re-engage in the event of shaking from unexpected shock events (e.g., 6-18 G). This function can be assisted by lubrication between kinematic components.

[0050] In this design, where the camera channels 340 are mounted directly and independently to the frame 339, the accumulation or buildup of positioning tolerances can be minimized. This facilitates the use of strict constraints because there is little to no direct mechanical interaction between adjacent camera channels 340, in contrast to the situation in the aforementioned joint patent application where strict spatial constraints necessitated the use of a direct kinematic interface between adjacent camera channels.

[0051] During the design phase, the choice between precise constraints (kinematic) and the partial use of kinematic components (partially kinematic) can be made based on the requirements of the multi-camera system 300, such as systems requiring higher rigidity or lower positional accuracy. For example, the system may have a cylindrical body or housing 345 of camera channels positioned in pilot holes or slots 335 of the frame 330, which can be simply secured with screws. The orientation can be set using screws or additional pins. While this system offers increased rigidity, the clearance between the camera channel housing 345, screws, pins, and corresponding holes can lead to greater positional variations. Some of these variations can be mitigated by high-precision machining, targeted component selection, or by introducing kinematic components to at least some of the constraints (partially kinematic). The multi-camera system 300 may also include active or passive isolation (not shown) at the mounting points to the vehicle or fixtures on which the system is installed, in order to reduce the effects of shocks and vibrations during operation. For example, if System 300 is mounted on an aircraft, ambient vibrations and shock stimuli can be generated by rotors, jet engines, other propulsion systems, or by temperature changes, air and wind turbulence, or shocks during takeoff and landing. This isolation significantly reduces the transmission of shocks and vibrations at the system mounting interface, and further reduces the impact of residual environmental loads reaching System 300 due to its kinematic characteristics.

[0052] The use of plastic lens elements within the lens 240 also increases the sensitivity of the camera channel (340) to external temperature changes, causing thermal shifts in focus. Thermal shifts in focus, primarily caused by the material of the lens barrel or housing 345 (e.g., aluminum), can have a significant impact on optical performance. At least partial thermal denial can be achieved by using a material with a better coefficient of thermal expansion (CTE) or by replacing part of the aluminum housing 345 with a material having an inverse or negative CTE. Figure 5 shows a lens barrel or housing 345 with a taper angle 346 from two perspectives: assembled and disassembled. In these examples, the image sensor substrate 347 is bonded to a plate 367 attached to a structural composite material 365 having a negative CTE to compensate for optical changes. Compensatory thermal shift motion can be achieved by controlling the length of the composite spacer 365 between the image sensor and the mounting point on the lens housing 345. As an example, Allvar Alloys Inc.'s negative coefficient of thermal expansion composite structural material "Allvar" can be used. This material can be constructed in various forms, such as plates or pins, to provide compensatory thermal motion. As a result, the optimized lens can withstand temperatures from -15°C. + Within a temperature range of 55°C, it exhibits only a few microns of residual thermal focus shift.

[0053] Because the lens housing and its internal lens elements are tapered inward, allowing the camera channel 340 to be densely arranged around the frame 330, special care may be required when aligning the image sensor orientation with respect to the case 345 or the camera channel 340. A rectangular image sensor must be aligned with the cross-section of the outer lens element 243 so that the entire square or rectangular image formed fits within the effective pixel area of ​​the image sensor. In contrast, in a standard camera system with a circular lens, the orientation of the image sensor is less critical, and the image sensor can be mounted with an accuracy of a few degrees, after which the camera can be rotated within the mount or frame to adjust the image capture of adjacent cameras to be parallel to each other.

[0054] For example, considering the horizontal field of view, a particular image sensor has a width of 2160 pixels, of which 1740 pixels are used to capture the image. This leaves 420 pixels (210 pixels on each side) to extend the field of view. A multi-camera system (100, 300) can be designed so that 210 pixels, equivalent to 1.5 degrees of the total FOV, overlap with the XFOV (264) of an adjacent channel. This overlapping field of view 266 is used to absorb errors due to calibration, camera boundary setting, and manufacturing tolerances such as variations in camera channel alignment and sensor alignment. Assuming no other error factors exist, with a pixel size of 2 μm, the image sensor would need to shift 210 μm or rotate 1.28 degrees before falling out of the image field of view. The alignment of the image sensor (on substrate 347) relative to camera channel 340 needs to be much more precise so that a sufficient XFOV is ensured for software functionality, even when combined with other errors. An example of margin allocation for the 210 pixels of the XFOV is to allocate up to 9 pixels for part and assembly tolerances, 57 pixels for sensor positioning errors, and 35 pixels for software calibration of external parameters and camera boundary generation.

[0055] When assembling the camera channel 340 for use in the multi-camera system 300, the use of precision measuring instruments and custom fixtures may be required to achieve the assigned sensor alignment error. Most importantly, a unique method is required for aligning the image sensor to the cutting lens or housing edge of the camera channel. This fixture leverages the precise kinematic properties used to mount the camera channel 340 to the frame, controlling the relative position of the image sensor to the camera channel with six degrees of freedom, and first temporarily fixing the camera channel to the fixture. The image sensor alignment fixture includes a temporary masking fixture and a pre-calibrated light source, which illuminates the image sensor with an optical reference to generate a measurable reference image, which is used for image sensor alignment. After the predetermined alignment is achieved, the image sensor can be bonded to the camera channel housing 345.

[0056] Figure 6 shows a cross-sectional view of an alternative single-row visor-type multi-camera system 300, along with an exploded view of the mechanical gap or seam 320 between adjacent low-parallax camera channels 340. In this example, a distance measuring sensor 380, such as an inductive or capacitive proximity sensor, can be used to monitor the width of the seam 320 between adjacent camera channels 340. A sensing plate within the sensor forms a capacitor with the adjacent channel, which changes depending on the distance to the object. This capacitance formed by the sensor plate and channel determines the oscillator frequency, which is adjusted to a monitorable output.

[0057] The capacitive distance measuring sensor 380 typically includes an oscillator, signal conditioning circuit, output driver, and controller. For example, seam width can change dynamically due to the effects of residual shocks and vibrations transmitted through the vibration isolation mechanism and corresponding kinematic features and mating force mechanism, causing a change or displacement from the nominal seam width. However, the distance measuring sensor 380 can provide real-time seam width data, which can be analyzed to determine relative changes on an instantaneous or time-averaged basis. By placing multiple distance measuring sensors 380 in the seam 320, data on inclination changes between adjacent camera channels 340 can be provided. The obtained data can be used to dynamically correct extrinsic calibration and image synthesis processing applicable to image data from adjacent cameras 340. Furthermore, changes in values ​​obtained during system operation can be used as feedback for recalibration. The distance sensor 380 can be inserted into gaps of less than 1 mm and has an accuracy of approximately 1 / 10th of a micron.

[0058] In the preceding figures (e.g., Figures 2, 3, 4A–4C, and 6), a visor-type multi-camera system (100, 300) is shown, having a single row of low-parallax cameras (140, 340) surrounding a portion of the cylindrical circumferential surface. Alternatively, it is possible to provide a halo-like or complete annular system by arranging the cameras along the entire circumference. Figure 7 shows another alternative configuration, in which two rows of adjacent low-parallax cameras 340 are arranged to provide image capture from a more conical FOR and to expand the vertical field of view (FOV). In this example, the multiple cameras 340 are mounted on an annular or barrel-shaped frame, providing controlled seams and image sensor alignment between adjacent cameras in both the horizontal and vertical directions. Using kinematic connections, the cameras in the first row can be coupled to the cameras in the second row, or the cameras in the first row can be individually coupled to adjacent cameras in the second row. In the example in Figure 7, the cameras in the upper and lower rows are shown to be aligned approximately vertically. For example, each camera is aligned vertically with another camera and horizontally with at least one other camera. However, in other examples, the upper row of cameras may be offset relative to the lower row of cameras. For example, the seams between adjacent upper row cameras may be aligned vertically with the lenses of the lower row of cameras.

[0059] Multiple cameras 340 can provide conventional visible light imaging, infrared (IR) imaging, or hybrid visible light and IR imaging (VIS&SWIR). Multiple cameras 340 can utilize various types of optical sensors, including combinations of conventional visible light or infrared (IR) image sensors and event sensors (e.g., Prophesee.ai (Paris, France) or Oculi Inc. (Baltimore, Maryland, USA)). For example, an event sensor camera 341 can be positioned at the outer edge, front, or boundary of the multi-camera system 300 and, leveraging its high dynamic range and fast capture time (e.g., 10,000 fps), can detect fast-moving objects. The event sensor camera 341 can be either a low-parallax camera or a conventional camera. The captured image data can then be used to calculate the predicted vector path of the fast-moving object's image as it moves across the conventional image sensor. Next, a region of interest (ROI) is targeted, and conventional camera image capture settings (e.g., resolution, capture time, or frame rate) can be targeted to improve the conventional camera's image capture of the object.

[0060] Figure 8 shows an alternative configuration of the high-fill-ratio multi-camera visor system 400. The cameras 440 are offset 450 and alternately arranged in two nominally parallel arc-shaped sub-visors 442. As shown, conventional cameras within a given visor 442 have conventional circular or cylindrical outer lens elements and lens housing shapes. The lens housings of these cameras 440 may also have a circular cross-section, a cylindrical cross-section along the longitudinal direction, or be tapered into a moderately inclined frustoconical shape. Each camera 440 has an associated image sensor (not shown) or a mask located adjacent to it, which functions as a field stop, focusing the image light onto a rectangular or square FOV 445. Using conventional cameras eliminates the need for special custom low-parallax lenses and lens housing designs, and the associated costs. However, the type of system in Figure 8 can be larger, heavier, and have lower optical performance.

[0061] In systems 400 employing conventional lens designs, the lens housing is typically cylindrical or slightly tapered (e.g., taper angle ≤ 5 degrees), whereas the lens housing of the low-parallax camera shown in Figures 2 and 3A is frustoconical. As a result, these cameras 440 cannot be easily arranged closely together unless optical bending by mirrors or prisms is included in the optical path. In the latter case, the use of folding mirrors or prisms limits the total number of camera channels that can be placed in a compact mechanical assembly by the space required for optical bending. Therefore, a system 400 with a single-row visor 442 having conventional cameras 440 has a low optical-mechanical aperture ratio (e.g., 20-40%) along the arcuate region. The effective optical coefficient can be improved by having individual cameras 440 capture image light from a larger FOV 445 and allowing them to overlap. This reduces the blind spot area between cameras 440, but the image resolution decreases unless the number of pixels in the image sensor is increased. When large FOVs overlap between adjacent cameras, the image stitching process for creating panoramic composite images increases computational load and artifacts caused by image stitching compared to conventional systems using low-parallax cameras (e.g., Figures 2, 3A, and 4A-C).

[0062] Alternatively, Figure 8 shows a multi-arc array visor imaging system 400 in which two adjacent arc-shaped camera arrays or visor arrays 442 are stacked vertically in a cylindrical shape with an offset 450. Any number of parallel arc-shaped arrays can be used, but it is most likely that two or three stacked arrays or layers will be used. In this example, two rows of arc-shaped arrays of conventional cameras 440 are provided, and the camera channels in a particular arc-shaped visor array 442 exhibit a low optical filling ratio along the arc. However, by providing two arc-shaped visor arrays 442 in which cameras 440 are positioned, the effective optical filling ratio is improved, in which case the cameras 440 in the visor are positioned at an angle to each other along the cylindrical shape. Two adjacent arc-shaped arrays can be nominally parallel to each other but further offset from each other by a small vertical gap (not shown). Adding a second arc-shaped visor array 442 increases the number of cameras 440 but decreases the required FOV and sensor resolution per camera. Figure 8 shows that the visor arrays 442 and associated frames in the upper and lower arc-shaped sections have a cylindrical shape. However, one or both of these can instead have a polygonal cylindrical frame (e.g., octagonal), with cameras mounted on its orthogonal rectangular side surfaces. Multiple multi-camera arc-shaped arrays in the system of Figure 8 can also be stacked in a barrel shape, which allows one arc-shaped visor array 442 to be tilted perpendicularly, inward, or outward relative to a second arc-shaped array 442.

[0063] The multi-row multi-camera system 400 shown in Figure 8 employs conventional cameras 440 and is a potential alternative to multi-camera systems (100, 300) with special low-parallax lenses, as shown in some already illustrated examples (e.g., Figures 2 and 3). When providing equivalent ranging range and FOV, a two-row array of conventional cameras 440 can reduce overall cost compared to a single array of custom low-parallax cameras (140, 340) or lenses 240, or high-resolution, larger FOV cameras. Therefore, conventional cameras 440 do not have the parallax reduction effect of the optical-mechanical design techniques used for the camera lenses in Figures 2 and 3. However, parallax can be reduced in these multiple cameras 440 by mechanically aligning the cameras so that the horizontal edge of the FOV 445 of one camera 449 is parallel to the FOV 445 of the next camera 440. The overlap of the horizontal FOVs between cameras can be reduced to as little as 1-2 degrees. At least the cameras 440 of the first and second arrays 442 are arranged to be angularly offset from one another along an arc, so that at least one camera of the first array is nominally spaced between two cameras of the second array, and the three adjacent cameras function as a continuous imaging array when collecting image light from object space. In other examples, the “conventional” camera 440 shown in Figure 8 can be replaced with a low-parallax camera such as the camera 110 described herein.

[0064] The dual-row system shown in Figure 8 can occupy a larger volume and be heavier compared to the systems in Figures 2 and 3A, which have multiple low-parallax cameras. These differences can be significant in applications such as aircraft-mounted DAAs where size, weight, and power (SWaP) constraints are strict. Furthermore, in the dual-row system 400 (Figure 8), establishing and maintaining rotational alignment between cameras in the upper arc array 442 and adjacent cameras in the lower arc array 442 becomes more difficult compared to the single-row system (Figures 2 and 3A). Kinematic characteristics can be used for the placement and assembly of cameras 440 within or between the visor arrays 442. While this system can employ cameras 440 using standard design methods without parallax correction via PSA sum or principal ray directivity, it employs a structure with cut outer lens elements to reduce the weight of the camera channels and the inter-channel spacing / seam width. A system 400 having multiple visors 442 may also have a low-parallax camera, such as the type shown in Figure 3B. For an arc-shaped arrangement, conventional camera channels can be cut horizontally, vertically, or both. However, this cutting can cause a vignetting effect.

[0065] Even if a vertical offset of several inches 450 occurs between two arc rows 442, the loss of vertical resolution is not significant if the imaging pixels at a distance of 3-4 miles correspond to an area 2-3 feet wide. However, this difference can be significant when imaging a target aircraft at a very close distance of about half a mile. Similarly, a vertical offset 450 between adjacent two-row camera visors 442 complicates camera calibration, inter-camera factory adjustments, and composite tiling processes of images from adjacent cameras 440, for example, when the target bogie aircraft and its associated ROI move from the imaging FOV 445 of one camera 440 to another. For example, an adjacent camera 440 that is vertically offset (450) may capture different direct light (e.g., sunlight, glare, object reflections) that can vary depending on the angle and position, and these differences are amplified by the offset 450.

[0066] Similar to the systems shown in Figures 2 and 3A (100, 300), camera placement and mounting accuracy are essential in the multilayer system of Figure 8 to avoid excessive consumption of the extended field of view due to directional errors. In vertically arranged systems, vertical directional accuracy becomes more critical because interlayer alignment and the respective vertical extended fields of view must be considered. Nevertheless, the same kinematic mounting principles can be applied to this system to achieve the required accuracy under operating conditions. In all these systems, gaps or joints between channels further improve structural stability, a characteristic that is more difficult to achieve in systems where camera channels are close together and therefore need to be constrained by each other.

[0067] The multi-camera systems of the type shown in Figures 2, 3A, or 8 can be "ground" mounted, for example, on poles or buildings, and then used to monitor air traffic of UAVs (unmanned aerial vehicles) and eVTOLs (electric vertical take-off and landing aircraft). Thus, the obtained image data can be used for collision avoidance (e.g., DAA), airspace surveillance for safety (e.g., preventing drone intrusion into airports), intrusion prevention (e.g., anti-UAS operations), etc. For this application, it is advantageous to add cameras facing upward or overhead so that the system is configured in a more hemispherical shape.

[0068] In a multi-camera system such as the two-row system 400 in Figure 8, which uses conventional cameras 400, it can be advantageous to adapt the calibration and compositing tools in Figure 10 to enhance low-parallax imaging. However, proper calibration is required before using compositing. Figure 11 shows a cross-sectional portion of an arc-shaped array of cameras that can be used in the system of Figure 8. In this example, the camera 440 mounted on the frame 430 has a conventional commercially available double Gauss lens that images light onto the image plane 446. The projections of the incident principal rays 455 and 457 are directed toward the entrance pupil 424, which has a finite size (e.g., LP smuse).

[0069] For example, in the system shown in Figure 11, the captured image can be digitally cropped to a shape (e.g., a rectangle) in object space where the FOV 445 matches. This is achieved by a calibration process that creates a mapping from pixel space to object space to identify the cropping position and avoid FOV overlap between cameras 400. Within the large conical region of camera 440, there is a series of principal rays 455 parallel to the principal rays 455 from adjacent cameras 440, but calibration is required to identify them. For example, by using a dot pattern for camera intrinsic calibration, the parallel principal rays can be identified within a given tolerance (e.g., ≤0.3 degrees), and the core FOV can be defined to determine the digital cropping position of the image. Then, during system assembly into a frame, the cameras can be aligned using targets, external calibration, and digital cropping so that the parallel cropped FOV edges of adjacent cameras are adjacent to each other. After cropping the FOV between channels, during image capture, compositing can be applied using a small amount of image overlap (e.g., 3% of half the FOV). Furthermore, there are other principal rays 457 that incident on the outer lens element at a more external field of view position. These converge toward the entrance pupil 44, but cross the optical axis at a different position than the projection of the principal rays 455. These rays contribute to the capture of image light with residual parallax, while covering blind spots through overlap with adjacent cameras.

[0070] This ray mapping technique can also be applied to multi-camera systems using conventional cameras with internal folding mirrors or prisms to allow for a more densely mechanically arranged arrangement of multiple cameras. This technique can be further improved by selecting conventional cameras whose advantageous entrance pupil positions have been confirmed through analysis or testing. For example, lenses can be selected that, when mounted in a frame, preferentially achieve an entrance pupil spacing (or COP offset) that is approximately the same as the offset size of the feature to be detected (e.g., less than 1 / 5 of the feature size). The LP smear size should also be approximately the same as the entrance pupil offset (e.g., about 1 / 10 of the EP offset). A physical mask can also be placed outside the camera, which is aligned to the cropped FOV shape (e.g., a rectangle) and acts as a fuzzy field stop, improving contrast at the FOV edges. Alternatively, a target can be used during calibration to measure the lens magnification and calculate the number of pixels required to obtain the target FOV. The target can be used to measure the required tilt and displacement of the camera relative to a physical reference point or feature point on the housing, while aiming and aligning the central pixel to the center of the target. This data can be used when aligning the camera to the frame.

[0071] (Object detection, recognition, depth estimation, and tracking) This low-parallax camera imaging technology, exemplified by the systems in Figures 2 and 3A, can be applied to detection and avoidance applications (Figure 1), enabling situational awareness through real-time, seamless panoramic imaging. Image data acquired by the image sensor is output to an image processing unit equipped with a GPU, FPGA, or SOC, where an algorithm analyzes the airspace sampled within each camera's field of view (FOV) to detect one or more bogie aircraft. If a bogie aircraft such as a Cessna 172 is detected, it is then used by DAA software to track it within its field of view (FOV). This data is output to another processor, which can assess the current collision risk and determine appropriate collision avoidance actions. This data can then be transmitted to the autopilot, pilot, or remote operator.

[0072] The DAA bogie detection software can simultaneously monitor the entire or a portion of each camera's field of view (FOV) using iterative windowing. Figure 9 shows an example where an image of a bogie aircraft 160 is tracked within a region of interest (ROI) 280 as it approaches the engagement or overlapping field of view (266) of two adjacent cameras (140, 340). Real-time detection of illegal or uncooperative aircraft flying in the airspace is a challenging task and can be computationally very demanding; therefore, windowing, which scans the entire FOV of the cameras to search for new objects at a low frame rate (e.g., 1-5 fps), can be useful. Once a potential bogie 160 is detected, it can be adaptively tracked using a lightweight, non-advanced program to monitor changes in lighting, attitude, and orientation over time. The software can also simultaneously track multiple objects within the FOV 135 of a single camera 110 or within the FOVs of multiple cameras.

[0073] DAA software includes algorithms for recognizing or classifying objects, allowing for priority processing of the fastest or closest bogeys over others. Bogey detection utilizes the Haar cascade classifier to identify specific objects based on features such as size, shape, and color. Bogey distance estimation is then possible through bogey recognition, stereo camera detection, LiDAR scanning, or radar. Once a potential bogey is detected, its movement can be tracked over time using lightweight tracking algorithms such as the Kanade-Lucas-Tomasi (KLT) tracker. Tracking windows, regions of interest (ROIs), and instantaneous field of view (IFOVs) can be used to assist bogey tracking; these can be set slightly larger than the captured image of the bogey, but are limited to a much smaller range than the total FOV of the camera channel. Multi-object trackers, such as the Multiple Object Tracking (MOT) algorithm, allow for simultaneous tracking of multiple objects.

[0074] Various sensors, such as stereo cameras, LiDAR, and radar, can be used to estimate the distance or range of a bogey. Stereo camera detection can estimate distance by calculating a depth map using techniques such as the semi-global matching (SGM) algorithm. LiDAR and radar can estimate distance based on time-of-flight (TOF) or Doppler shift using signal processing algorithms. Depth estimation can be a difficult problem when using only a monocular camera. Methods for determining depth from monocular images involve identifying objects through object recognition and referencing their size from a lookup table. Knowing the size of an object and its corresponding number of pixels allows for the estimation of its distance. Another method is the focal length method, which involves adjusting the position of the image sensor to find the optimal focal point. This knowledge can be used to determine the approximate distance to an object. Machine learning and neural networks can also be used to estimate distance from large training datasets.

[0075] When a low-parallax multi-camera system (e.g., Figure 1) is mounted on a first aircraft (the aircraft itself) and used for capture to support aircraft collision avoidance by DAA software analysis, situations may arise where the bogie aircraft 160 passes through or moves within an overlapping region 127 between two adjacent cameras as it flies toward or away from the first aircraft. In this type of application, regardless of whether the visor system is deployed on the aircraft or ground vehicle, multiple cameras enable panoramic situational awareness of events and objects in the observed environment. In some applications, it may be advantageous to apply synthesis (e.g., Figure 10) to multiple overlapping regions to generate a seamless panoramic image for object or DAA detection analysis. On the other hand, in applications where constraints severely limit system capabilities, such as aerial DAA, it may be preferable to analyze the images from each camera 110 individually and prioritize allocating computing power at any given time to the image content of a portion of the camera's FOV detected by the bogie 160. In such cases, the image synthesis technique (Figure 10) can be selectively applied only when the bogie aircraft passes through the overlapping region and for a short period before and after that passage (Figure 9). In this situation, synthesis is preferentially applied locally to track the bogie as it passes through the overlapping region FOV 107 from the first camera 110 to the second camera 110 within an enlarged digital window containing the bogie image. Alternatively, synthesis can be applied to a wider portion, or even the entirety, of the overlapping region between the two cameras 110 without applying it to the overlapping region between other camera pairs.

[0076] Image transitions from one camera source to another can be managed by an image rendering technique called compositing. When using adjacent low-parallax cameras, parallax errors, background differences, and dynamic scene issues are mitigated, and the amount of FOV overlap between cameras is also reduced. Image compositing combines two images while ensuring identical pixel values. This intermediate image compositing process can be used advantageously without the heavy burden of image stitching or the abruptness of image tiling without image averaging.

[0077] In a multi-camera system, adjacent images captured by adjacent cameras can be assembled into a panoramic composite image by image tiling, stitching, or compositing. In image tiling, each adjacent image is cropped to a predetermined field of view (FOV) and then placed side by side to form a composite image. Before tiling, each image can be enhanced by intrinsic, colorimetric, and extrinsic calibration and correction. While this method is computationally fast, it can introduce image artifacts and discrepancies at or near the tiled edges.

[0078] In comparison, image stitching is the process of combining overlapping views of multiple images to generate segmented panoramic or high-resolution images. Most image stitching techniques require nearly perfect overlap and identical exposure between images to obtain seamless results. For example, algorithms combining direct pixel-by-pixel comparison with gradient descent can be used to estimate these parameters. Characteristic feature points can be found in each image and efficiently matched to quickly establish correspondences between image pairs. When multiple images exist within a panorama, techniques have been developed to calculate a consistent alignment set overall and efficiently discover which images overlap each other. Ultimately, a composite plane is needed to distort or projectively transform and position all aligned images, as well as algorithms to seamlessly combine overlapping images even when parallax, lens distortion, scene motion, and exposure differences exist. However, differences in lighting and exposure, background differences, scene motion, camera performance, and parallax can create detectable artifacts. When using adjacent low-parallax cameras, parallax errors, background differences, and scene motion issues are mitigated, and the overlap of the cameras' FOVs is also reduced. An intermediate process called image stitching, a type of image rendering, can then be advantageously used without the significant burden of image stitching. Image stitching combines two images so that the content from adjacent cameras has substantially identical pixel values ​​or a smooth transition within the local overlapping region. If the residual parallax error in the extended FOV capturing content within or near the seam is also sufficiently small, and the two adjacent cameras are properly aligned, the overlapping image content captured by the two cameras can be quickly cropped, locally averaged, or stitched and included in the output panoramic image. For example, stitching can apply a weighted average to the image content captured by the two adjacent cameras based on the distance from the image center or an estimated distance.

[0079] Image synthesis techniques can also be optimized for specific applications, for example, by identifying and prioritizing cameras that provide locally superior image quality using frequency decomposition, or by performing local corrections from an ideal virtual pinhole assumption using parallax data from camera lenses 240. For example, when predicting the position of an object using a Kalman filter and tracking the object across the boundary between adjacent cameras, calibration data of intrinsic and extrinsic parameters from both cameras can be used to form a perspective projection of pixels defined by the Kalman filter. During such operation, the DAA system, including the visor camera system (100, 300), can use data from the inertial measurement unit (IMU) to assist in compensating for changes in the aircraft's own motion and vibration. Data collected by distance sensors 380 installed within the seam between adjacent lens housings (Figure 6) can also be used to dynamically adapt the application of extrinsic calibration data, image synthesis (e.g., Figure 10), or both, at least in local areas where the bogie and ROI cross the seam or where the FOVs overlap. For example, if the seam width changes, the image synthesis can be actively modified by correcting the application of stored external parameter data using measured position or tilt data, thereby correcting the application of relative internal parameter data, and it is also possible to directly change the parameters of the image synthesis algorithm.

[0080] For this type of DAA application, or for traffic monitoring of UAVs and eVTOLs, and other applications, employing a dual-row visor or halo system is advantageous. In this system, the second visor or halo system is positioned out-of-plane parallel offset to the first system. This second visor or halo system can also image the same spectral band (e.g., visible light, with or without RGB color) and, in conjunction with the first system, enables stereo imaging and distance and depth detection. Alternatively, the second visor can be equipped with another detection modality such as a monochrome, LiDAR, IR, or event sensor camera. Monochrome cameras can be augmented with color data using a trained neural network, integrating color data with the image from a high-resolution monochrome camera using high-resolution enhancement technology. When using an event sensor, high frame rates of 10kFPS or higher can be used to detect sound in the video feed.

[0081] Image synthesis can be applied generally or selectively in the overlapping region 266 of one or both cameras, as needed. Furthermore, offset camera arrays can be positioned so that their overlapping regions coincide or with a radial offset. In the latter case, image data from one camera array can be used as a guide for image synthesis in the corresponding overlapping region of the other camera array.

[0082] As mentioned above, the disparity data of lens 240 can be applied using modeled or measured data, thereby modifying the weighting coefficients within the lens field of view applied during image synthesis, allowing for a more accurate synthesis of the image content of key features in the scene. As another example, the image data of the overlapping region 107 can be analyzed by frequency decomposition to identify the optimal image data obtained from any of the adjacent cameras 110. This ensures that high-quality image data is preferred for at least the key image features during local synthesis in the overlapping region. Image synthesis can also be selectively applied to overlapping regions or parts thereof where high-quality photogrammetry image data is required, while regions with few features are skipped. This synthesis method or its variations are also applicable to the multi-camera systems shown in Figure 7300 and Figure 8400, respectively.

[0083] However, in the multi-camera system of the present invention, the flowchart in Figure 10 illustrates a preferred image synthesis method that can be employed by the processor in the multi-camera system to create a composite within an overlapping region (Figures 3D and 9) where two cameras are providing image data. The overlapping region of the field of view is achieved by designing the camera lenses to have an extended field of view. At least four factors can degrade the quality of the composite or tiled image at the boundary or overlapping region where two adjacent images are joined: color changes, alignment errors, data loss (optical gaps), and differences in parallax alignment with respect to depth or angle, which can cause image artifacts. For example, the "JNDs index" can be used to measure local discontinuities in color, pattern, and content between images of the same object captured by two adjacent cameras within the overlapping region. The first step of the exemplary synthesis technique shown in Figure 10 identifies whether the images correspond to an overlapping region and the two cameras providing the image data. For example, when creating an isometric projection, it determines which cameras in the system have valid image data contributing to the projection of those pixels. Pixel selection can depend on both the camera angle within the overlapping or extended FOV and the distance to the object.

[0084] In the second step of the exemplary compositing method shown in Figure 10, the FOV angle of each camera is determined. In the third step, the FOV angle distance of each camera and the distance to the bisector plane are calculated to determine which quadrant of the overlapping region the image pixels are in. In the fourth step, appropriate linear coefficients are used to estimate the distance to the edge of the overlapping region. This step may include determining and applying the average RMS reprojection error at multiple object space conjugate points to obtain a field overlap measurement, thereby applying and improving the pinhole variation determined along the lens edge. In the fifth step, this information is used to determine how much each camera contributes to the final RGB values. This method can be called spatially varied alpha compositing. In this method, image data from multiple cameras are combined as a weighted average. The weights are normalized so that their sum equals 1.0 and are proportional to the relative "proximity" to the center pixel of a particular camera.

[0085] More specifically, when applying image synthesis in the fifth step (e.g., Figure 10), the image brightness of any pixels detected by both cameras is averaged out. The output image is first corrected for radiometric or colorimetric variations using predetermined calibration data. Similarly, the inter-pixel correspondence of image pixels in the overlapping region between the cameras is predetermined using predetermined intrinsic and extrinsic geometric calibration data. Next, the output pixel values ​​of corresponding pixels in the overlapping region are averaged using one or more weighting coefficients. The corrected images can be kept individually or merged into a larger panoramic image. Thus, the effect of this synthesis method is to provide a smooth transition from the color of one camera to the color of the other camera.

[0086] The image synthesis method shown in Figure 10 can be used as a spatially variable alpha transmission synthesis, combining image data from multiple cameras as a weighted average. The weights are normalized so that their sum equals 1.0 and are proportional to the relative "proximity" to each camera region or the optical axis (central pixel) of each camera. Another method for synthesizing multiple cameras within an overlapping region can be called spatially variable stochastic synthesis. This method is similar to alpha synthesis, but instead of combining image data from multiple cameras, it uses weights to control the stochastic sampling of the corresponding cameras. In particular, stochastic sampling is a type of Monte Carlo method that samples images at appropriate non-equally spaced positions rather than equally spaced ones. All of these image synthesis methods are independent of the image content.

[0087] Note that the synthesis method in Figure 10 is adaptable for use in multi-camera systems where the imaging algorithm for creating isometric projections is integrated into an FPGA (Field-Programmable Gate Array) or equivalent processor, and image synthesis can be achieved using continuous or on-demand pixel projection recalculation. Synthesis correction values ​​can be recalculated quickly in real time with minimal memory load. Alternatively, the image synthesis method in Figure 10 can be applied to multi-camera systems to evaluate overlapping regions and control the synthesis between cameras in overlapping regions using a "Glassfire" based algorithm. The Glassfire algorithm is used to represent the length of the shortest path from a pixel to the boundary of the region containing that pixel, and is advantageous for applications that can support the use of large, pre-calculated Glassfire mapping LUTs, which require a large amount of memory when creating panoramic image reprojections.

[0088] The image stitching method (e.g., Figure 10) can be selectively applied to a portion of the overlapping region 127 when an object or region of interest is identified. Alternatively, if a panoramic stitched image is required, image stitching can be selectively applied to the overlapping region or the ROI within it, provided that the image data within that region is of high quality (e.g., MTF) and highly reliable.

[0089] In some applications, it may be advantageous to apply the image merging method (Figure 10) to multiple overlapping regions to generate a seamless panoramic image for object or DAA detection analysis. On the other hand, in applications where constraints severely limit system capabilities, such as aircraft-mounted DAA, it may be preferable to analyze the images from each camera individually and prioritize allocating computing power at any given time to the image content of a portion or multiple regions of the FOV of the camera in which the bogey was detected. In such cases, the image merging method (Figure 10) can be selectively applied only when the bogey aircraft crosses the overlapping region, and only for a short period before and after that. Under these circumstances, the merging method can be preferentially applied locally, specifically by tracking the bogey from the first camera to the second camera as it passes through the overlapping region within an enlarged digital window containing the bogey image. Alternatively, the merging method may not necessarily be applied to overlapping regions between other camera pairs, but may be applied to a wider portion, or even the entirety, of the overlapping region between two cameras.

[0090] Furthermore, it should be noted that for applications requiring accurate distance data for objects and features, such as photogrammetry and collision avoidance, optimizing the optical design of the low-parallax camera 110 can enable coaxial imaging and LIDAR. For example, the camera's optical design can include a configuration combining a low-parallax objective lens and an imaging relay lens system, the latter having an extended optical path, allowing for the placement of an image sensor in one optical path and a LIDAR scanning system in the other by arranging a beam splitter. Alternatively, the beam splitter can be integrated into the low-parallax objective lens design, allowing both the image sensor and the LIDAR scanning system to operate directly in the objective lens optics and optical path. Another alternative is that a single LIDAR scanning system can be shared among multiple low-parallax objective lenses. In this system, light from a laser light source passes through a beam shaping optical system, is illuminated by a MEMS scanning mirror, and scanned by a predetermined camera system. The beam splitter directs the image light out of the page plane. The resolution of the LIDAR beam may not match the imaging resolution of the camera, but this can be partially compensated for by controlling the address resolution of the LIDAR scan.

[0091] For example, in photogrammetry applications, LiDAR has a lower resolution than low-parallax imaging cameras, which can cause subsampling of the imaged object and its 3D model. However, LiDAR data can improve the accuracy of distance or depth measurements to the imaged object or its internal features. Using the sampled distance data and the difference between the 3D points estimated by photogrammetry and the actual values ​​determined by LiDAR, the positions of scanned 3D points and midpoints can be accurately determined through interpolation. Since LiDAR data adds depth information to spherical image data, it becomes possible to fuse multiple RGB-D spherical images to generate a 3D or 4D vector space representation.

[0092] While the present technology has been described in relation to preferred embodiments, those skilled in the art will readily understand that various changes and / or modifications can be made to the present technology without departing from the intent or scope of the present technology. For example, each claim may depend on any or all of the claims in multiple dependencies, even if such a claim was not initially made.

Claims

1. A multi-camera imaging system, A cylindrical frame, A first camera kinematically mounted to the cylindrical frame, the first camera comprising a first housing and a plurality of first optical elements disposed within the housing, the first optical elements including first outer optical elements cut to have at least a pair of first nominally parallel edges, and the first camera configured to capture a first field of view having first corner edges nominally parallel to the first nominally parallel edges, A second camera kinematically mounted to the cylindrical frame, the second camera comprising a second housing and a plurality of second optical elements disposed within the second housing, the second optical elements including a second outer optical element cut to have at least a pair of second nominally parallel edges, and configured to capture a second field of view having a second angular edge nominally parallel to the second nominally parallel edge, Equipped with, By controlling the position and range of the entrance pupil, the first camera is optically designed to have nominal low parallax along the first nominally parallel edge, and the second camera is optically designed to have nominal low parallax along the second nominally parallel edge. The first camera is positioned adjacent to the second camera such that the first field of view overlaps with the second field of view in an overlapping region along the optical gap width between the edges of the nominally parallel edges of the first and the edges of the nominally parallel edges of the second. A multi-camera imaging system in which image artifacts in the overlapping region are reduced by both reduced perspective difference and reduced dynamic shift between the first camera and the second camera.

2. The multi-camera imaging system according to claim 1, further comprising one or more kinematic elements configured to kinematically attach at least one of the first camera or the second camera to the cylindrical frame, the kinematic elements comprising one or more of a ball, cylinder, V-shape, tetrahedron, cone, or flat.

3. The multi-camera imaging system according to claim 1, wherein the holding force is provided by a spring, spring pin, flexure, magnet, elastic body, adhesive, or a combination thereof for attaching the camera to the cylindrical frame and providing alignment support.

4. The multi-camera imaging system according to claim 1, wherein the first image content captured in the overlapping region by the first camera and the second image content captured in the overlapping region by the second camera are combined to provide a smooth transition when the first image content and the second image content are combined.

5. The multi-camera imaging system according to claim 4, wherein the synthesis is dynamically provided within the overlapping region when a region of interest related to specific captured image content crosses at least a portion of the overlapping field of view.

6. The multi-camera imaging system according to claim 4, wherein the synthesis is dynamically provided within the overlapping region based on data acquired by a sensor that measures the change in distance between the first housing and the second housing.

7. The multi-camera imaging system according to claim 1, wherein the parallax of the first camera is determined at least in part on the distance from the entrance pupil of the first camera or the viewpoint center (COP) associated with the first camera to the center of the multi-camera imaging system.

8. The multi-camera imaging system according to claim 7, wherein the parallax of the first camera or the second camera is further determined by the distance between the COP of the first camera and the COP of the second camera.

9. The multi-camera imaging system according to claim 7, wherein the parallax of the first camera is further determined by the length of the low parallax volume including the entrance pupil of the first camera or the projection of the COP for both the on-axis principal rays and non-axis principal rays incident on the first camera.

10. The first camera or the second camera is affected by thermal focus shift, The aforementioned multi-camera imaging system is, Image sensor and A structure comprising a material with a negative coefficient of thermal expansion (CTE) for coupling the image sensor to the first housing, The multi-camera imaging system according to claim 1, further comprising:

11. The first camera and the second camera comprise a first camera array, and the multi-camera imaging system is The multi-camera imaging system according to claim 1, further comprising a second camera row having at least two adjacent cameras adjacent to the first camera row.

12. The multi-camera imaging system according to claim 11, wherein the seam between the first camera and the second camera is offset with respect to the seams between adjacent cameras in the second camera row.

13. The multi-camera imaging system according to claim 11, wherein the first field of view and the second field of view are horizontally offset with respect to the field of view of the cameras in the second camera row.

14. The multi-camera imaging system according to claim 1, wherein the first camera and the second camera are optically designed to control the parallax in combination, so as to prevent image artifacts resulting from the difference in residual parallax and dynamic shift between the first camera and the second camera.

15. The multi-imaging system according to claim 1, wherein the first housing or the second housing is nominally maintained in position and orientation with respect to the cylindrical frame in six degrees of freedom (DOF) by a retaining force mechanism complementary to a series of kinematic features.

Images