Hypercamera with shared mirror

Abstract

The imaging system may include first and second cameras configured to capture first and second sets of oblique images along first and second scan paths, respectively, over the region of interest. A driver is coupled to a scanning mirror structure having at least one mirror surface and configured to rotate the structure about a scan axis based on a scan angle. The first and second cameras each include a respective lens having an optical axis set at an oblique angle relative to the scan axis and focusing the first and second imaging beams reflected from the mirror surface onto an image sensor disposed in each of the cameras. The first and second imaging beams captured by the respective cameras may vary depending on the scan angle. Each of the image sensors captures a respective set of oblique images by sampling the imaging beam at first and second values ​​of the scan angle.

Description

[Technical Field]

[0001] The present invention relates to an efficient aerial camera system and efficient method for creating orthomosaics and textured 3D models from aerial photographs. [Background technology]

[0002] The background information provided herein is intended to provide a general context for this disclosure. The research of the inventors named herein is not considered prior art to this disclosure, either explicitly or implicitly, to the extent that the research is described in this background art section, nor any other description that would otherwise be considered prior art at the time of filing.

[0003] Orthomosaics, orthophotos, are mosaics of precisely georeferenced orthophotos that can be created from aerial photographs. In such cases, these photographs can provide useful images of an area, such as the ground. Creating an orthomosaic requires the systematic acquisition of overlapping aerial photographs of the region of interest (ROI), both to ensure complete coverage of the ROI and to ensure that the images have sufficient redundancy to allow for precise bundle adjustment, orthorectification, and alignment of the photographs.

[0004] Bundle adjustment is the process of refining redundant ground point estimations and camera orientation. Bundle adjustment can be performed on manually identified ground point locations, or increasingly, on automatically identified ground feature locations that are automatically matched across overlapping photographs.

[0005] Overlapping aerial photographs are typically taken by navigating a survey aircraft in a meandering pattern over an area of ​​interest. The survey aircraft is equipped with an aerial scanning camera system, and the meandering flight pattern ensures that the photographs taken by the scanning camera system overlap both along the flight lines within the flight pattern and between adjacent flight lines. [Overview of the project] [Problems that the invention aims to solve]

[0006] While such scanning camera systems may be useful in some cases, they are not without drawbacks. Examples of such drawbacks include: (1) the difficulty of fitting multiple long focal length lenses and aligned aperture mirrors within the configured space on the vehicle to capture vertical and oblique images; (2) the fact that camera holes in aerial vehicles are generally rectangular, but the spatial requirements for yaw-compensated gimbals are defined by circles, thus resulting in inefficiencies in spacing; and (3) the presence of low-resolution images (including out-of-focus images and vignetting). [Means for solving the problem]

[0007] This disclosure is directed to an imaging system comprising: a first camera configured to capture a first set of oblique images along a first scanning path over a target area; a second camera configured to capture a second set of oblique images along a second scanning path over a target area; a scanning mirror structure having at least one mirror surface; and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the first camera has an optical axis set at an oblique angle with respect to the scanning axis and includes a lens that focuses a first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera; and the second camera has an optical axis set at an oblique angle with respect to the scanning axis. The system includes a lens that has an optical axis set at an oblique angle and focuses a second imaging beam reflected from a scanning mirror structure onto the image sensor of a second camera, wherein at least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, changes according to the scanning angle, the image sensor of the first camera captures a first set of oblique images along a first scanning path by sampling the first imaging beam at a first value of the scanning angle, and the image sensor of the second camera captures a second set of oblique images along a second scanning path by sampling the second imaging beam at a second value of the scanning angle.

[0008] This disclosure is directed to an imaging method, which includes reflecting a first imaging beam from a target area to a first image sensor of a first camera using a scanning mirror structure having at least one mirror surface to capture a first set of oblique images along a first scanning path of a target area, the first camera comprising a first lens for focusing the first imaging beam onto the first image sensor; reflecting a second imaging beam from a target area to a second image sensor of a second camera using a scanning mirror structure to capture a second set of oblique images along a second scanning path of the target area, the second camera comprising a second lens for focusing the second imaging beam onto the second image sensor; rotating the scanning mirror structure around a scanning axis based on a scanning angle, such that at least one of the elevation and azimuth angles of the first and second imaging beams changes according to the scanning angle; setting the optical axes of the first and second cameras at an oblique angle with respect to the scanning axis; and sampling the first and second imaging beams at the value of the scanning angle.

[0009] The present disclosure is directed to an imaging system mounted on a vehicle, comprising: a first camera configured to capture a first set of oblique images along a first scanning path over a target area; a scanning mirror structure having at least one mirror surface; a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a processing circuit configured, at least in part, to set the scanning angle of the scanning mirror structure based on the yaw angle of the vehicle, wherein the first camera comprises a lens having an optical axis set at an oblique angle with respect to the scanning axis and focusing a first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera, wherein the azimuth angle of the first imaging beam captured by the first camera changes with respect to the scanning angle and the yaw angle of the vehicle, and the image sensor of the first camera captures a first set of oblique images along a first scanning path by sampling the first imaging beam at the value of the scanning angle.

[0010] This disclosure is directed to a method which includes reflecting a first imaging beam from a region to a first image sensor of a first camera using a scanning mirror structure having at least one mirror surface to capture a first set of oblique images along a first scanning path of a region to a region, the first camera comprising a lens that focuses the first imaging beam onto the first image sensor; rotating the scanning mirror structure around a scanning axis based on a scanning angle, the value of which is determined at least in part based on the yaw angle of a vehicle including the scanning mirror structure, and the azimuth angle of the first imaging beam captured by the first camera varies in accordance with the scanning angle and the yaw angle of the vehicle; and sampling the first imaging beam at the value of the scanning angle.

[0011] This disclosure is directed to an imaging system, comprising: a camera configured to capture a set of oblique images along a scanning path over a region of interest; a scanning mirror structure having at least one surface for receiving light from the region of interest, wherein the at least one surface has at least one mirror portion of at least one second portion configured to be positioned on the outer periphery of a first mirror portion and composed of a low-reflectivity material, the low-reflectivity material having a lower reflectivity than the first mirror portion; and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a rotation axis based on a scanning angle, wherein the camera comprises a lens for focusing an imaging beam reflected from at least one surface of the scanning mirror structure onto the camera's image sensor, wherein at least one first mirror portion is configured to reflect light from the region of interest over a set of scanning angles selected to produce a set of oblique images, and at least one second portion is configured to wrap around the first mirror portion and block light received by the camera at scanning angles beyond the set of scanning angles, and the camera's image sensor captures a set of oblique images along a scanning path by sampling the imaging beam at the values ​​of the scanning angles.

[0012] This disclosure is directed to an imaging system housed in a vehicle, comprising a camera configured to capture a set of images along a scanning path over a target area, a scanning mirror structure having at least one mirror surface, and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the camera comprises a lens that focuses an imaging beam reflected from the scanning mirror structure onto the camera's image sensor, at least one of the elevation angle and azimuth angle of the imaging beam captured by the camera changes with respect to the scanning angle, the camera's image sensor captures a set of images along the scanning path by sampling the imaging beam at values ​​of the scanning angle, the illuminance of the image sensor by the imaging beam is reduced by at least one of partial obstruction by a constrained space in which the imaging system is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of scanning angles, the values ​​of the scanning angle along the scanning path being selected based on a model representing the illuminance of the image sensor by the imaging beam.

[0013] The present disclosure provides a method for reducing vignetting, comprising: reflecting an imaging beam from a target area to a camera image sensor using a scanning mirror structure having at least one mirror surface to capture a set of images along a scanning path of the target area, wherein the illuminance of the image sensor by the imaging beam is reduced by at least one of partial occlusion by a constrained space on which the imaging system including the scanning mirror structure is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of scanning angles; rotating the scanning mirror structure around a scanning axis based on a scanning angle that changes at least one of the elevation angle and azimuth angle of the imaging beam, wherein the value of the scanning angle is at least partially based on a model of illumination of the image sensor by the imaging beam; sampling the imaging beam at the value of the scanning angle; cropping at least some portions of images in the set of images affected by vignetting; and stitching one or more images in the set of images after the cropping has removed at least some portions affected by vignetting.

[0014] This disclosure is directed to an imaging system mounted in a confined space within a vehicle, which comprises a camera configured to capture a set of images along a scanning path over a target area, and includes an aperture, a lens, and an image sensor; a scanning mirror structure having at least one mirror surface; and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the lens focuses an imaging beam reflected from at least one mirror surface of the scanning mirror structure onto the image sensor, at least one of the azimuth and elevation angles of the imaging beam reflected to the camera changes with respect to the scanning angle; the camera's image sensor captures a set of images along the scanning path by sampling the imaging beam at the value of the scanning angle; and the camera's aperture is configured to be dynamically adjusted such that at least one of the following holds true: the aperture remains within the projection geometry of at least one mirror surface onto the aperture when capturing a set of images, or the aperture remains within a region of light that is not obstructed by the confined space over the scanning path.

[0015] This disclosure is directed to a method for controlling an imaging system mounted in a vehicle, which includes reflecting an imaging beam from a region to a camera image sensor using at least one mirror surface of a scanning mirror structure to capture a set of images along a scanning path of the region to be captured, the camera comprising a lens and an aperture, rotating the scanning mirror structure around a scanning axis based on a scanning angle, the azimuth and elevation angles of the imaging beam reflected to the camera vary with the scanning angle, sampling the imaging beam at a value of the scanning angle, and dynamically adjusting the aperture of the camera such that the aperture remains within the projection geometry onto the aperture of at least one of the following: the aperture remains within a region of light that is not obstructed by a constraining space above the scanning path.

[0016] This disclosure is directed to an imaging system mounted within the confined space of a vehicle, comprising: a scanning mirror structure having at least one mirror surface; and a camera configured to capture a set of images along a scanning path over a target area, wherein the camera comprises a lens that focuses an imaging beam reflected from at least one mirror surface of the scanning mirror structure onto the camera's image sensor; a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a circuit configured to form eclipse data in one or more scanning path arrangements due to a decrease in illumination of the image sensor by the imaging beam, and to update the pixel values ​​of one or more images in the set of images according to the eclipse data at the corresponding scanning angle, wherein at least one of the elevation angle and azimuth angle of the imaging beam captured by the camera changes with respect to the scanning angle, and the camera's image sensor captures a set of images along the scanning path by sampling the imaging beam at the values ​​of the scanning angle, wherein the decrease in illumination of the image sensor by the imaging beam is caused by at least one of partial obstruction by the confined space in which the imaging system is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of the scanning angle.

[0017] The present disclosure provides a method for reducing vignetting, comprising: reflecting an imaging beam from a target area to an image sensor of a camera using a scanning mirror structure having at least one mirror surface to capture a set of images along a scanning path of a target area, wherein the camera comprises a lens that focuses the imaging beam onto the image sensor; rotating the scanning mirror structure about a scanning axis based on a scanning angle, wherein at least one of the azimuth and elevation angles of the imaging beam changes with respect to the scanning angle; forming vignetting data for one or more arrangements along the scanning path due to partial occlusion of the imaging beam, wherein the reduction in illumination of the image sensor due to the imaging beam is caused by at least one of partial occlusion by a constrained space on which the imaging system including the scanning mirror structure is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of the scanning angle; and updating the pixel values ​​of one or more images in the set of images according to the vignetting data.

[0018] This disclosure is directed to an imaging system, which is a camera configured to capture an image of a target area from an imaging beam from the target area, and comprises: a camera comprising an image sensor and a lens; one or more glass plates positioned between the image sensor and the lens of the camera; one or more first drive units coupled to each of the one or more glass plates; a scanning mirror structure having at least one mirror surface; a second drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a motion compensation system configured to determine at least one of a plate rotation speed and a plate rotation angle based on the relative dynamics between the imaging system and the target area and the optical properties of the one or more glass plates, and to control one or more first drive units to rotate one or more glass plates around one or more predetermined axes based on the corresponding plate rotation speed and plate rotation angle.

[0019] This disclosure is directed to an imaging method comprising: reflecting an imaging beam from a target area to an image sensor of a camera using at least one mirror surface of a scanning mirror structure to capture a set of images along a scanning path of the target area, wherein the camera comprises a lens and an image sensor; capturing an image from the imaging beam from the target area reflected by at least one mirror surface using the image sensor of the camera; positioning one or more glass plates between the image sensor of the camera and the lens; determining a plate rotation speed and plate rotation angle based on one of the characteristics of the camera, the characteristics and positioning of one or more glass plates, and the relative dynamics between imaging and the target area; and rotating one or more glass plates about one or more predetermined axes based on the corresponding plate rotation speed and plate rotation angle.

[0020] A more complete understanding of this disclosure is provided by referring to the following detailed description when considered in relation to the attached drawings. [Brief explanation of the drawing]

[0021] [Figure 1a] This figure shows a scanning pattern of a scanning camera system captured from a stationary flying vehicle, according to an exemplary embodiment of the present disclosure. [Figure 1b] This figure shows an overlapping set of scan patterns of a scanning camera system taken from a stationary flying vehicle, according to an exemplary embodiment of the present disclosure. [Figure 2] This figure shows a meandering flight path that an aerial vehicle may take to capture an image using a scanning camera system, according to an exemplary embodiment of the present disclosure. [Figure 3] This figure shows a distribution view of a scanning camera system in various ground-based configurations according to an exemplary embodiment of the present disclosure. [Figure 4a] This figure shows a scanning drive unit from a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 4b] This figure shows a scanning drive unit from a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 4c] This is a top view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 4d] This is a perspective view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 4e] This figure shows a first set of potential geometries for a scanning mirror structure within a scanning drive unit, according to an exemplary embodiment of the present disclosure. [Figure 4f] This figure shows a second set of potential geometries for a scanning mirror structure within a scanning drive unit, according to an exemplary embodiment of the present disclosure. [Figure 4g] This figure shows the potential geometry for a scanning mirror structure and a paddle flap according to one exemplary embodiment of the present disclosure. [Figure 5a] This figure shows another scanning drive unit from a first viewpoint, according to an exemplary embodiment of the present disclosure. [Figure 5b] This figure shows a scanning drive unit from a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 5c] This is a top view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 5d] This is a perspective view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 5e] This figure shows the potential geometry for a primary mirror in a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 5f] This figure shows the potential geometry for a secondary mirror in a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 6a] This figure shows another scanning drive unit from a first viewpoint, according to an exemplary embodiment of the present disclosure. [Figure 6b] This figure shows a scanning drive unit from a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 6c] This is a top view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 6d] This is a perspective view of a scanning pattern captured by a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 6e] This figure shows the potential geometry for a primary mirror in a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 6f] This figure shows the potential geometry for a secondary mirror in a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 7a] This figure shows a scanning camera system from a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 7b] This figure shows a scanning camera system from a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 7c] This figure shows a scanning camera system from a third viewpoint according to an exemplary embodiment of the present disclosure. [Figure 7d] This figure shows a scanning camera system from a fourth viewpoint according to an exemplary embodiment of the present disclosure. [Figure 7e] This is a top view of a scanning pattern captured by a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 7f] This is a perspective view of a scanning pattern captured by a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 8a] These are a top view and a perspective view of a scanning pattern taken from a flying vehicle in forward motion, according to an exemplary embodiment of the present disclosure. [Figure 8b] These are top and perspective views of multiple sets of scanning patterns taken from a flying vehicle in forward motion, according to an exemplary embodiment of the present disclosure. [Figure 8c]These are top and perspective views of multiple sets of scanning patterns according to an exemplary embodiment of the present disclosure. [Figure 9] This is a system diagram according to one exemplary embodiment of the present disclosure. [Figure 10] This is another system diagram according to one exemplary embodiment of the present disclosure. [Figure 11] This is another system diagram according to one exemplary embodiment of the present disclosure. [Figure 12] This figure illustrates the refraction of light at a glass plate according to one exemplary embodiment of the present disclosure. [Figure 13a] This is a perspective view showing an arrangement configuration for motion compensation in a camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 13b] This is a side view showing an arrangement configuration for motion compensation in a camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 13c] This is a view along the optical axis showing the arrangement configuration for motion compensation in the camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 14a] This is a perspective view showing another arrangement configuration for motion compensation in a camera of a scanning camera system, according to an exemplary embodiment of the present disclosure. [Figure 14b] This is a side view showing an arrangement configuration for motion compensation in a camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 14c] This is a view along the optical axis showing the arrangement configuration for motion compensation in the camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 15a] This is a perspective view showing another arrangement configuration for motion compensation in a camera of a scanning camera system, according to an exemplary embodiment of the present disclosure. [Figure 15b] This is a side view showing an arrangement configuration for motion compensation in a camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 15c]This is a view along the optical axis showing the arrangement configuration for motion compensation in the camera of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 16] This figure shows the trajectories of the inclination (up), inclination velocity (middle), and inclination acceleration (down) for plate tilt motion according to an exemplary embodiment of the present disclosure. [Figure 17a] This figure shows various target area projection geometries and corresponding sensor plots for motion compensation according to an exemplary embodiment of the present disclosure. [Figure 17b] This figure illustrates the motion-compensated pixel velocity (top) and the corresponding tilt velocities (bottom) for the first and second optical plates, according to an exemplary embodiment of the present disclosure. [Figure 18a] This figure illustrates a target area projection geometry and corresponding sensor plot for motion compensation according to one exemplary embodiment of the present disclosure. [Figure 18b] This figure illustrates the motion-compensated pixel velocity (top) and the corresponding plate velocities (bottom) for the first and second optical plates, according to an exemplary embodiment of the present disclosure, as shown in Figure 18a. [Figure 19a] This figure shows the tilt trajectory of a first optical plate from Figure 18b, which may be used to achieve motion compensation for a required tilt speed according to one exemplary embodiment of the present disclosure. [Figure 19b] This figure shows the tilt trajectory of a second optical plate from Figure 18b, which may be used to achieve motion compensation for a required tilt speed according to one exemplary embodiment of the present disclosure. [Figure 20a] This figure illustrates the pixel speed and tilt speed for a first scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 20b] This figure illustrates the pixel speed and tilt speed for a second scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 21a]This figure illustrates the pixel speed and tilt speed for a first scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 21b] This figure illustrates the pixel speed and tilt speed for a second scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 22a] This figure illustrates the pixel speed and tilt speed for a first scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 22b] This figure illustrates the pixel speed and tilt speed for a second scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 23a] This figure illustrates the pixel speed and tilt speed for a first scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 23b] This figure illustrates the pixel speed and tilt speed for a second scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 24] This is a diagram of a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 25] These are top and bottom views of a scanning camera system in an inspection hole in the absence of roll, pitch, or yaw, according to an exemplary embodiment of the present disclosure. [Figure 26] These are top (upper) and bottom (lower) views of a scanning camera system in an inspection hole, with the roll corrected using a stabilization platform, according to an exemplary embodiment of the present disclosure. [Figure 27] These are top and bottom views of a scanning camera system in an inspection hole, with pitch corrected using a stabilization platform, according to an exemplary embodiment of the present disclosure. [Figure 28] These are top (upper) and bottom (lower) views of a scanning camera system in an inspection hole, with yaw compensated using a stabilization platform, according to an exemplary embodiment of the present disclosure. [Figure 29]These are top (upper) and bottom (lower) views of a scanning camera system in an inspection hole, according to an exemplary embodiment of the present disclosure, in which the stabilization platform is not correcting for yaw. [Figure 30a] A top view and a perspective view of a scanning pattern for a scanning camera system when a flying vehicle has yaw, according to an exemplary embodiment of the present disclosure. [Figure 30b] A top view and a perspective view showing three sets of scanning patterns having forward overlap to a scanning camera system when a flying vehicle has yaw, according to an exemplary embodiment of the present disclosure. [Figure 31] These are top and bottom views of a scanning camera system in an inspection hole, according to an exemplary embodiment of the present disclosure, when the flying vehicle has a yaw corrected by an offset scanning angle. [Figure 32a] A top view and a perspective view of a scanning pattern for a scanning camera system when a flying vehicle has yaw, according to an exemplary embodiment of the present disclosure. [Figure 32b] A top view and a perspective view showing three sets of scanning patterns having forward overlap to a scanning camera system when a flying vehicle has yaw, according to an exemplary embodiment of the present disclosure. [Figure 33a] This figure illustrates an example embodiment of the present disclosure in which an image is captured without a ghost image beam. [Figure 33b] This figure illustrates an example embodiment of the present disclosure in which an image is captured with a ghost image beam. [Figure 34a] This figure illustrates a hybrid mirror having a low-reflection material according to one exemplary embodiment of the present disclosure. [Figure 34b] This figure illustrates the use of a hybrid mirror to prevent ghost images, according to one exemplary embodiment of the present disclosure. [Figure 35a] This figure illustrates diameter erosion caused by an inspection hole according to one exemplary embodiment of the present disclosure. [Figure 35b] This figure illustrates diameter erosion caused by an inspection hole according to one exemplary embodiment of the present disclosure. [Figure 36a] This figure shows an image of a surface without uniform texture affected by erosion, according to an exemplary embodiment of the present disclosure. [Figure 36b] This figure illustrates the erosion of the aperture in various configurations in the images from Figure 36a, according to an exemplary embodiment of the present disclosure. [Figure 36c] This figure shows an image obtained using a modified aperture according to an exemplary embodiment of the present disclosure, which has less aperture vignetting. [Figure 36d] This figure shows an example of a region that can define an aperture according to one exemplary embodiment of the present disclosure. [Figure 36e] This figure shows an example of a region that can define an aperture according to one exemplary embodiment of the present disclosure. [Figure 36f] This figure shows an example of a region that can define an aperture according to one exemplary embodiment of the present disclosure. [Figure 36g] This figure shows an example of a region that can define an aperture according to one exemplary embodiment of the present disclosure. [Figure 36h] This figure shows an example of a region that can define an aperture according to one exemplary embodiment of the present disclosure. [Figure 37] This figure illustrates a post-processing step that may be performed after an image has been taken from an aerial survey, according to one exemplary embodiment of the present disclosure. [Figure 38a] These are a top view and a perspective view of a set of scanning patterns with sampled sensor pixels according to an exemplary embodiment of the present disclosure. [Figure 38b] These are top and perspective views of another set of scanning patterns with sampled sensor pixels, according to an exemplary embodiment of the present disclosure. [Figure 39a]These are top and perspective views of a set of scanning patterns with sensor pixels sampled at more scanning angles than Figure 38a, according to an exemplary embodiment of the present disclosure. [Figure 39b] Another top and perspective view of a set of scanning patterns with sensor pixels sampled at more scanning angles than Figure 38b, according to an exemplary embodiment of the present disclosure. [Figure 40] This figure shows various preferred investigation parameters for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 41] This figure shows various preferred investigation parameters for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 42a] This is a top view of a scanning pattern according to an exemplary embodiment of the present disclosure. [Figure 42b] This is a perspective view of a scanning pattern from Figure 42a according to an exemplary embodiment of the present disclosure. [Figure 42c] This is a top view of a scanning pattern according to an exemplary embodiment of the present disclosure. [Figure 42d] This is a perspective view of a scanning pattern from Figure 42c according to an exemplary embodiment of the present disclosure. [Figure 42e] This is a top view of a scanning pattern according to an exemplary embodiment of the present disclosure. [Figure 42f] This is a perspective view of a scanning pattern from Figure 42e according to an exemplary embodiment of the present disclosure. [Figure 43a] This figure shows the potential scanning mirror structure geometry for a vertically oriented sensor according to one exemplary embodiment of the present disclosure. [Figure 43b] This figure shows the potential scanning mirror structure geometry for a vertically oriented sensor, including one for over-rotation, according to one exemplary embodiment of the present disclosure. [Figure 43c] This figure shows the potential primary mirror geometry for a vertically oriented sensor according to an exemplary embodiment of the present disclosure. [Figure 43d]This figure shows the potential secondary mirror geometry for a vertically oriented sensor according to an exemplary embodiment of the present disclosure. [Figure 44a] This is a top view of a scanning pattern obtained using a scanning camera system comprising a vertically oriented sensor according to an exemplary embodiment of the present disclosure. [Figure 44b] This is a perspective view of a scanning pattern obtained using a scanning camera system comprising a vertically oriented sensor according to an exemplary embodiment of the present disclosure. [Figure 44c] This is a top view of multiple scanning patterns by realistic forward motion according to an exemplary embodiment of the present disclosure. [Figure 44d] This is a perspective view of multiple scanning patterns by realistic forward motion according to an exemplary embodiment of the present disclosure. [Figure 45a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 45b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 45c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 45d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 45e] This figure shows the potential primary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 45f] This figure shows the potential secondary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 46a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 46b] This is a perspective view of a scanning pattern to a scanning drive unit from Figure 46a, according to an exemplary embodiment of the present disclosure. [Figure 46c]This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 46d] Figure 46c shows a perspective view of a scanning pattern to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 46e] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 46f] Figure 46e shows a perspective view of a scanning pattern to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 47a] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 47b] This is a perspective view of a scanning pattern from Figure 47a according to an exemplary embodiment of the present disclosure. [Figure 47c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 47d] This is a perspective view of a scanning pattern from Figure 47c according to an exemplary embodiment of the present disclosure. [Figure 48a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 48b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 48c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 48d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 48e] This figure shows the potential primary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 48f] This figure shows the potential secondary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 49a]This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 49b] This is a perspective view of a scanning pattern to a scanning drive unit from Figure 49a, according to an exemplary embodiment of the present disclosure. [Figure 49c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 49d] Figure 49c shows a perspective view of a scanning pattern to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 49e] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 49f] Figure 49e shows a perspective view of a scanning pattern to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 50a] This figure shows a scanning camera system from a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 50b] This figure shows a scanning camera system from a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 50c] This figure shows a scanning camera system from a third viewpoint according to an exemplary embodiment of the present disclosure. [Figure 50d] This figure shows a scanning camera system from a fourth viewpoint according to an exemplary embodiment of the present disclosure. [Figure 50e] Figures 50a to 50d are top views of the scanning patterns for the scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 50f] Figures 50a to 50d show perspective views of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 51a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 51b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 51c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 51d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 51e] This figure shows the potential primary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 51f] This figure shows the potential secondary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 52a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 52b] This is a perspective view of a scanning pattern to a scanning drive unit from Figure 52a, according to an exemplary embodiment of the present disclosure. [Figure 52c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 52d] Figure 52c shows a perspective view of a scanning pattern relative to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 52e] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 52f] Figure 52e shows a perspective view of a scanning pattern to a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 53a] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 53b] This is a perspective view of a scanning pattern from Figure 53a according to an exemplary embodiment of the present disclosure. [Figure 53c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 53d] This is a perspective view of a scanning pattern from Figure 53c according to an exemplary embodiment of the present disclosure. [Figure 53e]This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 53f] This is a perspective view of a scanning pattern from Figure 53e according to an exemplary embodiment of the present disclosure. [Figure 54a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 54b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 54c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 54d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 54e] This figure shows the potential primary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 54f] This figure shows the potential secondary mirror geometry according to one exemplary embodiment of the present disclosure. [Figure 55a] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 55b] This is a perspective view of a scanning pattern from Figure 55a according to an exemplary embodiment of the present disclosure. [Figure 55c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 55d] This is a perspective view of a scanning pattern from Figure 55c according to an exemplary embodiment of the present disclosure. [Figure 55e] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 55f] This is a perspective view of a scanning pattern from Figure 55e according to an exemplary embodiment of the present disclosure. [Figure 56a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 56b] This is a perspective view of a scanning pattern from Figure 56a according to an exemplary embodiment of the present disclosure. [Figure 56c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 56d] This is a perspective view of a scanning pattern from Figure 56c according to an exemplary embodiment of the present disclosure. [Figure 56e] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 56f] This is a perspective view of a scanning pattern from Figure 56e according to an exemplary embodiment of the present disclosure. [Figure 57a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 57b] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 57c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 57d] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 57e] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 57f] This is a perspective view of a scanning pattern from Figure 57e according to an exemplary embodiment of the present disclosure. [Figure 58a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 58b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 58c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 58d]This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 58e] This figure shows the scanning mirror structure geometry according to one exemplary embodiment of the present disclosure. [Figure 58f] This figure shows the scanning mirror structure geometry, including that for over-rotation, according to one exemplary embodiment of the present disclosure. [Figure 59a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 59b] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 59c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 59d] This is a perspective view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 60a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 60b] This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 60c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 60d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 60e] This figure shows the scanning mirror structure geometry according to one exemplary embodiment of the present disclosure. [Figure 60f] This figure shows the scanning mirror structure geometry, including that for over-rotation, according to one exemplary embodiment of the present disclosure. [Figure 61a] This figure shows a scanning drive unit in a first viewpoint according to an exemplary embodiment of the present disclosure. [Figure 61b]This figure shows a scanning drive unit in a second viewpoint according to an exemplary embodiment of the present disclosure. [Figure 61c] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 61d] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 61e] This figure shows the scanning mirror structure geometry according to one exemplary embodiment of the present disclosure. [Figure 61f] This figure shows the scanning mirror structure geometry, including that for over-rotation, according to one exemplary embodiment of the present disclosure. [Figure 62a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 62b] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 62c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 62d] Figure 62c shows a perspective view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 62e] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 62f] Figure 62e shows a perspective view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 63a] This is a top view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 63b] This is a perspective view of a scanning pattern for a scanning drive unit according to an exemplary embodiment of the present disclosure. [Figure 63c] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 63d]Figure 63c shows a perspective view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 63e] This is a top view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Figure 63f] Figure 63e shows a perspective view of a scanning pattern for a scanning camera system according to an exemplary embodiment of the present disclosure. [Modes for carrying out the invention]

[0022] As used herein, singular descriptions are defined as one or more. As used herein, the word “plural” is defined as two or more. As used herein, the word “another” is defined as at least a second or subsequent. As used herein, the words “include” and / or “have” are defined as including (i.e., non-restrictive). Wherever “one embodiment,” “several embodiments,” “embodiment,” “one implementation,” “one example,” or similar phrases are used throughout this specification, this means that a particular feature, structure, or characteristic described in relation to an embodiment is included in at least one embodiment of this disclosure. Thus, the appearance of such phrases throughout this specification, or in various places, does not necessarily refer to all of the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in one or more embodiments in any preferred manner, without limitation.

[0023] A scanning camera system may include multiple cameras mounted inside or on a vehicle and a coupled beam steering mechanism. For example, a scanning camera system may be mounted in an inspection port of an aerial vehicle or in an external space such as a pod. For the sake of clarity and to facilitate the description of the various embodiments presented herein, an aerial vehicle is used, but it will be understood by those skilled in the art that the vehicle is not limited to an aerial vehicle.

[0024] The scanning camera system is controlled to capture a series of images of the area of ​​interest (typically the ground) as the aerial vehicle follows a path over the survey area. Each image captures a projected area on the area of ​​interest having an elevation angle (the angle of the central ray or "line of sight" of the image relative to the horizontal plane) and an azimuth angle (the angle of the central ray around the vertical axis with respect to a defined zero azimuth axis). The elevation angle may be expressed as an oblique angle (the angle of the central ray or "line of sight" of the image relative to the vertical axis), with a high elevation angle corresponding to a low oblique angle, and a 90-degree elevation angle corresponding to a 0-degree oblique angle. While this disclosure uses the ground as an exemplary area of ​​interest for the various embodiments described herein, it will be understood that in other embodiments the area of ​​interest may not be the ground. For example, it may consist of a building, bridge, wall, other infrastructure, vegetation, terrain such as cliffs, a body of water, or any other object imaged by the scanning camera system.

[0025] The calculation of the projection geometry on the target area from the camera can be performed based on the lens focal length, camera sensor size, camera placement and orientation, distance to the target area, and the geometry of the target area. The calculation can be refined based on nonlinear distortions in the imaging system, such as barrel distortion, atmospheric effects, and other corrections. Furthermore, if the scanning camera system includes beam steering elements such as mirrors, these must be taken into account in this calculation, for example, by modeling a virtual camera based on the beam steering elements used in place of the actual camera in the projection geometry calculation.

[0026] A scanning camera system may consist of one or more scanning drive units, each including scanning elements such as a scanning mirror for performing beam steering. The scanning mirror may be driven by, directly coupled to, or belt-driven any suitable rotary motor (such as a piezo rotary stage, stepping motor, DC motor, or brushless motor) coupled by a gearbox. Alternatively, the mirror may be coupled to a linear actuator or linear motor via gears. Each scanning drive unit includes a lens for focusing a light beam onto one or more camera sensors, the lens may be selected from the group including refractive lenses, reflective lenses, and reflective refractive lenses. Each scanning drive unit also includes one or more cameras configured to capture a series of images, or frames, of a region of interest. Each frame has an elevation and azimuth angle of view determined by the scanning drive unit geometry and scanning angle, and may be represented on the region of interest by projection geometry. Projection geometry is the region on the region of interest captured by the camera.

[0027] The projected geometry of a series of frames captured by a scanning drive unit may be combined to give a scanning pattern. Referring to the drawings, similar reference numbers indicate identical or corresponding parts across multiple figures. Figure 1a shows the scanning pattern for a scanning camera system 300 having three scanning drive units 301, 302, and 303 from a top view (left) and a perspective view (right) of the flying vehicle 110. Note that the scanning pattern in Figure 1a assumes that all frames are captured relative to the same position of the flying vehicle 110. In a real system, the flying vehicle 110 moves between frame captures, as will be discussed later. The x and y axes in the plot intersect at the ground position directly below the flying vehicle 110. Grid lines 117 and 118 correspond to the left-right distance of the flying vehicle 110, equal to its altitude. Similarly, grid lines 119 and 116 correspond to the forward and backward distances of the flying vehicle 110, equal to its altitude. The two curved scanning patterns 111 and 112 correspond to the two cameras of the scanning drive unit 301, while the two scanning patterns 113 and 114 are symmetrical with respect to the y-axis and correspond to the single cameras of the scanning drive unit 302 and the scanning drive unit 303, respectively. The dashed single projection geometry 115 corresponds to a low-resolution overview camera image.

[0028] The flying vehicle 110 may follow a meandering flight path as illustrated in Figure 2. This path consists of a series of straight flight lines 210, 211, 212, 213, 214, 215 along the flight direction (y-axis) connected by curved turning paths 220, 221, 222, 223, 224, 225. The meandering flight path is characterized by a flight line spacing 226, which is the distance between adjacent flight lines (210 to 211, 211 to 212, etc.) perpendicular to the flight direction (i.e., along the x-axis in Figure 2). Generally, the flight line spacing is fixed, but may be adaptive to capture several regions as the image density increases. Note that the combined width of the scanning pattern may be considerably wider than the flight line spacing.

[0029] Each scan pattern is repeated as the flying vehicle moves along the flight path over the survey area, resulting in dense coverage of the scene within the survey area with appropriate overlap of captured images for photogrammetry, photomosaic formation, and other applications. From end to end of the flight line, this can be achieved by keeping the scan angles of the frames within the scan pattern close enough. Along the flight line, this can be achieved by setting forward spacings between sufficiently small scan patterns (i.e., sets of frames captured as the scan angle changes). Timing constraints for each scan drive unit can be estimated based on the number of frames per scan pattern, the forward spacing, and the speed of the flying vehicle on the ground. These constraints may include a time budget per frame capture and a time budget per scan pattern.

[0030] Figure 1b shows the scanning pattern of the scanning camera system 300 from Figure 1a, along with additional scanning patterns for each scanning drive unit 301, 302, and 303 positioned one forward spacing unit in front of and behind the original target area geometry. In this configuration, the scanning angle step and forward spacing are selected to give a 10% overlap of the frame. In other configurations, the scanning angle step and forward spacing may be selected to give a fixed number of pixels of overlap within the frame, or overlap corresponding to a specified distance on the target area, or other criteria.

[0031] Generally, scanning camera systems have tighter timing constraints than fixed camera systems. However, scanning camera systems allow for wider flight line spacing for a given number of cameras, resulting in potentially more efficient camera systems. They also make more efficient use of the limited space (internal, such as survey ports, or external, such as pods) that can be mounted on commercially available flying vehicles.

[0032] The flight lines 210, 211, 212, 213, 214, and 215 of the meandering flight path shown in Figure 2 are marked by placements appropriately spaced forward for three scan drive units 301, 302, and 303. These can be thought to mark the position of the flight vehicle 110 on the meandering flight path, from which the initial frames of each scan pattern will be captured for each of the three scan drive units 301, 302, and 303. The forward spacing used for scan drive units 302 and 303 corresponding to scan patterns 113 and 114 in Figure 1a is approximately half the forward spacing used for scan drive unit 301 corresponding to the two curved scan patterns 111 and 112 in Figure 1a, for an equal percentage of forward overlap of the scan angle.

[0033] The flight line of a meandering path can take on any azimuth orientation. It may be preferable to align the flight line (y-axis in Figures 1a and 1b) to either the northeast or northwest direction. In this configuration, the scanning camera system 300 illustrated in Figures 1a and 1b has advantageous characteristics for capturing oblique images aligned to the basic directions (north, south, east, and west).

[0034] Figure 3 shows the distribution of views (elevation and azimuth angles) for nine different ground configurations of a scanning camera system 300, which has a scanning pattern as shown in Figure 1a and is flown on a more realistic meandering flight path (more and longer flight lines) compared to the exemplary survey flight path in Figure 2. Each plot is a Lambert azimuthal equal-area plot where the y-axis is parallel to the flight line. The point at coordinates x=0, y=0 corresponds to a view of the ground directly below the flying vehicle 110, where the oblique angle is zero.

[0035] The lines of sight circles at fixed elevation angles 236, 237, and 238 represent views with oblique angles of 12°, 39°, and 51°, respectively. The curved lines of sight paths in hemispheres 294, 295, 296, and 297 represent views with oblique angles between 39° and 51°, separated at an azimuth angle of 90°. The curved lines of sight paths in hemispheres 294, 295, 296, and 297 may represent preferred views of oblique images along the basic bearing when meandering flight follows a northeast or northwest flight line direction.

[0036] Each of the line-of-sight directions 230, 231, 232, 233, 234, and 235 corresponds to a pixel in the image captured by the scanning camera system 300 and represents the line-of-sight direction (elevation and azimuth) of the ground configuration at the time of image capture for the flying vehicle 110 on which the scanning camera system 300 is mounted. Neighboring pixels in the image correspond to neighboring ground configurations with similar line-of-sight directions. The line-of-sight directions 230, 231, 232, 233, 234, and 235 fall into either a horizontal band passing through the center or a circular band around an elevation of 45 degrees. The field-of-sight directions 230 and 235 within the horizontal band correspond to images captured by the cameras of the scanning drive unit 302 and scanning drive unit 303, while the line-of-sight directions 231, 232, 233, and 234 around the circular band correspond to images captured by the scanning drive unit 301. Some views are suitable for oblique images (e.g., viewing directions 231, 232, 233, 234), and some views are suitable for vertical images (e.g., viewing direction 235). Other views are suitable for other image products; for example, they may be useful for generating 3D texture models of regions.

[0037] The efficiency of aerial imaging is typically characterized by the area captured per unit time (e.g., square kilometers per hour). For meandering flight paths with long flight lines, empirically, this is proportional to the aircraft's speed and the flight line spacing, or the swath width of the survey. A more accurate estimate would take into account the time spent maneuvering between flight lines. Flying at high altitudes can increase efficiency because the flight line spacing is proportional to altitude and the speed also increases proportionally to altitude, but the image resolution will decrease unless the optical elements are modified to compensate (e.g., by increasing the focal length or decreasing the sensor's pixel pitch).

[0038] The data efficiency of a scanning camera system can be characterized by the amount of data captured per unit area during a survey (e.g., gigabytes (GB) per square kilometer (km)). Data efficiency increases as image overlap decreases and the number of views for each point on the ground decreases. Data efficiency determines the amount of data storage required in the scanning camera system for a given survey and also affects data processing costs. Generally, data efficiency is a less important factor in the economic evaluation of conducting a survey compared to imaging efficiency, because the costs of data storage and processing are generally lower than the costs of deploying an aerial vehicle equipped with a scanning camera system.

[0039] The maximum flight line spacing of a particular scanning camera system can be determined by analyzing the combined projection geometry (scanning pattern) of the captured images relative to the ground, along with the elevation and azimuth angles of those images, as well as image overlap requirements such as the requirements for the photogrammetry used to generate the image product.

[0040] To produce high-quality imaging products, it may be desirable to (1) image all points on the ground at a variety of elevation and azimuth angles, and (2) ensure some required level of image overlap on the target area (for example, for photogrammetry or photomosaic formation purposes).

[0041] The image quality of a set of images captured by a given scanning camera system operating at a defined flight line interval may depend on various factors, including image resolution and image sharpness.

[0042] The image resolution, or level of detail, captured by each camera is typically characterized by the ground sampling distance (GSD), i.e., the distance between adjacent pixel centers when projected onto the area of ​​interest (the ground) within the camera's field of view. Calculating the GSD for a given camera system is well understood and can be determined with respect to the focal length of the camera lens, the distance to the area of ​​interest along the line of sight, and the pixel pitch of the image sensor. The distance to the area of ​​interest is a function of the altitude of the aerial camera relative to the ground and the oblique angle of the line of sight.

[0043] Image clarity is determined by several factors, including the modulation transfer function (MTF) of the lens / sensor, the focal point of the image on the sensor plane, the surface quality of any reflective surface (mirror) (e.g., surface roughness and flatness), the stability of the optical elements of the camera system, the stabilization performance of the camera system or its components, the movement of the camera system relative to the ground, and the performance of any motion compensation unit.

[0044] The combined effect of various dynamic influences on image capture can be determined by tracking the image shift on the sensor during exposure time. This combined motion causes blurring in the image, reducing sharpness. This blurring can be expressed in terms of a decrease in MTF. Two important contributions to image shift are the linear motion (sometimes called forward motion) of the scanning camera system with respect to the area of ​​interest and the rotational speed of the scanning camera system (i.e., roll, pitch, and yaw rates). If the scanning camera system is mounted on a stabilization system or gimbal, the rotational speed of the scanning camera system does not have to be the same as the rotational speed of the flying vehicle.

[0045] Images captured by scanning camera systems can be used to create many useful image-based products, including photomosaics (including orthomosaics and panoramas), oblique images, 3D models (with or without textures), and raw image viewing tools.

[0046] In addition to resolution and sharpness, the image quality of the captured images used to produce these products may depend on other factors, including overlap of projected images, the distribution of views (elevation and azimuth angles) above ground points captured by the camera system during the survey, and differences in the appearance of areas due to differences in time and view during image acquisition (moving objects, altered lighting conditions, altered atmospheric conditions, etc.).

[0047] Overlap of projected images is a critical parameter when generating photomosaics. The use of low-resolution overview cameras is known to improve system efficiency by reducing the necessary overlap between high-resolution images required for accurate photogrammetry. This, in turn, improves data efficiency and increases the time budget for image acquisition.

[0048] The image quality of an image set for vertical images depends on the oblique angle statistics of the captured images above the ground point. Any deviation from zero oblique angle results in the vertical walls of a building being captured, causing the building to appear tilted in the vertical image. Maximum oblique angle is the maximum deviation from the vertical in the image and is an important metric indicating the image quality of vertical images. Maximum oblique angle can vary between 10° for high-quality surveys and 25° for low-quality surveys. Maximum oblique angle is a function of the flight line spacing of the scanning drive unit and the area projection geometry of the captured image (or scanning pattern).

[0049] Orthomosaics blend image pixels from captured images in a way that minimizes the bevel angle of the pixels used while simultaneously minimizing adjacent artifacts from pixel values ​​from different original captured images. Therefore, the maximum bevel angle parameter described above is a crucial parameter for orthomosaic generation; the larger the maximum bevel angle, the more the building will appear tilted. The quality of the orthomosaic also depends on the overlap of adjacent images captured in the survey. The greater the overlap, the more carefully the seams between pixels taken from adjacent images can be placed where there is less texture or where the 3D geometry of the images is suitable for blending the images with minimal visual artifacts. Furthermore, differences in the appearance of areas between composite image pixels result in increased artifacts at the seams, affecting the quality of the generated orthomosaic.

[0050] The image quality of oblique image products can be understood along similar lines to that of vertical images and orthomosaics. Some oblique image products are based on a specific viewpoint, such as 45-degree elevation images where the azimuth angle is aligned to a particular direction (e.g., one of the four basic cardinal directions: north, south, east, or west). The captured image may differ from the desired viewpoint in both elevation and azimuth angles. Depending on the image product, the degradation of image quality due to elevation and azimuth errors will vary. Oblique image products (sometimes referred to as panoramas) that are blended or stitched together can also be produced. The image quality of such products also depends on the angle error of the view and the overlap between image views, in a manner similar to the description for orthomosaic images above.

[0051] The image quality of the set of images used to generate a 3D model depends primarily on the distribution of views (elevation and azimuth angles) above the ground point. Generally, both reducing the spacing between views and increasing the number of views have been observed to improve the expected quality of the 3D model. Heuristics for expected 3D quality can be generated based on such observations and used as a guide for the design of scanning camera systems.

[0052] Figures 4a–4f, 5a–5f, and 6a–6f show scanning drive units 301, 302, and 303 that may be used to achieve the scanning pattern of Figure 1a. The first scanning drive unit 301, shown in Figures 4a and 4b, may be used to capture scanning patterns 111, 112 having an arc centered at an elevation angle of 45°. Top and perspective views of scanning patterns 111, 112 from the two cameras 310, 311 of the scanning drive unit 301 are shown in Figures 4c and 4d, respectively.

[0053] Two geometric diagrams of the scanning drive unit 301 from different viewpoints are shown in Figures 4a and 4b. The scanning drive unit 301 has a vertical scanning axis (elevation angle θ). S = -90° and azimuth angle φ SIt includes a scanning mirror structure 312 attached to a scanning drive device 313 at (elevation angle θ = 0°). In one embodiment, the scanning mirror structure 312 has two sides. The geometric illustration shows that the first mirror surface 314 is oriented in the normal direction facing towards the first camera 310 along the y-axis (elevation angle θ 1 M = 0° and azimuth angle φ 1 M = 0°), indicating a configuration where the scanning angle of the scanning drive device 313 is set to 0°. The second mirror surface 315 is mounted on the opposite side of the scanning mirror structure 312 and is oriented towards the second camera 311. The two cameras 310, 311 are angled downwards and have opposing azimuth angles (elevation angle θ of camera 310 S = -45° and azimuth angle φ S = 180°, elevation angle θ of camera 311 S = -45° and azimuth angle φ S = 0°).

[0054] In one example, cameras 310, 311 utilize Gpixel GMAX3265 sensors (9344×7000 pixels with a pixel pitch of 3.2 microns). The camera lenses can have a focal length of 420 mm and an aperture of 120 mm (corresponding to F3.5). The scanning mirror structure 312 can have a thickness of 25 mm. Unless otherwise specified, all of the exemplified cameras utilize Gpixel GMAX3265 sensors with lenses having a focal length of 420 mm and an aperture of 120 mm (F3.5), and all of the exemplified mirrors have a thickness of 25 mm.

[0055] The optical axis of a lens is generally defined as the axis of symmetry of the lens. For example, it can be defined by the rays passing through the lens element at or near the center of the sensor, or at or near the center of the lens element. The optical axis of the lens in the scanning drive unit can be modified by one or more mirror structures of the scanning drive unit. These may extend beyond the lens, reflect off one or more mirror surfaces, and continue to a point on the area of ​​interest. The distance from camera 310 to mirror surface 314 along the optical axis may be 247 mm. The distance from the second camera 311 to the second mirror surface 315 along the optical axis may also be 247 mm. In other embodiments, the distances between elements may be chosen so that the components fit within the required space and the scanning drive unit 301 can rotate within the required angular range (which may be between ±30.7° and ±46.2° for the two-plane configuration described here). The axis of rotation of the scanning mirror structure 312 is assumed to intersect with the optical axes of one or both cameras 310, 311. The distances between the components of all scanning drive unit components presented herein may be selected to best fit within the available space while allowing the required rotational angle range of the scanning mirror structure.

[0056] The shape of the reflective surface of the scanning mirror structure should be large enough to reflect the entire beam of light captured from the ground region onto the camera lens aperture, which is focused onto the camera sensor when the scanning angle of the scanning drive unit changes over a given range of scanning angles. In one embodiment of the scanning mirror structure 312, the standard range of scanning angles is from -30.7° to 30.7°. Existing methods that can be used to calculate a suitable scanning mirror structure shape that satisfies this criterion are described elsewhere.

[0057] A preferred method is to determine the geometry of the region on the surface of the scanning mirror structure that intersects the beam profile defined by the light rays passing between the target area and the camera sensor through the lens aperture at each sampled scanning angle. The beam profile can vary from a circle at the camera aperture to a rectangular shape corresponding to the sensor shape at the focal length. The sum of the geometries of these intersecting regions on the mirror surface gives the size of the scanning mirror structure required to handle the sampled set of scanning angles. In some cases, the calculated scanning mirror structure shape may be asymmetrical with respect to the axis of rotation, and therefore it may be possible to reduce the moment of inertia of the scanning mirror structure by shifting the axis of rotation. In this case, the scanning mirror structure geometry can be recalculated for the shifted axis of rotation. The recalculated shape may still be asymmetrical around the axis of rotation, in which case the process of shifting the axis of rotation and recalculating the geometry may be repeated until the scanning mirror structure is sufficiently close to symmetrical and the moment of inertia is minimized.

[0058] The method described above generates the geometry of the scanning mirror structure required for a specific sensor orientation within the camera. The sensors of the scanning drive unit 301, 302, and 303 shown in Figures 4a-4f, 5a-5f, and 6a-6f are oriented in an orientation that may be referred to as a horizontal orientation. When viewed from above, the projected geometry of the image captured at the position closest to the y-axis has horizontal geometry (the width along the x-axis is greater than the length along the y-axis). An alternative embodiment may use a sensor oriented at 90° to the orientations illustrated in Figures 4a-4f, 5a-5f, and 6a-6f, which may be referred to as a vertical orientation. When viewed from above, the projected geometry of the image captured at the position closest to the y-axis will have vertical geometry (the width along the x-axis is narrower than the length along the y-axis). Other embodiments may use any orientation between horizontal and vertical orientations.

[0059] It may be advantageous to use a scanning mirror structure geometry that is large enough to handle the vertical orientation of the sensor in addition to the horizontal orientation. Such a scanning mirror structure geometry can be generated as a merger of the mirror geometries for the horizontal and vertical orientations. Such a scanning mirror structure geometry may allow for greater flexibility in configurations using a scanning drive. Furthermore, it may be advantageous to use a scanning mirror structure geometry that can handle any orientation of the sensor by considering angles other than the horizontal and vertical orientations. Such a scanning mirror structure can be calculated assuming a circular sensor with a diameter equal to the length of the sensor's diagonal.

[0060] The scanning mirror structure may include aluminum, beryllium, silicon carbide, quartz glass, or other materials. The scanning mirror structure may include a hollow cavity to reduce mass and moment of inertia, or it may be solid (without a hollow cavity) depending on the material of the scanning mirror structure. The mirror surface may be coated using, for example, nickel, quartz glass, or other materials to improve reflectivity and / or flatness. The coating may be on both sides of the scanning mirror structure to reduce thermal effects when the temperature of the scanning mirror structure changes. The required flatness of the mirror surface may be set according to the required sharpness of the captured image and the acceptable loss of sharpness due to mirror reflection. The mirror surface may be polished to achieve the required flatness specification.

[0061] The thickness of the scanning mirror structure is generally set to be as small as possible to reduce mass and minimize spatial requirements, while maintaining the structural integrity of the scanning mirror structure, thereby allowing the scanning pattern to be dynamically rotated within the time budget of the captured image without compromising the optical quality of the captured image. In one embodiment, a thickness of 25 mm may be preferred.

[0062] Depending on the manufacturing process and materials used for the scanning mirror structure, it may be advantageous to use a convex mirror shape. In this case, the convex hull of the shape calculated above can be used as the scanning mirror structure shape. Furthermore, the scanning mirror structure shape may be extended to ensure that the manufacturing tolerances of the scanning mirror structure and other components of the scanning drive unit, or the control tolerances in setting the scanning angle, do not result in stray or scattered light in the system and the resulting loss of visual quality.

[0063] Figure 4e shows various scanning mirror structure geometries calculated for the scanning drive unit 301. These include the minimum geometry ("min"), the dilate minimum geometry ("dilate") which is extended by 5 mm beyond the minimum geometry at the periphery, and the convex geometry ("convex") which is the convex hull of the dilate minimum geometry. Any of these geometries, or other possible variations (e.g., for handling alternative sensor orientations), can be used to define the shape of the scanning mirror structure 312 for this scanning drive unit 301.

[0064] The rotation axis 316 was selected to intersect with the light rays along the optical axis of the lens passing through the center of the aperture. The scanning drive unit is mounted on the end extending beyond the scanning mirror structure 312. The center of mass of the scanning mirror structure 312 is aligned with the rotation axis 316, and therefore no shift of the rotation axis is required.

[0065] Figure 4f again shows the extended convex geometry ("convex"), and also the extended geometry that may be required when the scanning angle range is extended by 7.5° at each end of the scanning angle range ("over"). The angular spacing of the scanning angle samples is kept approximately the same as the original in the calculation by increasing the number of sample steps. This geometry will be further discussed later herein in relation to over-rotation for yaw correction.

[0066] Figure 4g is a close-up view of additional geometry of the mirror and / or paddle flap according to an embodiment. For example, as can be seen in Figure 4g, the paddle flap (diagonal line region) can cover the entire perimeter of the mirror, or one or more portions thereof. The mirror and / or paddle flap can be symmetrical or asymmetrical.

[0067] The capture of images on opposing mirror surfaces (e.g., mirror surfaces 314, 315) may or may not be synchronized. Generally, image capture is performed when the scanning mirror structure is completely stationary to achieve high image quality. In other configurations, image stabilization may be used to compensate for mirror movement during image exposure.

[0068] In a slightly modified configuration, the scanning mirror structure 312 may employ a single mirror surface (i.e., one of the mirror surfaces 314 or 315), and the scanning mirror structure 312 rotates 360° using the scanning drive unit 313, so that the single mirror surface can be used sequentially by the two cameras 311, 310. For example, in the modified configuration, the second mirror surface 315 does not need to be a mirror surface. This multiplexed configuration tightens the requirements for image acquisition timing because images are not captured simultaneously for both mirror surfaces 314, 315.

[0069] The second scanning drive unit 302 of the scanning camera system 300 is shown in Figures 5a to 5f. As shown in Figures 5c and 5d, the scanning drive unit 302 can be used to capture a single linear scanning pattern 113 perpendicular to the flight line at an oblique angle from 0° to 45°. The scanning pattern 113 extends to the right side of the flying vehicle 110 looking forward along the flight line. Two geometric illustrations of the scanning drive unit 302 from different viewpoints are shown in Figures 5a and 5b. The scanning drive unit 302 has a horizontal scanning axis (elevation angle θ S = -0° and azimuth angle φ SThe system comprises a single-sided scanning primary mirror 323 held on an elevation angle of 180° and a fixed secondary mirror 324. The geometric diagram shows a configuration in which the scanning angle of the scanning drive unit 322 is set to 0°, and at this angle the surface of the primary mirror 323 is oriented by a normal vector directed diagonally between the z-axis and the x-axis (elevation angle θ). 1 M = -45° and azimuth angle φ 1 M (=90°). The secondary mirror 324 is oriented in the direction of the normal opposite to the normal of the primary mirror 323 when the scanning angle is 0° (elevation angle θ). 1 M =45° and azimuth angle φ 1 M =-90°). Vertical z-axis (elevation angle θ S = -89° and azimuth angle φ S There is a single camera 321 oriented downward at an angle of 1 degree relative to (=-90°). A scanning drive unit 322 samples scanning angles from -23° to -0.5° to generate a scanning pattern 113.

[0070] In one embodiment, the distance from the lens of the camera 321 to the secondary mirror 324 along the optical axis may be 116 mm, and the distance from the primary mirror 323 to the secondary mirror 324 along the optical axis may be 288 mm. Of course, other distances may be used in other embodiments.

[0071] There are two mirror geometries to consider for the scanning drive unit 302. The exemplary geometry of the (scanning) primary mirror 323, including the minimum geometry ("min"), dilate geometry ("dilate"), and convex geometry ("convex"), which is essentially the same as the dilate geometry. Since the calculated center of gravity of the primary mirror was found to be shifted with respect to the scanning drive axis projected onto the mirror surface, Figure 5e shows the shifted scanning drive axis which can be used to reduce the moment of inertia as described above. The exemplary geometry of the (fixed) secondary mirror 324, including the minimum geometry ("min") and dilate geometry ("dilate"), is shown in Figure 5f.

[0072] The third scanning drive unit 303, illustrated in Figures 6a and 6b, is a clone of the second scanning drive unit 302 rotated 180° around the z-axis. Figures 6a and 6b include a camera 325, a primary mirror 327, a scanning drive unit 326, and a secondary mirror 328. As shown in Figures 6c and 6d, due to the symmetry of the scanning drive units 302 and 303, the scanning pattern 114 of the scanning drive unit 303 is a mirror image of the scanning pattern 113 relative to the scanning drive unit 302, following a straight path extending to the left of the flying vehicle 110 looking forward along the flight line. The mirror geometry and dynamics shown in Figures 6e and 6f are identical to those described above with reference to Figures 5e and 5f.

[0073] Figures 7a to 7d are a series of perspective views of the combined components of the scanning drive units 301, 302, and 303 of the scanning camera system 300 described with reference to Figures 4a to 4f, 5a to 5f, and 6a to 6f above, including cameras 310, 311, 321, and 325, a scanning mirror structure 312 having mirror surfaces 314 and 315 attached to the scanning drive unit 313, two primary mirrors 323 and 327 attached to the scanning drive units 322 and 326, and two fixed secondary mirrors 324 and 328.

[0074] Figures 7a to 7d show that the structure of the scanning drive unit 302 is configured such that its imaging path passes under the camera 310 of the scanning drive unit 301, and the imaging path of the scanning drive unit 303 passes under the camera 311 of the scanning drive unit 301. This configuration is spatially very efficient and advantageous for deployment in a wide range of flying vehicle camera (surveillance) holes.

[0075] Figures 7e and 7f show scanning patterns achieved using the scanning camera system 300, including curved scanning patterns 111, 112 for oblique images and linear scanning patterns 113, 114 for capturing vertical-to-oblique image sweeps along a direction perpendicular to the flight line. In addition to the imaging capabilities of the scanning drive unit, the scanning camera system 300 may also include one or more fixed cameras. These cameras may be standard RGB cameras, infrared cameras, grayscale cameras, multispectral cameras, hyperspectral cameras, or other suitable cameras. In one embodiment, the fixed camera may be a Phase One iXM100 camera sensor (11664 × 8750 pixels, 3.76 micron pitch) with an 80mm F5.6 lens. Single-point or multi-point LIDAR camera systems may also be incorporated into the scanning camera system.

[0076] A fixed camera may be used as an overview camera, and the shooting rate of the fixed camera may be set to achieve a desired forward overlap between captured images, such as 60%. The flight line spacing of the survey may be limited to achieve a second desired target, such as 40% lateral overlap of the overview camera images. The overview camera may be pointed vertically downward and rotated around a vertical axis so that the projected geometry on the area of ​​interest is not aligned with the orientation of the flying vehicle.

[0077] The scanning patterns 111, 112, 113, and 114 of the scanning camera system 300 described above with respect to Figures 1a, 4c, 4d, 5c, 5d, 6c, 6d, 7e, and 7f do not represent the forward motion of the flying vehicle 110, and they were generated assuming a stationary flying vehicle 110 above the target area. The ground projection geometry of the scanning patterns can be replotted to include the linear motion of the flying vehicle 110 above the ground to produce slightly modified scanning pattern plots in Figure 8a (for a single scanning pattern) and Figure 8b (for three scanning patterns). These scanning patterns give a more realistic view of the scanning patterns that can be used to calculate flight parameters for achieving overlap targets (e.g., 10% overlap). Note that these do not affect the field of view direction (elevation and azimuth angles) of the captured images, as the field of view angles are calculated as a function of the difference in the placement of the captured ground points with respect to the placement of the flying vehicle 110 at the time of image acquisition. Figure 8c is a top view and perspective view of multiple sets of scanning patterns captured by a scanning camera system according to an exemplary embodiment of the present disclosure. The scanning camera system in Figure 8c is a reduced system including only scanning drive unit 301 and scanning drive unit 302 without camera 311. This scanning camera system can be flown on a modified flight path in which each flight line 210 to 215 is flown in both directions.

[0078] It is understood that the geometry of the scanning camera system 300 can be modified in many ways without changing the essential function of each of the scanning drive units 301, 302, and 303. For example, the arrangement and thickness of the scanning drive and mirrors, the distance between elements, and the mirror geometry can be changed. In general, it is preferable to bring the mirrors as close to the lens as possible without resulting in mechanical obstructions that hinder the desired scanning angle range with respect to operation, or optical obstructions that result in a degradation of image quality.

[0079] Furthermore, changes may be made to the focal length of individual lenses or to the type and geometry of the sensors. In addition to corresponding geometric changes to the geometry and arrangement of the mirrors, these changes may result in changes to the appropriate flight line distance, the step between scan angles, the scan angle range, and the frame timing budget for the system.

[0080] The scanning camera system can be operated during the survey by the system control unit 405. A preferred high-level representation of the system control unit 405 is shown in Figure 9. Components enclosed in dashed boxes (e.g., the autopilot 401, the motion compensation (MC) unit 415) represent units that may be omitted in other embodiments. The system control unit 405 may interface with the scanning camera system 408, the stabilization platform 407, the data storage device 406, the GNSS receiver 404, the autopilot 401, the pilot display 402, and the pilot input 403. The system control unit 405 may include one or more distributed computing devices, such as a computer, laptop computer, microcontroller, ASICS, or FPGA, and may control the scanning drive unit and fixed cameras of the camera system during operation. The system control unit 405 may also assist the pilot or autopilot of the flying vehicle in following a preferred flight path over the ground area of ​​interest, such as the meandering flight path described with respect to Figure 2. The system control unit 405 may be centrally localized or distributed around the components of the scanning camera system 408. The system control unit 405 may use Ethernet, serial, CoaxPress (CXP), CAN Bus, i2C, SPI, GPIO, a custom internal interface, or any other interface suitable to achieve the required data rate and latency of the system.

[0081] The system control unit 405 may have one or more interfaces to a data storage device 406, which can store data relating to the survey flight path, scan drive geometry, scan drive unit parameters (e.g., scan angle), digital elevation model (DEM), global navigation satellite system (GNSS) measurements, inertial measurement unit (IMU) measurements, stabilization platform measurements, other sensor data (e.g., thermal, pressure), motion compensation data, mirror control data, focus data, captured image data, and timing / synchronization data. The data storage device 406 may also include multiple direct interfaces to individual sensors, control units, and components of the scanning camera system 408.

[0082] The scanning camera system 408 may include one or more scanning drive units 411, 412, an IMU 409, and a fixed camera 410. The IMU 409 may comprise one or more separate units having different performance metrics such as range, resolution, accuracy, bandwidth, noise, and sample rate. For example, the IMU 409 may comprise a KVH1775IMU supporting sample rates up to 5 kHz. IMU data from the separate units may be used individually or fused for use elsewhere in the system. In one embodiment, the fixed camera 410 may include a Phase One iXM100, Phase One iXMRS100M, Phase One iXMRS150M, AMS Cmosis CMV50000, Gpixel GMAX3265, or IOIndustries Flare 48M30-CX, and may use a suitable camera lens having a focal length between 50 mm and 200 mm.

[0083] The system control unit 405 may use data from one or more GNSS receivers 404 to monitor the position and speed of the flying vehicle 110 in real time. The one or more GNSS receivers 404 may be compatible with a variety of space-based satellite navigation systems, including the Global Positioning System (GPS), GLONASS, Galileo, and BeiDou.

[0084] The scanning camera system 408 may be mounted on a stabilization platform 407 which can be used to isolate the scanning camera system 408 from disturbances affecting the flight vehicle 110, such as attitude (roll, pitch, and / or yaw) and attitude rates (roll rate, pitch rate, and yaw rate). Active and / or passive stabilization methods may be used to achieve this. Ideally, the scanning camera system 408 is designed to be as well balanced as possible within the stabilization platform 407. In one embodiment, the stabilization platform 407 includes roll rings and pitch rings so that the scanning camera system 408 is isolated from disturbances of roll, pitch, roll rate, and pitch rate.

[0085] In some embodiments, the system control unit 405 may further control image acquisition and analysis with the aim of setting the correct focus of the camera lenses of the scanning drive unit 411, 412 and / or fixed camera 410. The system control unit 405 may set focus on multiple cameras based on images from other cameras. In other embodiments, focus may be controlled through thermal stabilization of the lens or based on known lens characteristics and the estimated optical path from the camera to the ground. Some cameras in the scanning camera system 408 may be fixed focus. For example, some of the fixed-focus cameras used for overview images may be fixed focus.

[0086] Each scanning camera system is associated with several scanning drive units. For example, scanning camera system 408 includes scanning drive units 411 and 412, but may include more. As another example, scanning camera system 300, shown in Figures 7a to 7d, comprises three scanning drive units 301, 302, and 303 as described above with respect to Figures 4a to 4f, 5a to 5f, and 6a to 6f. Alternative configurations of scanning camera systems with different numbers of scanning drive units will be discussed later. Each scanning drive unit 411 and 412 shown in Figure 9 may comprise a scanning mirror 413 and one or more cameras 414 and 416.

[0087] Each camera 414, 416 in Figure 9 may include a lens, a sensor, and optionally motion compensation units 415, 417. The lenses and sensors of cameras 414, 416 may be matched so that the field of view of the lens exposes the required area of ​​the sensor with a certain acceptable level of uniformity.

[0088] Each lens may incorporate a focusing mechanism and sensors for monitoring its environment and performance. It may feature a number of high-quality lens elements that are thermally stabilized and have anti-reflective coatings to achieve sharp imaging free from ghosting due to internal reflections. The system control unit 405 may perform focusing operations based on focus data 438 during image acquisition. This may utilize known techniques for autofocus based on sensor inputs such as images (e.g., image textures), LiDAR, digital elevation models (DEMs), temperature data, or other inputs.

[0089] The control of the scanning mirror 413 and the acquisition of images by one or more cameras 414, 416 of the scanning drive unit 411 are illustrated in the high-level process of Figure 10. The system control unit 405 uses data input from the data storage device 406 to iteratively set the scanning angle 430 and trigger one or more cameras 414, 416 to acquire images. The scanning angle 430 is set according to the scanning drive unit parameter 434, which defines the sequence of scanning drive angles corresponding to the sequence of images acquired for each scanning pattern, and the sequential timing of the frames of the scanning pattern. As described above, the sequence of scanning angles and the timing of frame acquisition may be set to achieve a desired overlap of the projection geometry of ground-captured images that is advantageous for a particular aerial imaging product.

[0090] Optionally, the sequence of scan angle 430 settings may be updated according to IMU data such as the attitude of the flight vehicle relative to the expected attitude (aligned to the flight line). For example, the scan angle 430 may be corrected to account for the yaw of the flight vehicle when the stabilization platform 407 does not handle yaw. In particular, for the scan drive unit 301 described with respect to Figures 4a-4f, which captures two arc-shaped scan patterns 111, 112, a scan angle correction of half the yaw angle may be used so that the scan patterns are corrected for yaw, as will be described in more detail later with respect to Figures 32-37. Alternatively, a smaller scan angle correction may be used when the stabilization platform 407 has only partial yaw correction.

[0091] The mirror control device 432 receives a command from the system control device 405 to set the scanning drive to a scanning angle 430, and optionally uses input from the mirror sensor 433, which reports the status of the mirror drive 431, to control the mirror drive 431 so that the scanning mirror 413 is set to a desired scanning angle 430. The mirror control device 432 transmits mirror control data 437, which is stored in the data storage device 406. When the scanning mirror 413 is set to the correct scanning angle according to the mirror control data 437, the system control device 405 may send a trigger command to one or more cameras 414, 416 associated with the scanning mirror 413.

[0092] Optionally, the system control unit 405 also controls the timing of the camera triggers to synchronize with the motion compensation operation of each camera 414, 416. Motion compensation (MC) data 435 related to motion compensation for cameras 414, 416 is stored in the data storage device 406 and can be used to achieve this synchronization.

[0093] Pixel data 439 corresponding to captured images is stored in data storage device 406. Optionally, the gimbal angle 470 may be stored in data storage device 406, which also contains information relating to the orientation of the scanning camera system 408 on the stabilization platform 407 (i.e., gimbal) at the time of image capture relative to the stored pixel data 439. Other data logged in sync with image capture may include GNSS data (ground velocity 462, latitude / longitude data 463, and altitude 464, as shown in Figure 11) and IMU attitude data 436.

[0094] It can be understood that the process illustrated in Figure 10 may be employed to capture motion-compensated images using projection geometry according to the scanning pattern of a scanning drive unit. This process may be slightly modified without affecting the scope of the systems and methods described herein.

[0095] Motion compensation can employ a variety of methods, including, but are not limited to, tilting or rotating transparent optical plates or lens elements in the optical path, tilting or rotating mirrors in the optical path, and / or translating the camera sensor. The dynamics of the motion compensation method may be synchronized with image capture so that undesirable image motion is minimized during exposure and the sharpness of the output image is maximized. It should be noted that motion compensation shifts the image on the sensor, affecting the camera's principal point, and may need to be considered in image processing, such as bundle adjustment and calibration.

[0096] A preferred process for the motion compensation unit 415 of the camera 414 is illustrated in the high-level process shown in Figure 11. The system control unit 405 transmits signals to control the operation of the motion compensation unit 415, synchronizes with the control unit of the scanning mirror 413, and triggers the camera 414 to capture a motion-compensated image with the desired projection geometry.

[0097] The motion compensation unit 415 uses the geometry estimator module 450 to determine the projected geometry 451 of the camera 414 of the scanning drive unit 411 in the current configuration, which is a function of the scanning angle. The projected geometry 451 is a mapping between the pixel arrangement in the sensor and the coordinates of the imaged arrangement on the ground. The coordinates on the area of ​​interest may be, for example, the x and y axes in the illustrated diagrams of the various scanning patterns shown in Figures 4a and 4b. The projected geometry 451 may be expressed in terms of the projected geometry when the ground is represented as a flat plane, or other representations may be used to deal with more general non-flat areas of interest.

[0098] The geometry estimator module 450 may calculate projected geometry 451 based on known scanning angles 430 reported in mirror control data 437, known scanning drive unit (SDU) geometry data 467, IMU attitude data 466 reporting the orientation of the scanning drive unit, and flight vehicle altitude data 464. Optionally, the geometry estimator module 450 may use local ground height profile data and flight vehicle latitude / longitude data 463 from a digital elevation model (DEM) 465 to form a more accurate projected geometry. The geometry estimator module 450 may operate at a fixed speed or, for example, at a specific time, based on the settling of the scanning mirror 413 provided through the mirror control data 437.

[0099] The projected geometry 451 can be used in combination with various motion sensor measurements to estimate the pixel velocity estimate. The pixel velocity estimate is an estimate of the motion of the focused image on the camera sensor during exposure. Two different pixel velocity estimators relating to the linear and angular motion of a flying vehicle are described herein. These are referred to as the forward motion pixel velocity estimator 452 and the attitude rate pixel velocity estimator 454, respectively.

[0100] The forward motion pixel velocity estimator 452 uses the projected geometry 451, in addition to the current ground velocity 462 of the flying vehicle generated by the GNSS receiver 404, to calculate the forward motion pixel velocity 453 corresponding to the linear motion of the scanning camera system 408 when the camera is exposed. The pixel velocity is expressed as the average velocity of the ground image above the camera sensor and may include a pair of velocities (e.g., expressed in pixels per millisecond) corresponding to the velocity of the ground image motion along the two axes of the sensor. Alternatively, it may include azimuth (e.g., in degrees or radians) and magnitude of motion (e.g., in pixels per millisecond), or other preferred vector representations.

[0101] The forward motion pixel velocity estimator 452 may calculate the forward motion pixel velocity 453 by mapping a ground arrangement corresponding to a set of points crossing the sensor based on the projection geometry, shifting those points according to the motion of the flying vehicle over a short time step (e.g., 1 ms or a value related to the camera's exposure time), and then projecting them back onto the sensor. The shift of each sensor arrangement from its original arrangement due to the motion of the flying vehicle may be divided by the time step to estimate the local vector velocity at the sensor arrangement. The pixel velocity of the image may be calculated by statistically combining (e.g., averaging) the local vector velocities across the sampled set of sensor arrangements.

[0102] The forward motion pixel velocity estimator 452 can operate at a fixed update rate, or it can operate to update when there are changes in the input data (ground velocity 462 and projected geometry 451), or based on some other appropriate criteria.

[0103] The attitude rate pixel velocity estimator 454 uses the projection geometry 451 in addition to the IMU attitude rate 468 generated by the IMU 409 to calculate the attitude rate pixel velocity 455 corresponding to the rate of change in attitude (e.g., yaw rate) of the scanning camera system 408 during camera exposure. The attitude rate pixel velocity 455 may be expressed in the same vector form as the forward motion pixel velocity 453. The attitude rate pixel velocity estimator 454 may use an estimation approach based on similar short time steps to determine the attitude rate pixel velocity 455. Pixel positions on the sensor may be mapped to positions on the ground through the projection geometry 451. A second projection geometry is then generated based on the projection geometry 451 rotated according to the change in attitude of the scanning camera system that would occur during a short time step due to the current attitude rate. Positions on the ground are remapped to sensor coordinates based on the second projection geometry. The attitude rate pixel velocity 455 may be estimated as the change in sensor position relative to the original position divided by the time step.

[0104] The attitude rate pixel velocity estimator 454 module may operate at a fixed update rate, or it may operate to update when there are changes in the input data (IMU attitude rate 468 and projected geometry 451), or based on some other appropriate criteria. The IMU attitude rate 468 has high-frequency components, and the attitude rate pixel velocity 455 may change over short periods of time.

[0105] Transmitting multiple updated attitude rate pixel velocity estimates to the motion compensation control unit 458 corresponding to a single image acquisition may be advantageous in terms of the dynamic requirements of the motion compensation drive unit 460. This is represented in the process flow by an additional ROI pixel velocity estimator 440. It may also be advantageous to use some kind of forward predictive estimator on the IMU data to reduce the difference between the actual attitude rate at measurement and at camera exposure. Preferred forward predictive methods may include a variety of known filters, such as linear filters, Kalman filters, and statistical methods such as least squares estimation. The forward predictive method may be tailored based on attitude rate data previously sampled from similar aircraft with similar stabilization platforms and camera systems.

[0106] In one embodiment, the scanning camera system 408 may be isolated from the roll rate and pitch rate by a stabilization platform 407, and the attitude rate pixel velocity 455 may be calculated based solely on the yaw rate of the flying vehicle. In other embodiments, the scanning camera system 408 may be isolated from roll, pitch, and yaw, and the attitude rate pixel velocity 455 may be assumed to be negligible.

[0107] In addition to motion sensor pixel velocity estimators such as the forward motion pixel velocity estimator 452 and the attitude rate pixel velocity estimator 454, direct measurements of pixel velocity may be calculated based on captured images. To reduce latency between image acquisition and generation of pixel velocity estimates, it may be advantageous to perform this analysis on a small region of interest (ROI) image 469, preferably taken in the texture region of the area. The ROI image 469 should be captured without motion compensation and may use a short exposure time relative to normal image frame acquisition, preferably after the mirror has been settled. Vector pixel shift can be estimated between ROI images captured at slightly different times using any suitable image alignment method (e.g., correlation-based methods in the Fourier domain or real space, gradient-based shift estimation methods, or other techniques). The vector pixel shift estimate can be converted to pixel velocity by dividing the shift by the time step between the ROI image acquisition times.

[0108] The ROI pixel velocity estimator 440 may combine pixel velocity estimates from more than two ROI images to improve accuracy, and this may operate at a fixed velocity or when ROI images are available. The estimated ROI pixel velocity 457 may be rejected if certain criteria are not met, for example, if the image does not have sufficient texture. The arrangement of captured images may be set to improve the likelihood of finding good texture in the captured area, for example, based on the analysis of other images captured by the scanning camera system or based on previous investigations of the same area.

[0109] The motion compensation process illustrated in FIG. 11 can be adapted when one or more scanning mirror structures are not stationary during imaging. Instead of stopping for each exposure, it can be advantageous to allow the mirror to move continuously during operation. An alternative process uses an additional scanning mirror pixel velocity estimator that analyzes the movement of the scanning mirror structure during exposure. The scanning mirror pixel velocity estimator can use a short-time step estimation approach to determine the scanning mirror pixel velocity. Pixel positions on the sensor can be mapped to positions on the ground through projection geometry 451. A second projection geometry is then generated based on projection geometry 451 calculated for a second scanning mirror angle corresponding to the expected scanning mirror angle at that time, after a short time has elapsed from the projection estimation time. The position on the ground is remapped to sensor coordinates based on the second projection geometry. The scanning mirror pixel velocity can be estimated as the change in the sensor position with respect to the original position divided by the time step. The scanning mirror pixel velocity can also be supplied to a motion compensation control device, where it can be combined with a forward motion pixel velocity 453 and / or an attitude rate pixel velocity 455.

[0110] The motion compensation control device 458 combines the available pixel velocity estimates input to determine an overall pixel velocity estimate and uses this estimate to control the drive of the motion compensation unit to trigger the dynamic behavior of the motion compensation elements to stabilize the image on the sensor during the exposure time of the camera. Also, the motion compensation control device 458 can receive a timing signal from the system control device 405 that gives the timing required for motion compensation and synchronize it with the settling of the scanning mirror structure and the exposure of the camera. The motion compensation control device 458 can optionally use motion compensation calibration data 461 that can be used to accurately convert the estimated overall pixel velocity to be compensated by the motion compensation unit 415 into dynamic information related to the required control of the motion compensation elements (e.g., rotation or tilt of an optical plate, mirror or other component used in motion compensation).

[0111] The estimated values of the attitude rate pixel velocity 455 and the forward motion pixel velocity 453 are estimates of the motion sensor-based pixel velocities corresponding to different motions of the aerial vehicle. These can be combined by adding the vector components. Alternatively, for example, if only one velocity is available or one velocity is not required (e.g., when the stabilization platform 407 effectively separates the scanning camera system 408 from all attitude rates), a single estimated value can be used.

[0112] The ROI pixel velocity 457 is a directly measured overall pixel velocity estimate that includes motion from the attitude rate and forward motion. The ROI pixel velocity 457 can be used in place of other pixel velocity estimates when it is available, or can be combined with other estimates statistically (e.g., based on a Kalman filter or other suitable linear or non-linear method).

[0113] To achieve appropriate dynamics for the components of the motion compensation unit 415, there may be some latency in the operation of the motion compensation drive 460. Thus, the motion compensation control device 458 can send a control signal for the motion of the motion compensation drive 460 that starts at some required time step before the image exposure, taking this latency into account. The motion compensation control device 458 can optionally update the control signal to the motion compensation drive 460 before image exposure based on an updated pixel velocity estimate such as the low latency attitude rate pixel velocity estimator 456. Such low latency updates can be used to achieve more accurate motion compensation and sharper images.

[0114] The operating principle of tilt optical plate motion compensation is based on the refraction of light at the plate surface, as illustrated in Figure 12. When a light ray 290 is incident on a tilted optical plate 291, it is refracted at the front surface 292 and then at the back surface 293 according to Snell's law, returning to its original direction. The effect on the light ray 290 is that it is offset by a transverse distance δ with respect to its original path. The size of the offset is proportional to the thickness 231 of the optical plate, approximately proportional to the tilt angle (for small angles), and also depends on the refractive index of the glass. The tilt angle (θ) of the optical plate t If the angle changes over time, the offset of the light rays also changes. When this principle is applied to a camera, changing the inclination of the optical plate between the lens and the sensor can be used to shift the light rays that focus to form an image on the sensor, thereby shifting the image on the sensor.

[0115] One or more inclined optical plates may be introduced between the camera lens and the sensor. Such optical plates affect the focus of light rays on the sensor, but this effect may be taken into consideration in the lens design so that the lens's MTF is kept high and a sharp image is obtained. This design is corrected by the design inclination angle of the optical plate, which can be zero inclination angle or a nominal inclination angle related to the expected dynamics of the plate during exposure. At angles other than the design angle of the optical plate, aberrations occur as a result of the change in the optical path, and the MTF decreases. For example, dispersion in the glass of the optical plate causes light rays of different wavelengths to take different deviations, resulting in some chromatic aberration and a decrease in MTF. This decrease in sharpness is small as long as the angle of the optical plate does not deviate too much from the design angle.

[0116] Optical plates can be manufactured according to tolerances relating to the flatness of the two surfaces and the wedge angle between the opposing surfaces. In one embodiment, they should be made from a material with a high refractive index and low dispersion. Such glass has a relatively high Abbe number. These plates are dynamically controlled to follow a desired rotational trajectory, in which case low-density, high-rigidity glass can be used. The total thickness and material of the optical plate to be placed between the lens and the sensor are important parameters in lens design. In one embodiment, BK7 glass can be used because it has excellent all-around properties relating to refractive index, dispersion, density, and rigidity, and is also readily available. Other suitable glasses include S-FPL51, S-FPL53, or SPHM-53.

[0117] Generally, the thicker the glass plate, the more advantageous it is because a smaller inclination is required to achieve a given motion compensation; however, the available space between the lens and the sensor limits the plate thickness. A suitable glass thickness is around 10 mm, but it can be understood that the motion compensation method described herein is effective for a wide range of glass plate thicknesses. Suitable tolerances for plate manufacturing may be a surface with a roughness of <λ / 4, parallel to <1 arcmin, and reflectivity <0.5%.

[0118] Figures 13a, 13b, and 13c illustrate a first arrangement configuration for motion compensation in the camera of a scanning camera system in perspective, side view, and view along the optical axis of the lens, respectively. The camera consists of a focusing lens 240, two optical plates 241 and 242, and a sensor 243. The sensor 243 is mounted in the appropriate focal plane to capture a sharp image of the area. Each optical plate 241, 242 is mounted to allow control of the plate tilt angle around the plate tilt axis. The tilt plate angle can be controlled by a gearbox, directly coupled, or using any suitable actuator or rotary motor (such as a DC motor or brushless motor) that is belt-driven.

[0119] In Figures 13a, 13b, and 13c, the tilt axis of the first optical plate 241 is perpendicular to the tilt axis of the second plate 242. In this configuration, the optical plates 241 and 242 can be tilted around their respective axes to shift the image on the sensor 243 in an orthogonal direction, although a non-orthogonal configuration is also possible. The image of a certain region can be shifted on the sensor 243 along an arbitrary vector direction at a speed that depends on the tilt ratios of the first and second optical plates 241 and 242. If the image of a certain region is moving over that region due to the dynamic movement of the camera relative to that region, the speeds of the two optical plates 241 and 242 can be set independently of the vector direction of the movement and the speed so that the speeds help stabilize the image.

[0120] The transverse shape and size of the optical plates 241 and 242 should be large enough to allow all focused rays to enter the sensor 243. The shape of the optical plates 241 and 242 may be circular, square, rectangular, square bevel, or rectangular bevel. One advantage of rectangular and square-based shapes is that they have a small moment of inertia around the tilt axis, thereby reducing the load on the drive motor used to control the movement of the optical plates in operation. If the aspect ratio of the sensor 243 is not uniform, a rectangular-based shape can have a very low moment of inertia while being large enough to encompass all the rays to be imaged. However, such optical plates require that the long axis of the rectangular optical plates 241 and 242 be precisely aligned with the long axis of the sensor 243. The optical plates 241 and 242 may be mounted so as to be dynamically controlled to tilt according to the required dynamics, as described herein. In one embodiment, the optical plates may be 5 mm thick BK7 glass.

[0121] Figures 14a, 14b, and 14c illustrate a second arrangement configuration for motion compensation in the camera of a scanning camera system in perspective, side view, and view along the optical axis of the lens, respectively. The camera consists of a focusing lens 240, a single optical plate 244, and a sensor 243. The sensor 243 is mounted in the appropriate focal plane to capture a sharp image of the region. The optical plate 244 is mounted to allow the plate tilt angle to be controlled around any axis in a plane perpendicular to the optical axis. This includes tilting around an axis aligned with respect to the sensor axis (exemplified in rotations 281 and 283) and any intermediate angle (such as those exemplified in rotations 282 and 284). The image of a region can be shifted over the sensor 243 along any vector direction determined by the rotation axis at a rate dependent on the tilt ratio of the optical plate 244. When an image of a certain region is moving over that region due to the dynamic movement of the camera relative to that region, the tilt axis and tilt ratio of the optical plate 244 can be set independently so that the vector direction and velocity of the movement act to stabilize the image.

[0122] The criteria for the transverse shape and size of the optical plate 244 are the same as for optical plates 214 and 242, i.e., it should be large enough so that all focused rays can enter the sensor 243. Plates of circular, rectangular, and square shapes may be used. However, since a single plate is used, the spatial limitations on the plate can be reduced compared to the twin-plate case (from Figures 13a, 13b, and 13c), i.e., it should be noted that this means it is possible to increase the thickness of the optical plate 244. As mentioned above, increasing the thickness increases the image shift for a given inclination. In one embodiment, the optical plate 244 may be 10 mm thick BK7 glass.

[0123] Figures 15a, 15b, and 15c illustrate, in perspective, side view, and view along the optical axis of the lens, respectively, different arrangement configurations for motion compensation in the camera of a scanning camera system. The camera consists of a focusing lens 240, two optical plates 245 and 246, and a sensor 243. The sensor 243 is mounted in the appropriate focal plane to capture a sharp image of the area. Each optical plate 245 and 246 is mounted at a fixed plate inclination angle, as can be seen in the side view of Figure 15b. Each optical plate 245 and 246 is also mounted so that it can be rotated around the optical axis by a controllable rotational speed and rotational phase. During operation, the two optical plates 245 and 246 are rotated at independently selected rotational speeds and independent rotational phases. The rotation of the optical plates 245 and 246 is controlled so that their tilts are opposite when the sensor 243 is exposed to capture an image, minimizing image quality loss. During exposure, the phase of the optical plates 245 and 246 determines the vector direction of the image motion, and the rotation speed of the optical plates 245 and 246 determines the velocity of the image motion generated by the camera motion compensation unit. When an image in a certain area is moving over that area due to the camera's dynamic movement relative to that area, the phase and rotation speed of the two optical plates 245 and 246 can be set independently so that the vector direction and velocity of the motion work to stabilize the image.

[0124] The criteria for the transverse shape and size of optical plates 245 and 246 are the same as for optical plates 214 and 242, i.e., they should be large enough so that all focused rays can enter the sensor 243. Due to the rotation of optical plates 245 and 246 around the optical axis, it may be advantageous to use circular optical plates. In one embodiment, optical plates 245 and 246 may be 5 mm thick BK7 glass tilted at 6°.

[0125] Referring again to Figure 11, in one embodiment, the motion compensation unit 415 may comprise a pair of optical plates 241, 242, as described with reference to Figures 13a to 13c. Each inclined optical plate 241, 242 may be tilted by a motion compensation drive unit 460 according to a trajectory defined by a motion compensation control unit 458. One or more motion compensation sensors 459 may be used to track motion and provide feedback to the motion compensation control unit 458.

[0126] Figure 16 shows several exemplary trajectories suitable for the movement of an inclined plate. Three sample trajectories are shown, one with a longer latency T. A lat It has, and one has shorter latency T B lat It has one that is generated by adding a portion of the longer latency trajectory to a portion of the shorter latency trajectory, and this is the mixed latency trajectory T A lat / T B lat It can be called that.

[0127] Figure 16 includes plots of slope (top plot), slope velocity (middle plot), and slope acceleration (bottom plot) associated with three trajectories. Each plot is centered at time (x-axis) 0, assumed to be the midpoint of the image exposure time, and is based on piecewise linear slope acceleration. Alternative trajectories may be formed based on different assumptions, such as piecewise constant slope acceleration, piecewise linear slope jerk, or other preferred assumptions that can be selected based on specific motion compensation control devices and drive devices.

[0128] The three trajectories in Figure 16 represent a time period -T centered at time 0. exp From T exp A constant tilt speed (zero tilt acceleration) is achieved over a certain period. This constant tilt speed duration may be longer than the total camera exposure time to allow for errors in the control of the tilt plate and the timing of exposure. The acceptable maximum and minimum tilts are ±θ of the tilt angle plot.max The following limitations may exist: To minimize the loss of sharpness due to non-zero slopes during exposure, the slope is zero at a time offset of 0 (midway through a period of constant slope rate).

[0129] Comparing the three trajectories, we can see that a longer, mixed latency trajectory may be advantageous for the required acceleration, while a shorter latency trajectory may be advantageous for the required maximum gradient. However, if the aircraft's dynamics have high-frequency components, a mixed, low-latency trajectory may be advantageous because it can use a more recent motion estimate with less error over exposure time.

[0130] Figure 17a includes 14 target area projection geometries G1 to G14 illustrating 14 frames of the scanning pattern of the third scanning drive unit 303 of the scanning camera system 300, which is described with reference to Figure 3 above. In this example, it is assumed that the scanning camera system 300 is aligned with the motion of the flying vehicle as can occur in the absence of yaw. Each ground projection geometry G1 to G14 has an arrow representing the forward motion vector of the flying vehicle. Figure 17a also includes 14 corresponding sensor plots S1 to S14 illustrating the corresponding motion-compensated pixel velocities with respect to the sensor geometry due to forward motion, as arrows in each rectangular sensor outline.

[0131] The upper plot of Figure 17b shows the components of the motion-compensated pixel velocity illustrated in Figure 17a as a function of frame number (1 to 14), with a pixel pitch of 3.2 microns. The lower plot of Figure 17b shows the corresponding plate tilts for the first and second optical plates (e.g., optical plates 241, 242) required for motion compensation. In this case, the plates may be 5 mm BK7 plates, aligned with the first axis at 0° and the second axis at 90°, with tilting the first plate resulting in an image shift along the x-axis and tilting the second plate resulting in an image shift along the y-axis. The conversion from pixel velocity to plate tilt velocity may be achieved using motion-compensated calibration data, which may consist of thickness, material (refractive index), and orientation data for each plate, or parameters of a function that can be used to convert image shift to plate tilt and plate tilt to image shift. Note that none of the pixel velocities in the upper plot of Figure 17b contain an x-axis component, and therefore the tilt velocity relative to the first plate is zero for all frames. In this particular case, the first plate is redundant.

[0132] Figure 18a includes 26 target area projection geometries G1 to G26 illustrating 26 frames of the scanning pattern of the first scanning drive unit 301 of the scanning camera system 300, which is described with reference to Figures 4a to 4f above. The scanning camera system 300 is assumed to be aligned with the motion of the flying vehicle, and each projection geometry has an arrow representing the forward motion vector of the flying vehicle. Figure 18a also includes 26 corresponding sensor plots S1 to S26 illustrating the corresponding motion-compensated pixel velocity with respect to the sensor geometry due to forward motion, as arrows in each rectangular sensor outline.

[0133] Figure 18b gives a plot of the pixel velocity components of the frame illustrated in Figure 18a (when the pixel pitch is 3.2 microns), and the corresponding tilt velocities of the first and second plates required for motion compensation. Here again, a 5 mm BK7 plate is assumed, with the first axis aligned at 0° and the second axis aligned at 90°. Due to the scanning pattern of the first scanning drive unit 301, the pixel velocity generally has non-zero components along both axes, and thus both optical plates are used.

[0134] Figure 19a shows the tilt trajectory for the first optical plate that can be used to achieve motion compensation for the required tilt velocity shown in the second lower plot of Figure 18b. This trajectory consists of 26 sections that are scaled copies of the longer latency trajectories of Figure 16 joined by stationary sections of zero plate tilt. The scaling of each section is set according to the required tilt velocity of the first optical plate. Alternative trajectories can be formed based on the shorter latency trajectories of Figure 16 or mixed latency trajectories, or a mixture of trajectories having different latencies or a mixture of latencies can be used. Figure 19b shows the tilt trajectory for the second optical plate that can be used to achieve motion compensation for the required tilt velocity shown in the second lower plot of Figure 18b. This trajectory is formed in the same way as the tilt trajectory for the first optical plate shown in Figure 19a. In the plots shown in Figures 19a and 19b, the increment between each pair of adjacent dashed vertical lines along the x-axis is equal to 75 milliseconds.

[0135] Figures 20a and 20b illustrate how the alignment of the optical plate affects the motion-compensated tilt velocity calculated through motion-compensated calibration data. Figure 20a shows an alternative set of motion-compensated plate tilt velocities calculated for a first scanning drive unit 301 and the same pixel velocity data as in Figure 18b, but for 5 mm BK7 plates oriented at 45° and 135°. Figure 20b shows an alternative set of motion-compensated plate tilt velocities calculated for a second scanning drive unit 302 and the same pixel velocity data as in Figure 18b, but for 5 mm BK7 plates oriented at 45° and 135°.

[0136] Figures 21a and 21b illustrate how the pixel (pitch: 3.2 microns) velocity and tilt velocity are affected by the alignment of the scanning camera system 300 with respect to the flight path, particularly in the case of a 15-degree yaw that is not corrected on the stabilizing platform. Figures 21a and 21b show the pixel velocity and tilt velocity for scanning drive unit 301 and scanning drive unit 302, respectively, and for 5 mm BK7 tilt plates oriented at 0° and 90°, respectively.

[0137] Figures 22a and 22b illustrate how the pixel (pitch: 3.2 microns) velocity and tilt velocity are affected by the attitude change rate of the scanning camera system 300, particularly for a yaw rate of up to 3° per second, which is randomly sampled in each frame and not corrected on the stabilization platform. Figures 22a and 22b show the pixel velocity and tilt velocity for scanning drive unit 301 and scanning drive unit 302, respectively, and for 5 mm BK7 tilt plates oriented at 0° and 90°, respectively.

[0138] Figures 23a and 23b illustrate how the pixel (pitch: 3.2 microns) velocity and tilt velocity are affected by the rate of change in the attitude and alignment of the scanning camera system 300 with respect to the flight path, particularly for a 15° yaw and a yaw rate of up to 3° per second, randomly sampled in each frame and not corrected on the stabilization platform. Figures 23a and 23b show the pixel velocity and tilt velocity for scanning drive unit 301 and scanning drive unit 302, respectively, and for 5 mm BK7 tilt plates oriented at 0° and 90°, respectively.

[0139] Techniques similar to those applied to generate the sample trajectories in Figures 17a, 17b, 18a, 18b, 19a, 19b, 20a, 20b, 21a, 21b, 22a, 22b, 23a, and 23b can also be applied to the single tilt optical plate in Figure 14. However, in this case, there is a single plate (i.e., optical plate 244) with a thickness approximately twice that of the single plate (e.g., 10 mm BK7), and the tilt plate drive would be operated to achieve the tilt velocity and tilt orientation. The tilt orientation is calculated based on trigonometric operations on the x and y components of the pixel velocity, while the magnitude of the tilt is calculated based on the magnitude of the pixel velocity vector.

[0140] The calculation of spin velocity and phase for a spinning tilt plate motion compensation unit, as described with reference to Figures 15a, 15b, and 15c, is more complex. The two plates (i.e., optical plates 245 and 246) should be controlled to spin in opposite directions so that they are oriented at opposite tilts midway through the exposure time. The opposite tilts should be oriented according to the vector direction of the required pixel velocity, and spin velocities equal to but opposite to the plates should be used, with a magnitude determined according to the plate thickness, plate material, and the magnitude of the required pixel velocity. Such trajectories can be achieved by using trajectories similar to those shown in Figure 16, however such trajectories require very large drive torques, and it may be more efficient to use continuous spin motion for several frames depending on the motion compensation pixel velocity requirements. In one embodiment, the optical plates may be 5 mm thick BK7 glass tilted at 6°.

[0141] If motion compensation requirements are largely due to the linear motion of the flying vehicle, motion compensation errors arising from the variable projection geometry on the sensor pixels can be reduced by introducing a small angle (i.e., a wedge) between the sides of one or both optical plates in the inclined plate case. If motion compensation requirements involve a significant contribution from attitude rate pixel velocity, any advantages of this wedge configuration are diminished.

[0142] An alternative diagram of the scanning camera system 300 based on a solid model of the camera system components fixed to the stabilization platform 407 is shown in Figure 24. When viewed from above, the mirror structure is almost completely obscured by the mounting structure that holds the camera system components in place. Figures 25, 26, 27, 28, and 29 illustrate how the attitude of the flying vehicle affects the orientation of the scanning camera system 300 within the stabilization platform 407.

[0143] Figure 25 shows top and bottom views of the scanning camera system 300 in the case of a flying vehicle aligned with the flight line (y-axis), as is possible for a flying vehicle flying without roll, pitch, or yaw. The inspection hole 305 is aligned with the flying vehicle and therefore with the flight line. It can be seen that the scanning camera system 300 fits into the inspection hole 305 with a small margin around it.

[0144] Figure 26 shows top and bottom views of the scanning camera system 300 when the flying vehicle is aligned with the flight line (y-axis) with a 6° roll corrected by the stabilization platform 407. This configuration is equivalent to the inspection hole 305 remaining aligned with the flight line, but rotating around the axis of the flight line with respect to the scanning camera system 300. The margin around the inspection hole 305 is slightly reduced due to the roll.

[0145] Figure 27 shows top and bottom views of the scanning camera system 300 when the flying vehicle is aligned to the flight line (along the y-axis) with a 6° pitch corrected by the stabilization platform 407. As with the roll shown in Figure 26, the margin around the inspection hole 305 is slightly reduced.

[0146] Figure 28 shows top and bottom views of the scanning camera system 300 when the flight vehicle is aligned to the flight line (y-axis) with a yaw of 15° corrected by the stabilization platform 407. The greater of the modeled yaw (15°) is selected to represent the range of dynamics that may be seen in the range of commercial flight vehicles in which the scanning camera system 300 may be deployed. In contrast to the roll and pitch cases in Figures 26 and 27, the margin around the inspection hole 305 is significantly reduced, and therefore the scanning camera system 300 can no longer fit into the inspection hole 305.

[0147] To reduce spatial requirements within the inspection hole 305, the stabilization system may be configured to compensate only for roll and pitch. This adds further advantages in reducing the size, cost, and complexity of the stabilization platform 407. Figure 29 shows top and bottom views of the scanning camera system 300 when the flight vehicle is aligned to the flight line (y-axis) with a 15° yaw that is not compensated for by the stabilization platform 407. The configuration of the scanning camera system 300 with respect to the stabilization platform 407 is identical to that shown in Figure 25, but the scanning camera system 300 rotates according to the yaw so that the captured scanning pattern rotates over the area of ​​interest. In one embodiment, the scanning angle may be set based on the difference between the yaw angle of the vehicle and a preferred yaw angle (e.g., zero). The scanning angle may be adjusted while one or more flight lights are illuminated or between them.

[0148] Figure 30a shows the ground scanning pattern of the scanning camera system 300 when the flying vehicle has a yaw of 15° with respect to the flight line (y-axis). The curved and linear scanning patterns that make up the overall system scanning pattern can all be rotated by the yaw angle around the z-axis. Images captured with these rotated scanning patterns may have reduced image quality compared to images captured without yaw, as seen in Figure 1a. This reduction in image quality may be due to loss of coverage at certain azimuth angles in oblique images (e.g., increased tolerance of captured images with respect to the base direction), a slight increase in the maximum oblique angle of vertical images due to the angle of the linear scanning pattern passing through the vertical, and / or other factors. Figure 30b illustrates three sets of scanning patterns with forward overlap that may be captured during operation of the scanning camera system in a flying vehicle with a yaw of 15°.

[0149] One aspect of the present disclosure is the design of a first scanning drive unit 301 for capturing oblique images. The selection of scanning angles within the scanning pattern can be advantageously modified to compensate for the yaw of the flying vehicle. In particular, a 1 / 2 yaw correction applied to each sampled scanning angle of the scanning mirror can be used to generate the same scanning pattern that would have been generated if there had been no yaw at the original scanning angle. Figure 31 shows top and bottom views of the scanning camera system 300 when the flying vehicle is aligned to a flight line (along the y-axis) with a 15° yaw corrected by the offset scanning angle of the scanning mirror (i.e., a 7.5° correction of the scanning angle of the scanning mirror with respect to the scanning mirror in Figures 25 to 29).

[0150] Figure 32a shows the ground scanning pattern of the scanning camera system 300 when the flying vehicle has a yaw of 15° with respect to the flight line (y-axis) with scanning angle yaw correction performed in the first scanning drive unit 301. The curved scanning pattern corresponding to the first scanning drive unit 301 matches that of Figure 1 (no yaw), and the linear scanning patterns corresponding to the scanning drive units 302 and 303 are rotated by the yaw angle around the z-axis. In this case, the degradation of image quality of oblique images is eliminated, but a slight degradation in image quality remains due to the slight increase in the maximum oblique angle of vertical images described above. Thus, the overall image quality of the generated images is improved through the yaw correction process based on adaptive control of the scanning angle of the first scanning drive unit 301. Figure 32b shows three sets of scanning patterns with forward overlap that can be captured during operation of the scanning camera system in the flying vehicle under the configuration described with respect to Figure 32a.

[0151] The scanning angle range of the first scanning drive unit 301 required to handle yaws between -15° and 15° is greater than the scanning angle range used for imaging when there is no yaw. In particular, the scanning angle range is extended by 7.5° in each direction from the standard range (-30.7° to +30.7°), giving an extended range (-38.2° to +38.2°). The standard mirror geometry designed for the standard scanning angle range, as described with reference to Figure 4e, would not be large enough to handle scanning angles beyond the standard range. If the mirror is set to a scanning angle beyond its design range, light from light beams emitted from other arrangements in that region can wrap around the outside of the mirror rather than reflect it off it. This light enters the lens and is focused on the sensor, resulting in ghost images (images of other regions superimposed on the acquired image) within the acquired image.

[0152] Figures 33a and 33b help illustrate the formation of ghost images caused by mirrors designed for scan angle ranges smaller than the current scan angle setting. Figure 33a shows a camera 250 imaging an area 251 reflected by mirror 252. The camera 250 is positioned inside the survey hole 253, and the imaged area 251 is very close to the camera 250, however the principle shown in Figure 33a can be generalized to areas at a considerable distance from the camera 250, as in aerial surveying. Light from the arrangement 254 imaged by the camera 250 forms a beam 255, which is focused onto the sensor of the camera 250 at specific pixels corresponding to points on the ground of the arrangement 254. Figure 33b shows the same arrangement configuration, but mirror 252 from Figure 33a is replaced by a smaller mirror 256 that circles a second beam 257 from a second arrangement 258 in the area 251 as it passes through. The second beam 257 is focused by the camera lens as a third beam 259 to the same pixel arrangement on the sensor of camera 250, which is a subset of the first beam 255 in Figure 33a, defined by the reduced mirror geometry.

[0153] Extending the diagram in Figure 33b, each pixel in the sensor may be exposed to light from reflected beams such as beam 259 and non-reflected light from beams such as beam 257. Thus, the exposure of the sensor includes a reflected image component due to the reflected beam of light and a ghost image component due to the direct image beam that bends around the mirror. Furthermore, the reflected image component may have reduced exposure compared to when the mirror is large enough to handle all beams focused onto the sensor and when reduced exposure can vary on the sensor (aperture vignetting).

[0154] Figure 4f illustrates the calculated extended mirror geometry for the case of over-rotation, i.e., for an extended rotation range that would be appropriate for capturing the curved path of the scanning pattern in Figure 32a without the formation of ghost images. The extended scanning mirror geometry is larger than the standard mirror geometry in Figure 4e, which is designed for the standard scanning angle range. In some cases, the cost and complexity of manufacturing the extended scanning mirror may be higher than that of the standard scanning mirror due to the increased size. Furthermore, the mass and moment of inertia of the extended mirror may be larger than that of the standard scanning mirror, the dynamic performance of the extended mirror may be reduced, and the cost and complexity of mounting and controlling its movement may increase.

[0155] In one embodiment of this disclosure, the increased cost, complexity, and dynamic performance degradation of extended mirrors can be mitigated through the use of a hybrid mirror structure. The hybrid mirror structure is based on a standard mirror structure extended to the geometry of an extended mirror using sections of lightweight, low-reflectivity material. The main advantage of the hybrid mirror is that the low-reflectivity material sections block unwanted light beams consisting of rays that would otherwise wrap around mirror scanning angles beyond the standard range, thereby preventing image degradation due to associated ghost images. The lightweight extension also results in a lower moment of inertia compared to a fully extended scanning mirror, thus improving dynamic performance.

[0156] Figure 34a illustrates a hybrid mirror of a scanning drive unit 301 according to one embodiment of the present invention. A low-reflection material 317 is added around the scanning mirror structure 312 to improve image quality when the scanning angle exceeds the standard range.

[0157] Figure 34b illustrates the operating principle of a hybrid mirror to prevent ghost images for the arrangement configuration shown in Figure 33b. Mirror 256 is modified by adding a section of low-reflectivity material 260 that blocks the beam 257 from a second arrangement 258, which contributes to the ghost image. The added low-reflectivity material 260 does not reflect the light beam 261 from the ground point arrangement 254, which is a subset of the original beam 255 in Figure 33a. However, beam 259, which is also a subset of beam 255, is reflected from the reflective surface of mirror 256 and focused onto the camera's sensor 250 through the camera lens. The surface quality of the reflective surface of mirror 259 must be high enough to produce a high-quality focused image that can be captured by the sensor. In this way, the ground arrangement 254 is imaged, but the ground arrangement 258 associated with the ghost image is not imaged. On the other hand, since there is no specular reflection from the low-reflection material 260, the surface quality (roughness, flatness, reflectivity) does not need to be high in order to maintain the overall sharpness and quality of the image captured on the sensor.

[0158] The exposure of pixels corresponding to region arrangement 254 is reduced because only a subset of the original beam 255 (i.e., beam 259) is reflected by mirror 256 and focused onto the sensor. The exposure of other pixels on the sensor may be reduced more or less due to the mirror geometry being smaller than necessary. As a result, vignetting is formed in which the exposure is a function of the arrangement on the sensor, and the captured image may appear darker over some areas compared to others. Vignetting is further described below with respect to Figures 36a and 36b. This vignetting can be modeled and corrected as further described below.

[0159] Low-reflectivity materials can be attached to the mirror in a secure and rigid manner so as to move with the mirror structure to block unwanted beams. If the section no longer needs to meet stringent optical specifications with respect to flatness and reflectivity, it can be manufactured from a lightweight, low-cost material, such as carbon fiber. This brings the additional benefit of reducing the moment of inertia and mass of the hybrid mirror structure with respect to the extended mirror structure. The reduction in the moment of inertia and mass of the mirror structure can enable high-speed rotation of the scanning mirror between the required scanning angles, and thus enable higher speeds for the scanning camera system. The low-reflectivity material section can alter the overall geometry of the hybrid mirror structure with respect to the standard mirror structure. For example, it can form a non-convex extension with respect to a convex standard mirror structure.

[0160] In another embodiment of the present disclosure, the aperture of the camera may be dynamically adjusted so that the geometry of the mirror surfaces 314, 315 of the scanning mirror structure 312 is large enough to reflect all the light rays focused onto the sensor. In particular, the aperture decreases when the scanning angle widens beyond the design parameters of the mirror (i.e., when it rotates too much). In one embodiment, the aperture may be reduced symmetrically. In other embodiments, the aperture may be reduced asymmetrically. The asymmetry of the aperture may be chosen to minimize the change in the aperture while eliminating all beams associated with the ghost image. This can minimize the loss of exposure on the sensor. The minimum required asymmetric change in the aperture can take any shape. Another approach is to use a simple dynamic change to the aperture, such as one or more sliding sections of opaque material, each moved to close the aperture from a particular side so that each selectively blocks a portion of the aperture. This can be achieved using a modified, possibly asymmetric, aperture to control the aperture. Alternatively, active elements such as LCDs can be used to create dynamic apertures that can be electronically controlled to form a wider variety of shapes up to the resolution of the element. Active apertures offer greater control over the aperture compared to sliding sections of material and can result in faster update speeds. On the other hand, they are less practical, do not configure the block effectively, and there is a risk that small portions may pass through the aperture.

[0161] As illustrated with reference to Figures 25, 26, 27, 28, and 29, the geometry of the survey hole can be a constraint in designing a scanning camera system suitable for deployment on an aerial vehicle. The components of the scanning camera system must be mounted inside the survey hole. Furthermore, if a stabilization platform is used to maintain the attitude of the scanning camera system during flight, there should be sufficient spatial margin for the scanning camera system to rotate with the stabilization platform without touching the survey hole wall.

[0162] In addition to these spatial constraints, there are optical constraints related to the placement of the scanning camera system within the inspection hole, as illustrated using Figures 35a and 35b. Figure 35a shows the camera 250 imaging the arrangement 254 of area 251, reflected by the mirror 252, after the inspection hole 253 has moved relative to the camera 250 and mirror 252. Such a situation can occur if the camera 250 and mirror 252 are mounted on a stabilization system above the inspection hole 253, and the attitude of the inspection hole 253 is changed, for example, through the roll or pitch of the flying vehicle to which it is mounted. In this case, the beam of light 255 consists of two parts: (1) a first part 262 of the beam is reflected from the mirror 252 and focused onto the sensor by the camera lens, and (2) a second part 263 of the beam is blocked by the inspection hole 253 and is not reflected from the mirror 252 and focused onto the sensor.

[0163] Pixels corresponding to region arrangement 254 receive less exposure due to occlusion. The exposure of other pixels on the sensor may be reduced to a greater or lesser extent due to occlusion. As a result, vignetting is formed where the exposure is a function of the arrangement on the sensor, and the captured image may appear darker over some areas compared to others.

[0164] It should be noted that some portions of the complete beam 255 may be blocked by inspection holes so as not to enter the low-reflection mirror section. This is illustrated in Figure 35b, where beam 263 is blocked by inspection holes 253 and therefore does not reach the low-reflection material 265 attached to the mirror 266.

[0165] Image vignetting resulting from the geometries shown in Figures 34b, 35a, and 35b is further illustrated by Figures 36a to 36h. Figures 36a to 36h illustrate the calculation of vignetting and ghost images resulting from the geometry of a scanning drive unit in an inspection hole, optionally mounted on a stabilizing platform. These calculations are based on projecting the geometries of various components and objects onto the camera aperture plane, assuming multiple sensor arrangements along the image beam path. This calculation of projected geometry illustrates, in one embodiment, a model of the illuminance of the camera's image sensor by the imaging beam. The illumination model takes into account elements such as the geometry of the confinement space housing the scanning camera system, the scanning angle of the scanning mirror structure, the geometry of the scanning mirror structure, and the roll / pitch / yaw of the vehicle housing the scanning camera system in order to model the illuminance of the camera's image sensor by the imaging beam.

[0166] Figure 36a shows an image of a uniform, non-textured surface affected by vignetting. Darker areas of the image (e.g., sensor placement 277) are more strongly affected by vignetting than brighter areas of the image (e.g., placement 273).

[0167] Figure 36a shows nine sensor configurations 271, 272, 273, 274, 275, 276, 277, 278, and 279, and the image aperture vignetting in each sensor configuration is further illustrated in the corresponding plots in Figure 36b. Each plot in Figure 36b illustrates the illumination of the aperture by light reflected from the mirror of the scanning drive unit. The center of each plot in Figure 36b represents the intersection of the optical axis of the lens and the aperture plane. The solid circular line represents the aperture, and the dashed contour represents the projection of the specular geometry onto the aperture space. If the dashed contour extends to or beyond the solid circle, the mirror is sufficiently large relative to the camera aperture. However, any part of the circle that is not inside the dashed contour is not illuminated by the reflected beam from the mirror. The dotted line is part of a larger contour representing the inspection hole. Within the plot, the survey hole is to the left of the dashed line, and therefore any portion of the solid circle to the right of the aperture is not illuminated by reflected light from the mirror due to occlusion by the survey hole. The diagonal hash line portion of the solid circle represents the portion of the aperture illuminated by reflected light from the mirror, which may relate to the exposure of the sensor pixels corresponding to the plot. It can be seen that the degree of aperture vignetting differs from end to end of the sensor and may depend on both the survey hole occlusion and the finite mirror geometry.

[0168] A vignetting image can be formed for a uniform non-textured region as described above with respect to Figures 36a and 36b. The vignetting image can be generated at full sensor resolution or at a lower resolution, in which case the vignetting at any given pixel can be estimated by interpolating the vignetting image. The vignetting image can be stored in the data storage device 406 as vignetting data 473. This vignetting data 473 can, in one embodiment, be used to update pixel values ​​to correct for vignetting.

[0169] Figure 36b further illustrates the requirements for dynamically adjusting the lens aperture to avoid ghost imaging. In particular, any portion of the circular aperture not contained within the dashed line corresponding to the projection mirror geometry should be masked by a dynamic aperture mask. This defines a minimum level of masking, and as described above, masking larger or more regular areas may be more practical.

[0170] Figure 36c illustrates the images that can be captured for the same geometry shown in Figures 34b, 35a, and 35b, but with the aperture modified. Illumination variations are substantially eliminated, and therefore the image will no longer be affected by vignetting or ghosting.

[0171] Figure 36d illustrates the irregular and asymmetrical region that defines the modified aperture, which can be achieved by dynamically shrinking the circular aperture in Figure 36b. The complete irregular region hashed in all sensor arrangements, indicating that the geometry of the system, including the inspection hole and mirror, does not affect the sensor exposure. This effectively eliminates the vignetting and ghost images that would result from the geometry. As in Figure 36b, the center of each plot in 36d represents the intersection of the optical axis of the lens and the aperture plane. The same is true for each plot in 36e, 36f, 36g, and 36h.

[0172] Figure 36e illustrates a first alternative irregular region that defines a modified aperture, which can be achieved by dynamically shrinking the circular aperture in Figure 36b. In particular, the circularly symmetric aperture is modified by blocking a segment defined by drawing a straight line across the circle. Most of the irregular region in Figure 36e is hashed in most images, but there are small portions that are not hashed in the sensor placement (e.g., 271, 273, 276, and 279). These small regions introduce some vignetting and may allow ghost images if the mirror does not have a low-reflectance material extension that blocks ghost images.

[0173] Figure 36f illustrates a second alternative irregular region that defines a modified aperture, which can be achieved by dynamically shrinking the circular aperture in Figure 36b. In particular, the circularly symmetric aperture is modified by blocking three segments, each defined by drawing a straight line across the circle. The complete irregular region hashed across all sensor placements, indicating that the system geometry, including the inspection hole and mirror, does not affect the sensor exposure. This effectively eliminates the vignetting and ghost images resulting from the geometry.

[0174] Figure 36g illustrates aperture plane geometry for a case similar to that shown in Figure 36b, but with the scanning mirror angle modified so that the mirror geometry projection is deformed and the investigation hole does not block any of the image beam incident on the full aperture. Most of the irregular regions in Figure 36e are hashed in most images, but there are small unhashed areas in the sensor placement (e.g., 271, 273, 274, 276, and 277). These small regions introduce some aperture vignetting and may allow ghost images if the mirror does not have a low-reflectance material extension to block ghost images.

[0175] Figure 36h illustrates a third alternative region defining a modified aperture that can be achieved by dynamically shrinking the circular aperture in Figure 36b, which symmetrically results in a smaller circular aperture. The entire region hashed across all sensor placements, indicating that the system geometry, including the inspection hole and mirror, does not affect the sensor exposure. This effectively eliminates the resulting vignetting and ghost images from the geometry.

[0176] The system controller 405 receives IMU attitude data (roll, pitch, and / or yaw), as well as scan drive unit parameters 434, including the scan angle. The system controller 405 is programmed to correlate the IMU attitude data and scan angle with the presence of an obstruction, for example, due to the inspection hole 253, and the aperture is not included in the projection mirror geometry to calculate the dynamic aperture setting for a given frame. The system controller 405 may calculate the dynamic aperture setting on the fly, and the calculation is based on parameters such as the geometry of the scanning camera system, the scan drive angle, the geometry of the obstruction object such as the constrained camera hole, camera parameters such as sensor geometry and focal length, and flight parameters such as roll, pitch, and yaw. Alternatively, a predefined lookup table of dynamic aperture parameters, which may be a function of the scan angle and / or aircraft parameters such as roll, pitch, and / or yaw, may be used. The system controller 405 controls the dynamic aperture through signals transmitted to the camera, exemplified as 414 and 416 in Figure 10. Based on a control signal, the aperture can be modified mechanically (for example, through the movement of one or more aperture elements), electronically (for example, for an LCD aperture), or in any other way. In one embodiment, the aperture can be modified using one or more motors (for example, a stepping motor, a DC motor). The aperture can be reduced symmetrically, as shown in Figure 36h, asymmetrically, as shown in Figures 36b and 36f, or a combination of the two, as shown in Figure 36d.

[0177] Figure 37 illustrates a post-processing analysis that may be performed after images have been taken for a given aerial survey. The post-processing analysis may be performed during or after the flight and may be performed on a computing platform such as a computer or a cloud processing platform. The analysis uses data from data storage device 406 which may be copied to other data storage devices during or after the flight. In one embodiment, the post-processing analysis may be performed using a network controller, such as an Intel Ethernet PRO network interface card from Intel Corporation of America, to interface with a network. As can be understood, the network may be a public network such as the Internet, a private network such as a LAN or WAN network, or any combination thereof, and may also include a PSTN or ISDN subnetwork. The network may be connected by wire, such as an Ethernet network, or by wireless means, such as a cellular network, including EDGE, 3G, 4G, and 5G wireless cellular systems. The wireless network may be Wi-Fi, Bluetooth®, NFC, a high-frequency identification device, or other known wireless methods of communication.

[0178] One or more individual captured images may optionally be processed by a vignetting analysis process 474 to generate vignetting data 473, which can be used to correct vignetting of image pixels caused by blockage by the inspection hole 305 or by the finite geometry of the scanning mirror structure of the scanning drive unit. The vignetting analysis process 474 may be performed as described above with reference to Figures 36a and 36b. This may use gimbal angles 470 corresponding to a given image from SDU geometry data 467, mirror control data 437, and pixel data 439. In addition, it may use data defining the inspection hole geometry 471 and mirror data 472 relating to the geometry of the scanning mirror structure to determine the aperture exposure for multiple pixels in the sensor, as illustrated in Figure 36b, and then generate vignetting images as described above.

[0179] In one embodiment, exposure data for a particular pixel is stored as a decomposed exposure, where the decomposed region is a decomposed portion in which a circular region corresponding to the aperture is filled with diagonal cross-hatching. A decomposed exposure in 1 would represent a full exposure corresponding to the case where the circular region in Figure 36b is completely filled with diagonal hatch regions. The eclipse image consists of decomposed exposure data corresponding to a particular pixel and may be stored as eclipse data 473. The eclipse data 473 may be used to correct individual pixels from the pixel data 439 by modifying the pixel values ​​according to the eclipse data 473 for that pixel. For example, the RGB values ​​of a pixel may be divided by the decomposed exposure corresponding to that pixel stored in the eclipse data. The eclipse data 473 may be interpolated to provide appropriate eclipse data for all pixels in the image. In another embodiment, the decomposed exposure may be weighted according to the angle of incidence of light rays on the aperture, for example, through cosine or other trigonometric functions.

[0180] The post-processing of pixel data illustrated in Figure 37 begins with a processing step 475 that estimates the camera attitude and position corresponding to each image in a global coordinate system. This attitude and position may correspond to a virtual camera representing the camera's apparent viewpoint and line of sight (i.e., under the assumption that there were no mirrors in the optical path at the time of image acquisition). Processing step 475 may use standard known techniques sometimes referred to as bundle adjustment, and may use pixel data 439 from one or more fixed overview cameras in addition to the scanning camera system. Processing step 475 may use various survey data corresponding to the acquired images, including latitude / longitude data 463, altitude data 464, IMU attitude data 466, motion compensation data 435, mirror control data 437, and SDU geometry data 467. Processing step 475 may optionally generate additional data relating to camera nonlinearities (e.g., barrel distortion) and other aspects of the environment in which the imaging system components and images were acquired (e.g., atmospheric effects).

[0181] The processing step 475 may optionally be followed by a refinement step 476 to improve various estimates or aspects of the imaging system and / or environment, such as attitude, position, and other characteristics. The camera attitude, position, and additional data 477 are stored for use when generating various image products based on the investigation.

[0182] The process for 3D surface reconstruction 478 may generate a 3D textured surface using known techniques described elsewhere, in addition to the camera's orientation, position, and additional data 477, and pixel data 439. Optionally, the 3D surface reconstruction 478 may improve the quality of the output by correcting for vignetting in the captured image by using vignetting data 473 and updating pixel values ​​using a model of the illuminance of the image sensor by the imaging beam.

[0183] The process for orthomosaic generation 479 may generate an orthomosaic 482 using known techniques described elsewhere herein, in addition to the camera's orientation, position, and additional data 477, and pixel data 439. The orthomosaic generation 479 may optionally improve the quality of the output by correcting vignetting in the captured image using vignetting correction data 473.

[0184] The process for vignetting correction 480 may use the camera's attitude, position, and additional data 477, as well as pixel data 439 and vignetting data 473, to generate a raw image in which vignetting in the captured image has been corrected.

[0185] In some embodiments, the captured image may be cropped, or region of interest imaging may be employed, and the captured frame used for the analysis described with respect to Figure 37 may have a variety of different pixel dimensions. This approach may have many advantages, such as reducing the data storage requirements for captured image pixels and removing low-quality pixels resulting from vignetting from the generated image product.

[0186] By capturing images at a scanning angle such that the captured images have overlapping portions, parts of the images can be stitched together to form a cohesive image even after other parts of the image affected by vignetting have been cropped. Cropping may include removing some or all of the vignetting-affected portions. The scanning angle can be selected based on a model of the illuminance of the image sensor by the imaging beam, and illumination may be reduced by partial occlusion from the constrained space, the structure of the scanning mirror being outside a predetermined range of the scanning angle, or a combination thereof. In one embodiment, the predetermined range of the scanning angle is determined by the mirror geometry. For example, the region described with respect to Figures 36a to 36h may be used to model the illuminance of the image sensor by the imaging beam to determine which image sensor arrangements are affected by vignetting and which are not. For portions with vignetting, the scan angle step can be reduced to obtain an image with sufficient overlap. In other words, different scan angle step sizes may be used for different ranges of the scanning angle. In one embodiment, the step size of the scanning angle value of the scanning mirror structure is based on at least one of the following: the yaw angle of the vehicle including the imaging system, the roll of the vehicle, the pitch of the vehicle, the geometry of the scanning mirror structure, the scanning angle, and the geometry of the constrained space.

[0187] Figure 38A illustrates the projection geometry of a preferred set of cropped image frames for the scanning camera system 300 and for two scanning patterns along the flight path. The overlap of the projection geometry of the frames along the curved paths of scanning patterns 111 and 112 appears more uniform than seen in Figure 1b, which can be seen to be achieved by cropping the sensor pixels associated with the outer edges of the curved paths of scanning patterns 111 and 112. In this case, the cropped pixels are found either at the top or bottom, assuming a horizontal orientation of the sensor. Outer cropped pixels with higher bevels are generally more affected by aperture erosion due to the outer edge of the survey hole, and therefore these pixels are rejected, which has the advantage of preserving higher quality pixels and lower bevels taken from sensor positions corresponding to the inner geometry of the curved paths for scanning patterns 111 and 112.

[0188] In some cases, it may also be advantageous to capture images at a higher rate so as to increase the forward overlap of the scanning pattern. The increased forward overlap may allow for rejecting an increased set of pixels along the outside of the scanning patterns 111, 112 without compromising the pixel overlap that may be required for photogrammetry and image post-processing.

[0189] It may be even more advantageous to crop the pixels of scanning patterns 111, 112 at the sides of the sensor, rather than at the top or bottom of the sensor. For example, cropping pixels on one or both sides of the sensor may be advantageous when excessive rotation of the mirror is used to achieve yaw correction. The arrangement and number of pixels to be cropped may be selected based on aperture vignetting caused by an inspection hole or low-reflection section mounted outside the scanning mirror.

[0190] Cropping pixels at the side of the sensor can reduce the overlap of adjacent image pixels, but the required overlap can be restored by increasing the sampling of the scanning angle of the scanning mirror used for the portion of the scanning pattern corresponding to the frame to be cropped. This is illustrated in Figure 38b, where it can be seen that the spacing of the frame projection geometry is reduced toward frames 125 and 126 of scanning patterns 111 and 112, respectively, due to cropping the side of the image. However, the number of frames is increased so that the required overlap (10% in this case) between adjacent frames is maintained. The sampling interval can vary according to any preferred criterion. The interval can alternate between discrete values ​​at a particular threshold of the scanning angle, and can be defined, for example, by a large interval over a particular range of the scanning angle and by a smaller interval beyond that range of the scanning angle. The particular range of the scanning angle may correspond to the range of scanning angles from which the scanning mirror geometry was determined. Alternatively, this interval can vary according to a function of the scanning drive angle. In one embodiment, this function may be based on a trigonometric function over a particular range of the scanning angle. Other suitable functional forms may be defined based on polynomial functions, rational functions, or transcendental functions such as exponential functions, logarithmic functions, hyperbolic functions, power functions, or other periodic functions.

[0191] Increasing scanning angle sampling can be advantageously performed across selected sections of the scanning pattern to enhance image redundancy. For example, it may be advantageous to acquire vertical images at a higher sample rate than other images. The higher the sample rate, the greater the resulting redundancy due to greater overlap between adjacent frames. Increased redundancy can allow for improvement of vertical products, especially when image quality varies among acquired images. Varying image quality can result from fluctuating dynamics during acquisition, specular reflections from the area, or other sources.

[0192] Figure 39a shows a modified set of scan patterns with increased scan angle sampling based on the scan pattern in Figure 38a. In particular, images on linear path scan patterns 113, 114 may have an increased scan angle sampling rate over selected frames 127, 128 toward the y-axis where the image oblique angle is smallest (i.e., the image is closest to vertical). Figure 39b shows a modified set of scan patterns with increased scan angle sampling around a selected set of low oblique frames 127, 128 based on the scan pattern in Figure 38b.

[0193] Figures 38a, 38b, 39a, and 39b illustrate scanning patterns of a scanning camera system that use scanning angle cropping and increased sampling of the scanning mirror to improve output quality and, in some cases, reduce data storage requirements for aerial surveying. Within the scope of the inventions described herein, it can be understood that the scanning angle cropping and sampling geometry can be modified or optimized in many ways to improve the performance of the scanning camera system and the quality of the resulting image-based products.

[0194] This scanning camera system is suitable for deployment on a wide range of aircraft operating at various operating altitudes and ground speeds, with varying GSD and imaging efficiencies. In addition, it is robust against various operating conditions, including deflection and turbulence conditions that result in dynamic instability such as roll, pitch, and yaw of the aircraft. For example, it includes, but is not limited to, twin-piston aircraft such as the Cessna 310, turboprop aircraft such as the Beechworth KingAir 200 and 300 series, and turbofan (jet) aircraft such as the Cessna Citation, enabling aerial photography from low altitudes to altitudes exceeding 40,000 feet and at speeds ranging from less than 100 knots to more than 500 knots. The aircraft may be unpressurized or pressurized, and each inspection port may be open or, as appropriate, include an optical glass window. Each inspection port may optionally be protected by a door that can be closed when the camera system is not in operation. Other suitable flying vehicles include drones, unmanned aerial vehicles (UAVs), airships, helicopters, quadcopters, balloons, spacecraft, and satellites.

[0195] Figure 40 is a table illustrating the range of suitable survey parameters for the scanning camera system 300, which varies from altitude 11,000 ft to 40,000 ft and ground speed from 240 knots to 500 knots. The camera sensor of the scanning camera system 300 is a Gpixel GMAX3265 sensor (9344 x 7000 pixels with a pixel pitch of 3.2 microns), and the focal length of the camera lens varies from 300 mm to 900 mm. Each configuration gives the GSD (Ground Sampling Distance), which is the minimum step between pixels in the captured image. Each configuration is defined according to the flight line spacing, and based on that, the maximum oblique angle (relative to the image used to create a vertical orthomosaic) and km, expressed in degrees. 2Efficiency expressed in units of time can be estimated. The maximum angle of inclination is estimated assuming a yaw range of ±15° on the stabilizing platform and no yaw correction. The table in Figure 40 illustrates numerous features of the scanning camera system 300. It can be seen that the GSD decreases with focal length and increases with altitude. Both the maximum angle of inclination and efficiency increase with flight line spacing.

[0196] Each configuration in Figure 40 also includes the timing budget for the scanning drive units 301, 302, and 303. This timing is based on an analysis of scanning patterns, such as those shown in Figure 1b or Figure 8b, where there is a required 10% overlap between adjacent frames. Each scanning pattern has a corresponding number of frames that increases with focal length due to smaller GSD and the resulting reduced projection geometry of the frames on the ground.

[0197] The timing budget in Figure 40 includes the average available time per frame for moving and setting the scanning mirror, the latency of the motion compensation unit, and the acquisition and transfer of image data from the camera to the data storage device 406. However, in practice, it may be advantageous to allocate a larger time budget for larger angular steps of the scanning mirror, for example, when the scanning angle is reset and a new scanning pattern is started. Furthermore, the time budget may be compromised by additional image acquisition, for example, for the purpose of focusing. It can be seen that the timing per frame decreases with GSD in Figure 40, i.e., decreases with focal length and increases with altitude. It also decreases with ground velocity.

[0198] Figure 41 is a table illustrating a range of suitable investigation parameters for the scanning camera system 300 when the sensor of the scanning camera system 300 is an AMS Cmosis CMV50000 CMOS sensor (7920 x 6004 pixels with a pixel pitch of 4.6 microns). The GSD is lower than in Figure 40 due to the larger pixel pitch, resulting in a larger timing per frame. However, the other parameters remain essentially unchanged. Other suitable sensors include the Vita25k, Python25k, or other RGB, monochrome, multispectral, hyperspectral, or infrared sensors. Different cameras in the scanning camera system may employ different sensors. In an alternative embodiment, the sensor used in each scanning drive unit may be a monochrome sensor, and the overview camera may be a standard RGB. Pan sharpening using coarse RGB overview pixels and fine monochrome pixels may be used to create high-quality color resolution images.

[0199] The scanning camera system may use an overhead camera to achieve several photogrammetry-related requirements. The flight line spacings shown in the tables of Figures 40 and 41 are selected based on the maximum oblique angle of the vertical image, and the sensor and focal length of the overhead camera should be selected such that the projection geometry 115 of the overhead camera is sufficient to achieve those requirements at a given flight line spacing.

[0200] Image quality over a survey area can be improved by narrowing the flight line spacing and flying over that area, or by performing multiple survey flights over the same area. For example, two meandering flight paths may be flights over areas with mutually orthogonal flight line orientations. This may also be achieved by flying with flight lines oriented along the north-south direction and then along the east-west direction. Three meandering paths may be flights with relative flight line orientations spaced, for example, at 60° intervals. Four meandering paths may be flights with relative flight line orientations spaced, for example, at 45° intervals. There is a cost to the efficiency of the imaging when multiple surveys or reduced flight line spacings are used. As those skilled in the art will understand, additional and / or alternative flight paths can be taken to increase angular diversity and help improve 3D mesh reconstruction.

[0201] In any given scanning drive unit, the orientation of the sensor within the camera can be rotated around the optical axis so that the projection geometry is modified. Changing the sensor orientation also changes requirements for flight parameters such as mirror geometry, scanning angle step between image captures, and forward spacing between subsequent scanning pattern captures.

[0202] Figures 42a and 42b illustrate the updated scanning patterns 121 and 122 of the scanning drive unit 301 when the sensor is rotated 90° relative to the vertical sensor orientation. Figures 42c and 42d illustrate the updated scanning pattern 123 of the scanning drive unit 302 when the sensor is rotated 90° relative to the vertical sensor orientation. Figures 42e and 42f illustrate the updated scanning pattern 124 of the scanning drive unit 303 when the sensor is rotated 90° relative to the vertical sensor orientation. Note that the scanning angle steps of scanning patterns 121, 122, 123, and 124 are smaller than the equivalent horizontal sensor orientation scanning patterns 111, 112, 113, and 114, respectively.

[0203] Figures 43a and 43b illustrate the calculated mirror geometry of the mirror surfaces 314 and / or 315 of the scanning mirror structure 312 for a vertical sensor orientation. These differ slightly from those for the horizontal orientation shown in Figures 4e and 4f. It may be advantageous to use a mirror geometry that can handle either sensor orientation. This can be achieved by using a mirror geometry that is a merger of the horizontal and vertical orientation geometries (e.g., the horizontal orientation "convex" geometry in Figure 4e and the vertical orientation "convex" geometry in Figure 43a). If low-reflectance sections are used to allow over-rotation of the mirror without introducing ghost images, these sections should also be a merger of the calculated section geometries for the horizontal orientation geometry (e.g., "over / dilate" in Figure 4f and "over / dilate" in Figure 43b).

[0204] Figure 43c illustrates the calculated mirror geometry of the primary mirror 323 of the scanning drive unit 302 for a vertical sensor orientation. Figure 43c also illustrates the calculated geometry of the primary mirror 327 of the scanning drive unit 303 for a vertical sensor geometry. These differ slightly from those for horizontal sensor orientations illustrated in Figures 5e and 6e, respectively. Figure 43d illustrates the calculated mirror geometry of the secondary mirror 324 of the scanning drive unit 302 for a vertical sensor orientation. Figure 43c also illustrates the calculated geometry of the secondary mirror 328 of the scanning drive unit 303 for a vertical sensor geometry. These differ slightly from those for horizontal sensor orientations illustrated in Figures 5f and 6f, respectively.

[0205] As with the scanning drive unit 301, it may be advantageous to use a mirror geometry that can handle either sensor orientation. This can be achieved by using a mirror geometry that is a combination of horizontal and vertical orientation geometries. For example, the scanning drive unit 302 may use a primary mirror 323 defined by a combination of a horizontal orientation "convex" geometry in Figure 5e and a vertical orientation "convex" geometry in Figure 43c. This geometry can also be used for the primary mirror 327 of the scanning drive unit 303. Similarly, a secondary mirror formed as a combination of "dilate" geometries in Figures 5f and 43d can be used for the secondary mirror 324 of the scanning drive unit 302 and for the secondary mirror 328 of the scanning drive unit 303.

[0206] Figures 44a and 44b show scanning patterns achieved using a scanning camera system 300 with a vertically oriented sensor. The scanning patterns include curved scanning patterns 121, 122 of oblique images and linear scanning patterns 123, 124 when the flying vehicle 110 does not move between image captures of the scanning patterns. Figures 44c and 44d show the same scanning patterns with the effect of realistic forward motion of the flying vehicle during image capture. This also shows multiple scanning patterns in the flight line, and the forward spacing between scanning patterns is increased compared to the horizontally oriented sensor orientation illustrated in Figure 8b.

[0207] Within the scope of this disclosure, alternative camera systems may be used in a mixed configuration of vertical and horizontal sensor orientations. For example, a scanning camera system may combine a vertical sensor orientation scanning drive unit 301 with horizontal sensor orientation scanning drive units 302, 303, or a horizontal sensor orientation scanning drive unit 301 with vertical sensor orientation scanning drive units 302, 303, or any other such combination.

[0208] If the vehicle survey aperture is sufficiently large, or if the vehicle has multiple apertures, one or more additional scanning drive units may be added to the scanning camera system to improve several aspects of the captured images, such as quality for 3D reconstruction. One suitable additional scanning drive unit 350 is illustrated in Figures 45a to 45f. This can be used to capture a single curved scanning pattern 130 extending from a 22.5° oblique angle in front of the flying vehicle 110 (on the y-axis) to a 45° oblique angle to the left of the flying vehicle 110 (on the x-axis), as illustrated in Figures 45c and 45d. Two geometric illustrations of the scanning drive unit 350 from different viewpoints are shown in Figures 45a and 45b. The scanning drive unit 350 has an oblique scanning axis (elevation angle θ) S = -52.5° and azimuth angle φ S The system comprises a single-sided scanning primary mirror 357 held on an elevation angle of 180° and a fixed secondary mirror 358. The geometric diagram shows a configuration in which the scanning angle of the scanning drive unit 356 is set to 0°, at which angle the surface of the primary mirror 357 is oriented by a normal vector directed between the z-axis and the y-axis (elevation angle θ). 1 M = -37.5° and azimuth angle φ 1 M (=0°). The secondary mirror 358 is oriented in the direction of the normal opposite to the normal of the primary mirror 357 when the scanning angle is 0° (elevation angle θ). 1 M = 52.5° and azimuth angle φ 1 M =180°). Vertical z-axis (elevation angle θ S = -82.5° and azimuth angle φ S There is a single camera 355 that is pointed downwards at an angle of 7.5° relative to 180°.

[0209] The scanning drive unit 356 samples scanning angles from -32.4° to 0.01° to generate the scanning pattern 130. The calculated minimum, expanded, convex, and symmetrical geometries for the primary mirror 357 are shown in Figure 45e, along with the axis of rotation and the shifted axis of rotation. The minimum and expanded geometries for the secondary mirror 358 are shown in Figure 45f.

[0210] Other suitable scanning drive units may be designed based on the scanning drive unit 350. For example, the scanning drive unit 351 is a mirror image of the scanning drive unit 350, which may be formed by reflecting all components in the y-axis in Figures 45a and 45b. The scanning drive unit 351 generates a single curved scanning pattern 131 extending from a 22.5° bevel in front of the flying vehicle 110 (on the y-axis) to a 45° bevel to the right of the flying vehicle 110 (on the x-axis), as illustrated in Figures 46a and 46b.

[0211] The scanning drive unit 352 is a mirror image of the scanning drive unit 350, which can be formed by reflecting all components in the x-axis in Figures 45a and 45b. The scanning drive unit 352 generates a single curved scanning pattern 132 that extends from a 22.5° oblique angle behind the flying vehicle 110 (on the y-axis) to a 45° oblique angle to the left of the flying vehicle 110 (on the x-axis), as illustrated in Figures 46c and 46d.

[0212] The scanning drive unit 353 is formed by rotating the scanning drive unit 350 180° around the z-axis in Figures 45a and 45b. The scanning drive unit 353 generates a single curved scanning pattern 133 that extends from a 22.5° oblique angle behind the flying vehicle 110 (on the y-axis) to a 45° oblique angle to the right of the flying vehicle 110 (on the x-axis), as illustrated in Figures 46a and 46b.

[0213] Scanning camera system 354 includes scanning camera system 300 having two additional scanning drive units 350, 351. Combined scanning patterns of scanning camera system 354 are illustrated in Figures 47a and 47b. Scanning camera system 355 includes scanning camera system 300 having four additional scanning drive units 350, 351, 352, 353. Combined scanning patterns of scanning camera system 354 are illustrated in Figures 47c and 47d.

[0214] The scanning drive units 350, 351, 352, 353 and scanning camera systems 354, 355 are illustrated in vertical sensor orientation in Figures 45a-45d, 46a-46d, and 47a-47d, but it should be understood that alternative sensor orientations (e.g., horizontal orientation) may be used with any of the cameras described herein within the scope of this specification.

[0215] Figures 48a to 48f illustrate a scanning drive unit 360 that has advantageous characteristics in terms of spatial compactness due to the use of a shared primary scanning mirror 367. The scanning drive unit 360 can be used to capture pairs of curved scanning patterns 135, 136, each starting on the y-axis and extending to the left and behind with respect to the flying vehicle 110, as shown in Figures 48c and 48d. Two geometric illustrations of the scanning drive unit 360 from different viewpoints are shown in Figures 48a and 48b. The scanning drive unit 360 has an oblique scanning axis (elevation angle θ) S =45° and azimuth angle φ S The system comprises a single-sided shared scanning primary mirror 367 held on an elevation angle of 0° (elevation angle θ), and a fixed secondary mirror 368. The geometric diagram shows a configuration in which the scanning angle of the scanning drive unit 366 is set to 0°, and at this angle, the surface of the shared scanning primary mirror 367 is oriented by a normal vector directed between the z-axis and the y-axis (elevation angle θ). 1 M = -45° and azimuth angle φ 1 M(=0°). The secondary mirror 368 is oriented in the direction of the normal opposite to the normal of the shared scanning primary mirror 367 when the scanning angle is 0° (elevation angle θ). 1 M =45° and azimuth angle φ 1 M (=180°). There are two cameras, 365 and 369. The first camera 365 is vertical z-axis (elevation angle θ). S The second camera 369 is oriented downwards along the vertical z-axis (elevation angle θ = -90°), and the second camera 369 is oriented downwards along the vertical z-axis (elevation angle θ = -90°). S = -67.5° and azimuth angle φ S It is angled downwards at an angle of 22.5° relative to (=0°).

[0216] The scanning drive unit 366 samples scanning angles from -0.01° to 28° to simultaneously generate scanning patterns 135, 136. The sampling of scanning angles may be the same or different for each of the cameras 365, 369. The calculated minimum, expanded, convex, and symmetrical geometries for the shared scanning primary mirror 367 are shown in Figure 48e, along with the rotation axis and shifted rotation axis. The minimum and expanded geometries for the secondary mirror 368 are shown in Figure 48f.

[0217] Other suitable scanning drive units may be designed based on the scanning drive unit 360. For example, the scanning drive unit 361 is a mirror image of the scanning drive unit 360, which may be formed by reflecting all components in the y-axis as shown in Figures 48a and 48b. The scanning drive unit 361 generates a pair of curved scanning patterns 137, 138 extending backward and to the right with respect to the flying vehicle 110 from a point on the y-axis, as illustrated in Figures 49a and 49b.

[0218] The scanning drive unit 362 is a mirror image of the scanning drive unit 360, which can be formed by reflecting all components in the x-axis in Figures 48a and 48b. The scanning drive unit 362 generates a pair of curved scanning patterns 139, 140 extending forward and to the left with respect to the flying vehicle 110 from a point on the y-axis, as illustrated in Figures 49c and 49d.

[0219] The scanning drive unit 363 is formed by rotating the scanning drive unit 360 180° around the z-axis in Figures 48a and 48b. The scanning drive unit 362 generates a pair of curved scanning patterns 141, 142 extending forward and to the left with respect to the flying vehicle 110 from a point on the y-axis, as illustrated in Figures 49e and 49f.

[0220] Figures 50a to 50d are a series of perspective views of the combined components of the scanning drive units 301, 360, and 361 of the scanning camera system 364 described with respect to Figures 4a to 4f, 48a to 48f, and 49a to 49f above. The scanning drive unit 360 and the scanning drive unit 361 are each located on either side of the scanning drive unit 301. This arrangement is spatially very efficient and advantageous for deployment in a wide range of flying vehicle camera (surveillance) holes. Figures 50e and 50f show scanning patterns achieved using the scanning camera system 364, including curved scanning patterns 111, 112 for oblique images and curved scanning patterns 135, 136, 137, 138 for images with variable oblique angles. In addition to the imaging capabilities of the scanning drive units, the scanning camera system 364 may also include one or more fixed cameras.

[0221] Figures 51a to 51f illustrate a scanning drive unit 370 that has similar geometric characteristics to the scanning drive unit 360 but does not use a shared scanning mirror. The scanning drive unit 370 can be used to capture a single curved scanning pattern 150 that extends to the left with respect to the flying vehicle 110, starting from a 22.5° oblique angle in front of the flying vehicle 110 (on the y-axis) and moving backward, as illustrated in Figures 51c and 51d. Two geometric illustrations of the scanning drive unit 370 from different viewpoints are shown in Figures 51a and 51b.

[0222] The scanning drive unit 370 controls the oblique scanning axis (elevation angle θ). S = -45° and azimuth angle φ S The system comprises a single-sided scanning primary mirror 377 held on an elevation angle of 0° (elevation angle θ), and a fixed secondary mirror 378. The geometric diagram shows a configuration in which the scanning angle of the scanning drive unit 376 is set to 0°, and at this angle, the surface of the primary mirror 377 is oriented by a normal vector directed between the z-axis and the y-axis (elevation angle θ). 1 M = -45° and azimuth angle φ 1 M (=0°). The secondary mirror 378 is oriented in the direction of the normal opposite to the normal of the primary mirror 377 when the scanning angle is 0° (elevation angle θ). 1 M =45° and azimuth angle φ 1 M =180°). Vertical z-axis (elevation angle θ S = -67.5° and azimuth angle φ S There is a single camera 375 oriented downward at an angle of 22.5° relative to (=0°). A scanning drive unit 376 samples scanning angles from -0.01° to 28° to generate a scanning pattern 150. The calculated minimum, expanded, convex, and symmetrical geometries for the primary mirror 377 are shown in Figure 51e, along with the axis of rotation and the shifted axis of rotation. The minimum and expanded geometries for the secondary mirror 378 are shown in Figure 51f.

[0223] Other suitable scanning drive units may be designed based on the scanning drive unit 370. For example, the scanning drive unit 371 is a mirror image of the scanning drive unit 370, which may be formed by reflecting all components in the y-axis in Figures 51a and 51b. The scanning drive unit 371 generates a single curved scanning pattern 151 extending to the right of the flying vehicle 110, starting from a 22.5° oblique angle in front of the flying vehicle 110 (on the y-axis) and moving backward, as illustrated in Figures 52a and 52b.

[0224] The scanning drive unit 372 is a mirror image of the scanning drive unit 370, which can be formed by reflecting all components in the x-axis in Figures 51a and 51b. The scanning drive unit 372 generates a single curved scanning pattern 152 that extends from a 22.5° oblique angle behind the flying vehicle 110 (on the y-axis) to a 45° oblique angle to the left of the flying vehicle 110 (on the x-axis), as illustrated in Figures 52c and 52d.

[0225] The scanning drive unit 373 is formed by rotating the scanning drive unit 370 180° around the z-axis in Figures 51e and 51f. The scanning drive unit 373 generates a single curved scanning pattern 153 that extends from a 22.5° oblique angle behind the flying vehicle 110 (on the y-axis) to a 45° oblique angle to the right of the flying vehicle 110 (on the x-axis), as illustrated in Figures 52a and 52b.

[0226] The scanning camera system 379 comprises scanning drive unit units 301, 360, 361, 372, and 373. The combined scanning patterns of the scanning camera system 379 are illustrated in Figures 53a and 53b.

[0227] The scanning camera system 381 includes a scanning camera system 300 having two additional scanning drive units 372 and 373. The combined scanning patterns of the scanning camera system 382 are illustrated in Figures 53c and 53b.

[0228] The scanning camera system 382 includes a scanning camera system 300 having four additional scanning drive units 370, 371, 372, and 373. The combined scanning patterns of the scanning camera system 382 are illustrated in Figures 53e and 53f.

[0229] Scanning drive units 301, 302, 303, 350, 351, 352, 353, 360, 361, 362, 363, 370, 371, 372, and 373 are examples of scanning drive units that use a scanning drive axis parallel to the flying vehicle on which the mirror surface rotates. Such scanning drive units may be referred to as inclined scanning drive units. Alternative scanning drive units may use a scanning drive axis that is not parallel to the plane of the mirror surface on which it rotates. Such scanning drive units employ a spinning mirror and may be referred to as spinning scanning drive units.

[0230] Figures 54a to 54f illustrate a spinning scanning drive unit 380 having a vertically oriented sensor orientation. The scanning drive unit 380 has a horizontal scanning axis (elevation angle θ S = -0° and azimuth angle φ S The system comprises a single-sided scanning primary mirror 383 held on an elevation angle of 0° (elevation angle θ), and a fixed secondary mirror 384. The geometric diagram shows a configuration in which the scanning drive unit 380 is set to a scanning angle of 0°, at which angle the surface of the primary mirror 383 is oriented by a normal vector directed between the z-axis and the y-axis (elevation angle θ). 1 M = -45° and azimuth angle φ 1 M (=0°). The secondary mirror 378 is oriented in the direction of the normal opposite to the normal of the primary mirror 383 when the scanning angle is 0° (elevation angle θ). 1 M =45° and azimuth angle φ 1 M =180°). Vertically downward (elevation angle θ). S =-90°, azimuth φ SThere is a single camera 376 oriented to (=0°). As shown in Figures 54c and 54d, the scanning drive unit 380 generates a single linear scanning pattern 155 that extends from a 45° oblique angle to the left (on the x-axis) of the flying vehicle to a 45° oblique angle to the right (on the x-axis) of the flying vehicle, as the scanning angle varies between -45° and 45°.

[0231] The scanning drive unit 380 samples scanning angles from -45° to 45° to generate scanning pattern 155. In some configurations, two or more scanning drive units 380 may be used, and the image acquisition of scanning pattern 155 is divided among the scanning drive units to meet the system's timing budget requirements. For example, scanning drive unit 380 may sample scanning angles from -45° to 0°, and a second scanning drive unit may sample scanning angles from 0° to 45° so that the entire range of scanning angles is sampled and the same scanning pattern is achieved with approximately twice the time budget per frame. Scanning drive units 302, 303 are similarly used to divide a single line scanning pattern into two scanning patterns 113, 114. Any of the scanning patterns described herein can similarly be divided into several parts to make an effective trade-off between the time budget for image acquisition and the spatial requirements and additional costs of extra scanning drive units.

[0232] The calculated minimum, expanded, convex, and symmetrical geometries for the primary mirror 383 are shown in Figure 54e, along with the axis of rotation and the shifted axis of rotation. The minimum and expanded geometries for the secondary mirror 384 are shown in Figure 54f.

[0233] As those skilled in the art will understand, any scanning camera system or obvious modification thereof described herein can be integrated with one or more scanning drive units or scanning camera systems described herein to achieve various timing requirements. Furthermore, the selection of the scanning angle that defines the scanning pattern can be chosen according to the requirements and constraints of operating conditions such as altitude and flight speed.

[0234] As those skilled in the art will understand, the position of the scanning drive in any scanning drive unit can be selected at either end of the mirror depending on the available space for installation and the geometry of the scanning drive. Furthermore, the precise distance between mirrors along the optical axis may also be changed, thereby achieving the most efficient use of space and minimizing obstructions that would degrade the quality of the captured images. Such small geometric changes alter the required mirror geometry but do not significantly alter the line of sight of the captured images. Such changes may allow more scanning drive units to be placed in constrained spaces with minimal or no obstruction, resulting in a better imaging system that produces a wider variety and / or higher quality captured images.

[0235] Figures 55a to 55f illustrate the scanning patterns of three scanning camera systems employing a scanning drive unit 380. Scanning camera system 391 includes scanning drive units 301 and 380. The combined scanning patterns of scanning camera system 391 are illustrated in Figures 55a and 55b. Scanning camera system 392 includes scanning camera system 391 and scanning drive units 370 and 371. The combined scanning patterns of scanning camera system 391 are illustrated in Figures 55c and 55d. Scanning camera system 393 includes scanning camera system 392 and scanning drive units 372 and 373. The combined scanning patterns of scanning camera system 393 are illustrated in Figures 55e and 55f.

[0236] As shown in Figures 56a and 56b, the scanning drive unit 385 is formed by rotating the scanning drive unit 380 45° around the z-axis in Figures 54a and 54b and sampling an extended scanning angle range from -50.4° to 50.4°. The scanning drive unit 385 generates a single linear scanning pattern 156 that extends from a 50.4° oblique angle in front of and to the left of the flying vehicle to a 50.4° oblique angle behind and to the right of the flying vehicle.

[0237] As shown in Figures 56c and 56d, the scanning drive unit 386 is formed by rotating the scanning drive unit 380 by -45° around the z-axis in Figures 54a and 54b, and sampling an extended scanning angle range from -50.4° to 50.4°. The scanning drive unit 386 generates a single linear scanning pattern 157 extending from a 50.4° oblique angle in front of and to the right of the flying vehicle to a 50.4° oblique angle behind and to the right of the flying vehicle.

[0238] The scanning camera system 394 comprises scanning drive units 385 and 386. Combined scanning patterns of the scanning camera system 394 are illustrated in Figures 56e and 56f. In some configurations, two or more scanning drive units 385 and 386 may be used, and the image acquisition of scanning patterns 156 and 157 is divided between the scanning drive units to meet the system's timing budget requirements.

[0239] As previously stated, any of the scanning camera systems and their apparent modifications described herein can be integrated with one or more of the scanning drive units or scanning camera systems described herein to achieve various timing requirements.

[0240] Figures 57a to 57e illustrate a number of scanning drive units and / or scanning camera systems based on the scanning drive unit 380, each employing a camera with a 600mm focal length and 120mm aperture lens that focuses light onto an AMS Cmosis CMV50000 CMOS sensor. The scanning drive unit 387 has the same geometry as the scanning drive unit 380 but samples a reduced scanning angle range from -15° to 30.2° to produce the short linear scanning pattern 160 shown in Figure 57a. The scanning drive unit 388 is formed by rotating the scanning drive unit 380 by 22.5° around the x-axis. The scanning drive unit 388 samples a reduced scanning angle range from -30.2° to 15° to produce the short linear scanning pattern 161 shown in Figure 57b. The scanning drive unit 389 is formed by rotating the scanning drive unit 380 by 22.5° around an axis 30° from the x-axis in the horizontal plane. The scanning drive unit 389 samples a reduced scanning angle range from -28° to 47.5° to generate the linear scanning pattern 162 shown in Figure 57c. The scanning drive unit 390 is formed by rotating the scanning drive unit 380 by 22.5° around an axis 30° from the x-axis in the horizontal plane. The scanning drive unit 390 samples a reduced scanning angle range from -47.5° to 28° to generate the linear scanning pattern 163 shown in Figure 57d.

[0241] The scanning camera system 395 includes scanning drive units 387, 378, 389, and 390 in addition to the modified scanning drive unit 301. The modified scanning drive unit 301 uses a vertically oriented AMS Cmosis CMV50000 CMOS sensor and a lens with a focal length of 600 mm and an aperture of 120 mm. Figures 57e and 57f illustrate the combined scanning patterns of the scanning camera system 395.

[0242] As shown in FIGS. 58c and 58d, FIGS. 58a and 58b show a perspective view of a scanning drive unit 501 having three cameras 506, 507, 508 that can be used to photograph three scanning patterns 160, 161, 162 having arcs centered at an elevation angle of 45°. The three scanning patterns 160, 161, 162 combine to form a complete circle, as illustrated in FIGS. 58c and 58d. The scanning drive unit 501 includes a scanning mirror structure 502 attached to a scanning drive 503 on a vertical scanning axis (elevation angle θ S =-90° and azimuth angle φ S =0°). In one embodiment, the scanning mirror structure 502 has two sides. The geometric illustration shows a configuration in which the first mirror surface 504 is oriented in a normal direction facing toward the first camera 506 along the y-axis (elevation angle θ 1 M =0° and azimuth angle φ 1 M =0°) when the scanning angle of the scanning drive 503 is set to 0°. The second mirror surface 505 is mounted on the opposite side of the scanning mirror structure 502 and is directed between the camera 507 and the camera 508.

[0243] The cameras 506, 507, and 508 are oriented downward at an oblique angle, but the azimuth angles are spaced 120° apart (elevation angle θ of camera 506 S =-45° azimuth angle φ S =180°, elevation angle θ of camera 507 S =-45° and azimuth angle φ S =60°, elevation angle θ of camera 508 S =-45° and azimuth angle φ S =-60°). The cameras 506, 507, 508 utilize Gpixel GMAX3265 sensors (9344×7000 pixels with a pixel pitch of 3.2 microns). The camera lens may have a focal length of 215 mm and an aperture of 120 mm (corresponding to F1.8). This shorter focal length results in a lower image resolution, but the wider scanning pattern can be advantageous in terms of flight line spacing and imaging efficiency.

[0244] FIG. 58e shows various mirror geometries calculated for the scanning drive unit 501. These include the minimum geometry ("min"), the dilated minimum geometry ("dilate") that extends 5 mm beyond the minimum geometry at the outer periphery, and the dilated convex geometry ("convex") that is the convex hull of the dilated minimum geometry. FIG. 58f shows the dilated convex geometry again ("convex") and also shows the extended geometry ("over") that may be required when the range of the scanning angle is extended by 7.5° at each end of the scanning angle range to increase the overlap region between the scanning patterns.

[0245] The scanning drive unit 509 is based on the scanning drive unit 302, but the camera 321 uses a Gpixel GMAX3265 sensor and a lens with a focal length of 215 mm and an aperture of 120 mm (corresponding to F1.8). Further, the scanning drive 322 samples a modified range of scanning angles from -10.25° to 10.25° to generate the linear scanning pattern 165 shown in FIGS. 59a and 59b. The scanning camera system 510 includes the scanning drive units 501, 509 for generating the combined scanning patterns illustrated in FIGS. 59c and 59d.

[0246] FIGS. 60a and 60b show a scanning drive unit 511 having four cameras 516, 517, 518, 519 from different viewpoints that can be used to capture four scanning patterns 170, 171, 172, 173 having arcs centered at an elevation angle of 45° that are combined to form a complete circle. The top view and perspective view of the scanning patterns from the four cameras 516, 517, 518, 519 of this scanning drive unit 511 are shown in FIGS. 60c and 60d. The scanning drive unit 511 has a vertical scanning axis (elevation angle θ S = -90° and azimuth angle φ SThe system includes a scanning mirror structure 512 mounted on a scanning drive unit 513 (elevation angle θ = 0°). In one embodiment, the scanning mirror structure 512 has two sides. The geometric diagram shows that the first mirror surface 514 is oriented in the normal direction along the y-axis toward the space between cameras 516 and 517 (elevation angle θ = 0°). 1 M =0° and azimuth angle φ 1 M The diagram shows a configuration in which the scanning angle of the scanning drive is set to 0° (=0°). The second mirror surface 515 is mounted on the opposite side of the scanning mirror structure 512 and is oriented between cameras 518 and 519. Cameras 516, 517, 518, and 519 are oriented downward at an oblique angle, but their azimuth angles are separated from each other at either 60° or 120° (elevation angle θ of camera 516). C =-45°azimuth φ C =150°, elevation angle θ of camera 517 C = -45° and azimuth angle φ C = -150°, elevation angle θ of camera 518 C = -45° and azimuth angle φ C = -30°, elevation angle θ of camera 519 C = -45° and azimuth angle φ C (=30°).

[0247] Each camera 516, 517, 518, and 519 samples the scanning angle of the scanning drive unit 513 over a 45° range to achieve a quarter-circle scanning pattern arc. The non-uniform azimuthal spacing of cameras 516, 517, 518, and 519 around the scanning mirror structure 512 may be advantageous in terms of the timing budget for acquisition and the simultaneous use of the scanning mirror structure 512 for acquiring images with cameras 516, 517, 518, and 519. The scanning drive unit 511 generates the same scanning pattern achieved by the scanning drive unit 301, which samples scanning angles in the range of -45° to 45°. The use of additional cameras may be advantageous because it reduces the size of the scanning mirror structure 512 required to achieve acquisition. This arrangement configuration may also be advantageous in terms of the yaw robustness of the flying vehicle 110 because the scanning pattern captures the full 360° range in azimuthal.

[0248] Figure 60e shows various mirror geometries calculated for the scanning drive unit 511. These include the minimum geometry ("min"), the dilate minimum geometry ("dilate") which is extended by 5 mm beyond the minimum geometry at the periphery, and the convex geometry ("convex") which is the convex hull of the dilate minimum geometry. Figure 60f shows the convex geometry again ("convex"), and also the over geometry ("over") which may be required when the scanning angle range is extended by 7.5° at each end of the scanning angle range to increase the overlap area between scanning patterns.

[0249] Figures 61a and 61b show perspective views of a scanning drive unit 521 having four cameras 526, 527, 528, and 529 that can be used to capture four scanning patterns 175, 176, 177, and 178 having arcs, as shown in Figures 61c and 61d. Top and perspective views of the scanning patterns 175, 176, 177, and 178 from the four cameras 526, 527, 528, and 529 of the scanning drive unit 521 are shown in Figures 61c and 61d.

[0250] The scanning drive unit 521 controls the vertical scanning axis (elevation angle θ). S = -90° and azimuth angle φ S The system includes a scanning mirror structure 522 mounted on a scanning drive unit 523 (elevation angle θ = 0°). In one embodiment, the scanning mirror structure 522 has two sides. The geometrical diagrams in Figures 61a and 61b show that the first mirror surface 524 is oriented in the normal direction directed toward the camera 526 and camera 527 along the y-axis (elevation angle θ = 0°). 1 M =0° and azimuth angle φ 1 MThe diagram shows a configuration in which the scanning angle of the scanning drive unit 523 is set to 0° (=0°). The second mirror surface 525 is mounted on the opposite side of the scanning mirror structure 522 and is directed between cameras 528 and 529. Cameras 526, 527, 528, and 529 are oriented downward at an oblique angle but are 90° apart from each other with respect to the azimuth (elevation angle θ of camera 526). C =-47°azimuth φ C =135°, elevation angle θ of camera 527 C = -43° and azimuth angle φ C =45°, elevation angle θ of camera 528 C = -47° and azimuth angle φ C = -45°, elevation angle θ of camera 529 C = -43° and azimuth angle φ C (=-43°).

[0251] Each camera 526, 527, 528, and 529 samples the scanning angle of the scanning drive unit 523 over a range of 60° to achieve a scanning pattern arc of 1 / 3 circle. The use of two different elevation angles for cameras 529 and 527 compared to cameras 526 and 528 directed towards the shared scanning mirror structure 522 means that the arcs do not overlap and capture complementary areas of the target area relative to the side of the flying vehicle 110. This may be advantageous in terms of the efficiency of the scanning camera system because a wider flight line spacing can be used while maintaining any necessary distribution of oblique image capture to the left and right sides of the flying vehicle 110. It may also be advantageous in improving the quality of image capture and 3D model generation for oblique images. This arrangement configuration may also be advantageous in terms of the yaw robustness of the flying vehicle 110 because the scanning pattern captures the full 360° range in azimuth.

[0252] Figure 61e shows various mirror geometries calculated for the scanning drive unit 521. These include the minimum geometry ("min"), the dilate minimum geometry ("dilate") which is extended by 5 mm beyond the minimum geometry at the periphery, and the convex geometry ("convex") which is the convex hull of the dilate minimum geometry. Figure 61f shows the convex geometry again ("convex"), and also the over geometry ("over") which may be required when the scanning angle range is extended by 7.5° at each end of the scanning angle range to increase the overlap area between scanning patterns.

[0253] The scanning drive unit 530 has the same geometry as the scanning drive unit 302, but samples a modified scanning angle range from -10.25° to 10.25° to generate the short linear scanning pattern 179 shown in Figures 62a and 62b. The scanning pattern 179 can be used to generate high-quality vertical image acquisition. The scanning camera system 531 includes scanning drive units 530, 511 for generating the combined scanning pattern shown in Figures 62c and 62d. The scanning camera system 532 includes scanning drive units 530, 521 for generating the combined scanning pattern shown in Figures 62e and 62f.

[0254] The scanning drive unit 535 has the same geometry as the scanning drive unit 380, but samples a reduced scanning angle range from -22.5° to 22.5° to generate the short linear scanning pattern 180 shown in Figures 63a and 63b. The scanning pattern 180 can be used to generate high-quality vertical image acquisition. The scanning camera system 536 includes scanning drive units 535 and 511 for generating the combined scanning pattern shown in Figures 63c and 63d. The scanning camera system 537 includes scanning drive units 535 and 521 for generating the combined scanning pattern shown in Figures 63e and 63f.

[0255] Clearly, in light of the above teachings, numerous modifications and variations are possible. Therefore, it will be understood that, within the scope of the appended claims, the present invention may be carried out in ways other than those described herein.

[0256] Accordingly, the foregoing description discloses and illustrates only exemplary embodiments of the present invention. As those skilled in the art will understand, the present invention can be embodied in other specific forms without departing from the spirit or essential features of the invention. Accordingly, the disclosure of the present invention is intended to be illustrative without limiting the scope of the invention or any other claims. This disclosure, including readily identifiable modifications of the teachings herein, partially defines the scope of the terms of the foregoing claims such that the subject matter of the invention is not dedicated to the public.

[0257] Embodiments of this disclosure may be as described in the following insert.

[0258] (1) An imaging system comprising: a first camera configured to capture a first set of oblique images along a first scanning path over a target area; a second camera configured to capture a second set of oblique images along a second scanning path over a target area; a scanning mirror structure having at least one mirror surface; and a drive device coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the first camera has an optical axis set at an oblique angle with respect to the scanning axis and includes a lens that focuses a first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera; and the second camera is set at an oblique angle with respect to the scanning axis. An imaging system comprising a lens having an optical axis and focusing a second imaging beam reflected from a scanning mirror structure onto the image sensor of a second camera, wherein at least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, changes according to the scanning angle, the image sensor of the first camera captures a first set of oblique images along a first scanning path by sampling the first imaging beam at a first value of the scanning angle, and the image sensor of the second camera captures a second set of oblique images along a second scanning path by sampling the second imaging beam at a second value of the scanning angle.

[0259] (2) The system according to (1), wherein at least one mirror surface includes a first mirror surface and a second mirror surface substantially opposite to the first mirror surface, and a first imaging beam is reflected from the first mirror surface and a second imaging beam is reflected from the second mirror surface.

[0260] (3) The system according to any one of (1) to (2), wherein the first scanning angle with respect to the first camera is the same as the first scanning angle with respect to the second camera.

[0261] (4) The system according to any one of (1) to (3), wherein the image sensor of the first camera and the image sensor of the second camera simultaneously capture images of a first set of oblique images and a second set of oblique images, respectively.

[0262] (5) The system according to any one of (1) to (4), wherein the geometry of at least one mirror surface is determined at least in part on at least one of a set of one or more predetermined orientations of the image sensor of the first camera and one or more predetermined orientations of the image sensor of the second camera, and a set of scanning angles of the scanning mirror structure.

[0263] (6) The scanning mirror structure is symmetrical with respect to the scanning axis, as described in any one of (1) to (5).

[0264] (7) The system according to any one of (1) to (6), wherein the scanning mirror structure is asymmetric with respect to the scanning axis.

[0265] (8) The system according to any one of (1) to (7), wherein the scanning angle is the inclination angle of the scanning mirror structure.

[0266] (9) The system according to any one of (1) to (8), wherein the step of the tilt angle is determined based on the size of the image sensor and the focal lengths of the first and second cameras.

[0267] (10) The system according to any one of (1) to (9), wherein the first camera and the second camera are inclined at a predetermined angle toward the scanning mirror structure.

[0268] (11) The system according to any one of (1) to (10), wherein the given angle is substantially 45 degrees.

[0269] (12) The system according to any one of (1) to (11), wherein the first scan path and the second scan path are symmetric.

[0270] (13) The system according to any one of (1) to (12), wherein the azimuth angle of the first camera is substantially 180 degrees from the azimuth angle of the second camera.

[0271] (14) The system according to any one of (1) to (13), wherein the first scan path and the second scan path are curved.

[0272] (15) The system according to any one of (1) to (14), further comprising at least one third camera configured to capture a vertical image, and at least one mirror configured to direct a third imaging beam corresponding to the vertical image towards at least one third camera.

[0273] (16) The system according to any one of (1) to (15), further comprising a third camera configured to capture a third set of oblique images along a third scanning path over a target area, the third camera comprising a lens for focusing a third imaging beam reflected from a scanning mirror structure onto an image sensor of the third camera.

[0274] (17) The system according to any one of (1) to (16), further comprising a fourth camera configured to capture a fourth set of oblique images along a fourth scanning path over a target area, the fourth camera further comprising a fourth camera equipped with a lens for focusing a fourth imaging beam reflected from a scanning mirror structure onto an image sensor of the fourth camera.

[0275] (18) The system according to any one of (1) to (17), further comprising a third camera configured to capture a third set of images, and a second scanning mirror structure configured to orient a third imaging beam corresponding to the third set of images so as to be received by the third camera.

[0276] (19) The system according to any one of (1) to (18), further comprising a fourth camera configured to capture a fourth set of images, and a third scanning mirror structure configured to orient a fourth imaging beam corresponding to the fourth set of images so as to be received by the fourth camera.

[0277] (20) A third camera configured to capture a third set of oblique images along a third scanning path over a target area; a fourth camera configured to capture a fourth set of oblique images along a fourth scanning path over a target area; a second scanning mirror structure having at least one mirror surface; and a second drive unit coupled to the second scanning mirror structure and configured to rotate the second scanning mirror structure around a second scanning axis based on a second scanning angle, wherein the third camera has an optical axis set at an oblique angle with respect to the second scanning axis and includes a lens that focuses a third imaging beam reflected from the second scanning mirror structure onto the image sensor of the third camera; and the fourth camera has an optical axis set at an oblique angle with respect to the second scanning axis. The system according to any one of (1) to (19), comprising a lens that focuses a fourth imaging beam reflected from a second scanning mirror structure onto the image sensor of a fourth camera, wherein at least one of the elevation angle and azimuth angle of the third imaging beam and at least one of the elevation angle and azimuth angle of the fourth imaging beam changes according to the second scanning angle, the image sensor of the third camera captures a third set of oblique images along a third scanning path by sampling the third imaging beam at a first value of the second scanning angle, and the image sensor of the fourth camera captures a fourth set of oblique images along a fourth scanning path by sampling the fourth imaging beam at a second value of the second scanning angle.

[0278] (21) A fifth camera configured to capture a fifth set of oblique images along a fifth scanning path over a target area; a sixth camera configured to capture a sixth set of oblique images along a sixth scanning path over a target area; a third scanning mirror structure having at least one mirror surface; and a third drive unit coupled to the third scanning mirror structure and configured to rotate the third scanning mirror structure around a third scanning axis based on a third scanning angle, wherein the fifth camera has an optical axis set at an oblique angle with respect to the third scanning axis and includes a lens that focuses the fifth imaging beam reflected from the third scanning mirror structure onto the image sensor of the fifth camera; and the sixth camera has an optical axis set at an oblique angle with respect to the third scanning axis. The system according to any one of (1) to (20), further comprising a lens that focuses a sixth imaging beam reflected from a third scanning mirror structure onto the image sensor of a sixth camera, wherein at least one of the elevation angle and azimuth angle of the fifth imaging beam and at least one of the elevation angle and azimuth angle of the sixth imaging beam varies according to the third scanning angle, the image sensor of the fifth camera captures a fifth set of oblique images along the fifth scanning path by sampling the fifth imaging beam at a third value of the third scanning angle, and the image sensor of the sixth camera captures a sixth set of oblique images along the sixth scanning path by sampling the sixth imaging beam at a fourth value of the third scanning angle.

[0279] (22) An imaging method comprising: reflecting a first imaging beam from a target area to a first image sensor of a first camera using a scanning mirror structure having at least one mirror surface to capture a first set of oblique images along a first scanning path of a target area, wherein the first camera comprises a first lens for focusing the first imaging beam onto the first image sensor; reflecting a second imaging beam from a target area to a second image sensor of a second camera using a scanning mirror structure to capture a second set of oblique images along a second scanning path of the target area, wherein the second camera comprises a second lens for focusing the second imaging beam onto the second image sensor; rotating the scanning mirror structure around a scanning axis based on a scanning angle, wherein at least one of the elevation angle and azimuth angle of each of the first and second imaging beams changes according to the scanning angle; setting the optical axes of each of the first and second cameras at an oblique angle with respect to the scanning axis; and sampling the first and second imaging beams at the value of the scanning angle.

[0280] (23) The method according to (22), wherein at least one mirror surface includes a first mirror surface and a second mirror surface substantially opposite to the first mirror surface, and the method includes reflecting a first imaging beam from the first mirror surface and reflecting a second imaging beam from the second mirror surface.

[0281] (24) The method according to any one of (22) to (23), wherein the scanning angle for the first camera is the same as the scanning angle for the second camera.

[0282] (25) The method according to any one of (22) to (24), comprising taking a first set of oblique images and a second set of oblique images at the same time.

[0283] (26) The method according to any one of (22) to (25), comprising determining the geometry of at least one mirror surface at least in part on at least one of one of one or more predetermined orientations of the image sensor of the first camera and one or more predetermined orientations of the image sensor of the second camera, and a set of scanning angles of the scanning mirror structure.

[0284] (27) The method according to any one of (22) to (26), wherein the scanning mirror structure is symmetric with respect to the scanning axis.

[0285] (28) The method according to any one of (22) to (27), wherein the scanning mirror structure is asymmetric with respect to the scanning axis.

[0286] (29) The method according to any one of (22) to (28), wherein the scanning angle is the inclination angle of the scanning mirror structure.

[0287] (30) The method according to any one of (22) to (29), comprising determining the step of the inclination angle based on the size of the image sensor and the focal lengths of the first and second cameras.

[0288] (31) The method according to any one of (22) to (30), wherein the first camera and the second camera are inclined at a predetermined angle toward the scanning mirror structure.

[0289] (32) The method according to any one of (22) to (31), wherein the given angle is substantially 45 degrees.

[0290] (33) The method according to any one of (22) to (32), wherein the first scan path and the second scan path are symmetric.

[0291] (34) The method according to any one of (22) to (33), wherein the azimuth angle of the first camera is substantially 180 degrees from the azimuth angle of the second camera.

[0292] (35) The method according to any one of (22) to (34), wherein the first scan path and the second scan path are curved.

[0293] (36) The method according to any one of (22) to (35), further comprising capturing a vertical image using at least one third camera and at least one mirror configured to direct a third imaging beam from a region of interest, corresponding to the vertical image, toward at least one third camera.

[0294] (37) The method according to any one of (22) to (36), further comprising capturing a third set of oblique images along a third scanning path over a target area using a third camera, wherein the third camera is equipped with a lens that focuses a third imaging beam reflected from a scanning mirror structure onto the image sensor of the third camera.

[0295] (38) The method according to any one of (22) to (37), further comprising capturing a fourth set of oblique images along a fourth scanning path over a target area using a fourth camera, wherein the fourth camera is equipped with a lens that focuses a fourth imaging beam reflected from a scanning mirror structure onto the image sensor of the fourth camera.

[0296] (39) The method according to any one of (22) to (38), further comprising capturing a third set of images using a third camera and a second scanning mirror structure configured to direct a third imaging beam corresponding to the third set of images so as to be received by the third camera.

[0297] (40) The method according to any one of (22) to (39), further comprising capturing a fourth set of images using a fourth camera and a third scanning mirror structure configured to orient a fourth imaging beam corresponding to the fourth set of images so as to be received by the fourth camera.

[0298] (41) Reflecting a third imaging beam from a target region to a third image sensor of a third camera using a second scanning mirror structure having at least one mirror surface to capture a third set of oblique images along a third scanning path of the target region, wherein the third camera comprises a third lens that focuses the third imaging beam onto the third image sensor, and reflecting a fourth imaging beam from a target region to a fourth image sensor of a fourth camera using the second scanning mirror structure to capture a fourth set of oblique images along a fourth scanning path of the target region. The method according to any one of (22) to (40), wherein the fourth camera comprises a fourth lens for focusing the fourth imaging beam onto a fourth image sensor; a second scanning mirror structure is rotated around a second scanning axis based on a second scanning angle, such that at least one of the elevation and azimuth angles of each of the third and fourth imaging beams changes according to the second scanning angle; the optical axes of each of the third and fourth cameras are set at an oblique angle with respect to the second scanning axis; and the third and fourth imaging beams are sampled at the value of the second scanning angle.

[0299] (42) Reflecting a fifth imaging beam from a target region to a fifth image sensor of a fifth camera using a third scanning mirror structure having at least one mirror surface to capture a fifth set of oblique images along a fifth scanning path of the target region, wherein the fifth camera comprises a fifth lens that focuses the fifth imaging beam onto the fifth image sensor, and reflecting a sixth imaging beam from a target region to a sixth image sensor of a sixth camera using a third scanning mirror structure to capture a sixth set of oblique images along a sixth scanning path of the target region. The method according to any one of (22) to (41), wherein the sixth camera comprises a sixth lens for focusing the sixth imaging beam onto the sixth image sensor; rotating a third scanning mirror structure around a third scanning axis based on a third scanning angle, such that at least one of the elevation and azimuth angles of each of the fifth and sixth imaging beams changes according to the third scanning angle; setting the optical axes of each of the fifth and sixth cameras at an oblique angle with respect to the third scanning axis; and sampling the fifth and sixth imaging beams at the value of the third scanning angle.

[0300] (43) An imaging system mounted on a vehicle, comprising: a first camera configured to capture a first set of oblique images along a first scanning path over a target area; a scanning mirror structure having at least one mirror surface; a drive device coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a processing circuit configured to set the scanning angle of the scanning mirror structure, at least in part, based on the yaw angle of the vehicle, wherein the first camera has an optical axis set at an oblique angle with respect to the scanning axis and includes a lens that focuses a first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera, the azimuth angle of the first imaging beam captured by the first camera changes according to the scanning angle and the yaw angle of the vehicle, and the image sensor of the first camera captures a first set of oblique images along a first scanning path by sampling the first imaging beam at the value of the scanning angle.

[0301] (44) The system according to (43), further comprising a second camera configured to capture a second set of oblique images along a second scanning path over a target area, the second camera having an optical axis set at an oblique angle with respect to the scanning axis and a lens that focuses a second imaging beam reflected from a scanning mirror structure onto the image sensor of the second camera, the azimuth angle of the second imaging beam changing in accordance with the scanning angle and the yaw angle of the vehicle, and the image sensor of the second camera capturing a second set of oblique images along the second scanning path by sampling the second imaging beam at a second value of the scanning angle.

[0302] (45) The system according to any one of (43) to (44), wherein the scanning mirror structure has opposing first and second mirror surfaces, the first mirror surface reflecting a first imaging beam to a first camera and the second mirror surface reflecting a second imaging beam to a second camera.

[0303] (46) The system according to any one of (43) to (45), wherein the processing circuit is configured to set a scanning angle based on the difference between the yaw angle of the vehicle and a preferred yaw angle.

[0304] (47) The system according to any one of (43) to (46), wherein the preferred yaw angle is zero.

[0305] (48) The system according to any one of (43) to (47), wherein the processing circuit corrects the scanning angle in the direction opposite to the yaw angle of the vehicle, based on half the difference between the yaw angle of the vehicle and a preferred yaw angle.

[0306] (49) The system according to any one of (43) to (48), wherein the vehicle is a flying vehicle, and the processing circuit adjusts the scanning angle in consideration of different yaw angles of the flying vehicle in one or more flight lines, or between flight lines, with respect to at least one of them.

[0307] (50) The system according to any one of paragraphs (43) to (49), further comprising a stabilization platform configured to compensate for roll and pitch rather than yaw of the vehicle, wherein the imaging system is located within the stabilization platform.

[0308] (51) A method comprising: reflecting a first imaging beam from a target area to a first image sensor of a first camera using a scanning mirror structure having at least one mirror surface to capture a first set of oblique images along a first scanning path of a target area, the first camera comprising a lens for focusing the first imaging beam onto the first image sensor; rotating the scanning mirror structure around a scanning axis based on a scanning angle, the value of which is determined at least in part based on the yaw angle of a vehicle including the scanning mirror structure, and the azimuth angle of the first imaging beam captured by the first camera varies according to the scanning angle and the yaw angle of the vehicle; and sampling the first imaging beam at the value of which is scanning angle.

[0309] (52) The method according to (51), comprising: using a scanning mirror structure to reflect a second imaging beam from a target area to a second image sensor of a second camera in order to capture a second set of oblique images along a second scanning path of a target area, wherein the azimuth angle of the second imaging beam varies according to the scanning angle and the yaw angle of the vehicle, and the second camera comprises a second lens that focuses the second imaging beam onto the second image sensor, the method comprising reflecting and sampling the second imaging beam at a value of the scanning angle.

[0310] (53) The method according to any one of (51) to (52), wherein the scanning mirror structure has opposing first and second mirror surfaces, and the method includes reflecting a first imaging beam to a first camera and simultaneously reflecting a second imaging beam to a second camera.

[0311] (54) The method of any one of (51) to (53), which includes determining a value for the scanning angle based at least in part on the difference between the yaw angle of the vehicle and a preferred yaw angle.

[0312] (55) The method according to any one of (51) to (54), wherein the preferred yaw angle is zero.

[0313] (56) The method according to any one of (51) to (55), further comprising correcting the scanning angle in a direction opposite to the yaw angle of the vehicle, based on half the difference between the yaw angle of the vehicle and a preferred yaw angle.

[0314] (57) The method of any one of (51) to (56), further comprising adjusting the scanning angle in one or more flight lines or between flight lines, taking into account different yaw angles of the vehicle, wherein the vehicle is an aerial vehicle.

[0315] (58) The method of any one of (51) to (57), further comprising using a stabilization platform to compensate for roll and pitch, rather than yaw, of the vehicle.

[0316] (59) An imaging system comprising: a camera configured to capture a set of oblique images along a scanning path over a target area; a scanning mirror structure having at least one surface for receiving light from the target area, the at least one surface having at least one mirror portion, and at least one second portion being composed of a low-reflectivity material disposed on the outer periphery of the first mirror portion, the low-reflectivity material having a lower reflectivity than the first mirror portion; and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a rotation axis based on a scanning angle, wherein the camera comprises a lens for focusing an imaging beam reflected from at least one surface of the scanning mirror structure onto the camera's image sensor, the at least one first mirror portion being configured to reflect light from the target area over a set of scanning angles selected to produce a set of oblique images, and the at least one second portion being configured to wrap around the first mirror portion and block light received by the camera at scanning angles exceeding the set of scanning angles, and the camera's image sensor captures a set of oblique images along a scanning path by sampling the imaging beam at the values ​​of the scanning angles.

[0317] (60) The system according to (59), wherein at least a portion of the low-reflectivity material comprises a plurality of sections that are symmetrically paired around an axis of rotation.

[0318] (61) The system according to any one of (59) to (60), wherein at least one of the elevation and azimuth angles of the imaging beam captured by the camera changes with respect to the scanning angle, and at least one of the azimuth and elevation angles of the light that will wrap around the second portion is independent of the scanning angle.

[0319] (62) The system according to any one of (59) to (61), wherein the scanning mirror structure is convex and the low-reflection material is non-convex.

[0320] (63) The system according to any one of (59) to (62), wherein the low-reflectance material is configured to prevent specular reflection.

[0321] (64) The system according to any one of (59) to (63), wherein the second part is configured to block the light beam from the area of ​​interest that will generate the ghost image.

[0322] (65) The system according to any one of paragraphs (59) to (64), wherein the low-reflection material is configured to prevent incident light from being reflected towards the camera and focused onto the image sensor.

[0323] (66) An imaging system housed in a vehicle, comprising: a camera configured to capture a set of images along a scanning path over a target area; a scanning mirror structure having at least one mirror surface; and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the camera comprises a lens that focuses an imaging beam reflected from the scanning mirror structure onto the camera's image sensor, at least one of the elevation angle and azimuth angle of the imaging beam captured by the camera changes according to the scanning angle, the camera's image sensor captures a set of images along the scanning path by sampling the imaging beam at a value of the scanning angle, the illuminance of the image sensor by the imaging beam is reduced by at least one of partial occlusion by a constrained space on which the imaging system is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of scanning angles, and the value of the scanning angle along the scanning path is selected based on a model representing the illuminance of the image sensor by the imaging beam.

[0324] (67) The system according to (66), wherein the step size of the scanning angle value of the scanning mirror structure depends on at least one of the vehicle's yaw angle, the vehicle's roll, the vehicle's pitch, the geometry of the scanning mirror structure, the scanning angle, and the geometry of the constrained space.

[0325] (68) The system according to any one of (66) to (67), wherein the set of images is oblique images, and the step size of the scan angle values ​​relative to the scanning mirror structure has a first set of values ​​relative to a first set of scan angles, and the step size of the scan angle values ​​relative to the scanning mirror structure has a second set of values ​​relative to a second set of scan angles.

[0326] (69) The system according to any one of (66) to (68), wherein the set of images is oblique images, and the step size of the value of the scanning angle relative to the scanning mirror structure varies trigonally with the scanning angle.

[0327] (70) The set of images is oblique images, and the step size of the scanning angle value relative to the scanning mirror structure is smaller with respect to the azimuth direction in the case of greater aperture eclipse, as described in any one of (66) to (68).

[0328] (71) The system described in any one of paragraphs (66) to (70), wherein at least some of the images in the set of images partially overlap.

[0329] (72) The system according to any one of (66) to (71), wherein the range is determined by the mirror geometry.

[0330] (73) The system described in any one of (66) to (72), wherein the geometry of the mirror is determined by the value of the scanning angle.

[0331] (74) The system according to any one of (66) to (73), further comprising a circuit configured to cut off at least some portions of images in a set of images affected by vignetting, and then suture one or more images in the set of images after at least some portions affected by vignetting have been cut off.

[0332] (75) A method for reducing vignetting, comprising: reflecting an imaging beam from a target area to an image sensor of a camera using a scanning mirror structure having at least one mirror surface to capture a set of images along a scanning path of a target area, wherein the illuminance of the image sensor due to the imaging beam is reduced by at least one of partial occlusion by a constrained space on which the imaging system including the scanning mirror structure is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of scanning angles; rotating the scanning mirror structure around a scanning axis based on a scanning angle that changes at least one of the elevation angle and azimuth angle of the imaging beam, wherein the value of the scanning angle is at least partially based on a model of the illuminance of the image sensor due to the imaging beam; sampling the imaging beam at the value of the scanning angle; cropping at least some portions of images in the set of images affected by vignetting; and stitching one or more images in the set of images after the cropping has removed at least some portions affected by vignetting.

[0333] (76) The method of (75), comprising determining the step size of the scanning angle value of the scanning mirror structure based on at least one of the yaw angle of the vehicle including the imaging system, the roll of the vehicle, the pitch of the vehicle, the geometry of the scanning mirror structure, the scanning angle, and the geometry of the constrained space.

[0334] (77) The method according to any one of (75) to (76), wherein the set of images is oblique images, and the method comprises determining a step size of the values ​​of the scanning angle relative to the scanning mirror structure such that there is a first set of values ​​for a first set of scanning angles, and determining a step size of the values ​​of the scanning angle relative to the scanning mirror structure such that there is a second set of values ​​for a second set of scanning angles.

[0335] (78) The method according to any one of (75) to (77), wherein the set of images is oblique images, and the method comprises determining the step size of the value of the scanning angle relative to the scanning mirror structure to vary trigonally with the scanning angle.

[0336] (79) The method according to any one of (75) to (78), wherein the set of images is oblique images, and the method comprises determining the step size of the scanning angle value for the scanning mirror structure to be smaller with respect to the azimuth direction in the case of a larger aperture eclipse.

[0337] (80) The method described in any one of items (75) to (79), wherein at least some of the images in the set of images partially overlap.

[0338] (81) An imaging system installed within the confined space of a vehicle, comprising a camera configured to capture a set of images along a scanning path over a target area, the camera comprising an aperture, a lens, and an image sensor, the camera comprising a scanning mirror structure having at least one mirror surface, and a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle, wherein the lens focuses an imaging beam reflected from at least one mirror surface of the scanning mirror structure onto the image sensor, at least one of the azimuth angle and elevation angle of the imaging beam reflected to the camera changes according to the scanning angle, the image sensor of the camera captures a set of images along the scanning path by sampling the imaging beam at the value of the scanning angle, and the aperture of the camera is configured to be dynamically adjusted such that at least one of the following is true: the aperture remains within the projection geometry onto the aperture of at least one mirror surface when capturing a set of images, or the aperture remains within a region of light that is not obstructed by the confined space over the scanning path.

[0339] (82) The system according to (81), wherein the aperture is configured to be reduced at a scanning angle in which the mirror has rotated too much.

[0340] (83) The system according to any one of (81) to (82), wherein the aperture control mechanism in the camera masks a portion of the aperture that is not within the projection geometry of the scanning mirror.

[0341] (84) The system according to any one of (81) to (83), wherein one of the aperture sizes is reduced so as to remain within the projection geometry of at least one specular plane onto the aperture, and the shape of the aperture is modified so as to remain within the projection geometry of at least one specular plane onto the aperture.

[0342] (85) The system according to any one of (81) to (84), wherein the aperture is symmetrically adjusted so that it remains within the projection geometry of at least one specular surface onto the aperture.

[0343] (86) The system according to any one of (81) to (85), wherein the aperture is asymmetrically adjusted so that it remains within the projection geometry of at least one specular surface onto the aperture.

[0344] (87) The system according to any one of (80) to (86), wherein the scanning mirror structure is configured to block light from a target area outside the projection geometry of at least one mirror surface.

[0345] (88) A method for controlling an imaging system installed in a vehicle, comprising: reflecting an imaging beam from a target area to an image sensor of a camera using at least one mirror surface of a scanning mirror structure to capture a set of images along a scanning path of a target area, the camera comprising a lens and an aperture; rotating the scanning mirror structure about a scanning axis based on a scanning angle, the azimuth and elevation angles of the imaging beam reflected to the camera being varied according to the scanning angle; sampling the imaging beam at a value of the scanning angle; and dynamically adjusting the aperture of the camera such that the aperture remains within the projection geometry onto the aperture of at least one of the following: the aperture remains within a region of light that is not obstructed by a constraining space on the scanning path.

[0346] (89) The method according to (88), which includes reducing the aperture at a scanning angle in which the mirror has rotated too much.

[0347] (90) The method of any one of (88) to (89), comprising masking a portion of an aperture that is not in the projection geometry of at least one specular plane onto the aperture.

[0348] (91) The method according to any one of (88) to (90), comprising reducing the size of the aperture so that it remains within the projection geometry of at least one specular plane onto the aperture, or changing the shape of the aperture so that it remains within the projection geometry of at least one specular plane onto the aperture.

[0349] (92) The method according to any one of (88) to (91), comprising adjusting the aperture symmetrically so that it remains within the projection geometry of at least one specular plane onto the aperture.

[0350] (93) The method according to any one of (88) to (92), comprising adjusting the aperture asymmetrically so that it remains within the projection geometry of at least one specular plane onto the aperture.

[0351] (94) An imaging system installed within the confined space of a vehicle, comprising: a scanning mirror structure having at least one mirror surface; a camera configured to capture a set of images along a scanning path over a target area, wherein the camera comprises a lens that focuses an imaging beam reflected from at least one mirror surface of the scanning mirror structure onto the camera's image sensor; a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a circuit configured to form eclipse data in one or more scanning path arrangements due to a decrease in the illuminance of the image sensor caused by the imaging beam, and to update the pixel values ​​of one or more images in the set of images according to the eclipse data at the corresponding scanning angle, wherein at least one of the elevation angle and azimuth angle of the imaging beam captured by the camera changes according to the scanning angle, the camera's image sensor captures a set of images along the scanning path by sampling the imaging beam at the value of the scanning angle, and the decrease in the illuminance of the image sensor due to the imaging beam is caused by at least one of partial obstruction by the confined space in which the imaging system is installed, and the scanning angle of the scanning mirror structure being outside a predetermined range of scanning angles.

[0352] (95) The system according to (94), wherein the eclipse data is based on at least one of the vehicle's roll, vehicle's pitch, vehicle's yaw, the geometry of the scanning mirror structure, the camera's focal length, the image sensor's aspect ratio, the image sensor's pitch, and the image sensor's orientation.

[0353] (96) The system described in any one of items (94) to (95), wherein the vignetting data is a vignetting image.

[0354] (97) A method for reducing vignetting, comprising: reflecting an imaging beam from a target area to an image sensor of a camera using a scanning mirror structure having at least one mirror surface to capture a set of images along a scanning path of a target area, wherein the camera comprises a lens that focuses the imaging beam onto the image sensor; rotating the scanning mirror structure about a scanning axis based on a scanning angle, wherein at least one of the azimuth angle and elevation angle of the imaging beam changes with respect to the scanning angle; forming vignetting data for one or more arrangements along the scanning path due to partial occlusion of the imaging beam, wherein the reduction in illumination of the image sensor due to the imaging beam is caused by at least one of partial occlusion by a constrained space on which the imaging system including the scanning mirror structure is mounted, and the scanning angle of the scanning mirror structure being outside a predetermined range of the scanning angle; and updating the pixel values ​​of one or more images in the set of images according to the vignetting data.

[0355] (98) The method according to (97), wherein the eclipse data is based on at least one of the roll of the vehicle equipped with the imaging system, the pitch of the vehicle, the yaw of the vehicle, the geometry of the scanning mirror structure, the focal length of the camera, the aspect ratio of the image sensor, the pitch of the image sensor, and the orientation of the image sensor.

[0356] (99) The method according to any one of (97) to (98), wherein the vignetting data is a vignetting image.

[0357] (100) An imaging system comprising: a camera configured to capture an image on a target area from an imaging beam from the target area, the camera comprising an image sensor and a lens; one or more glass plates positioned between the image sensor and the lens of the camera; one or more first drive units coupled to each of the one or more glass plates; a scanning mirror structure having at least one mirror surface; a second drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around a scanning axis based on a scanning angle; and a motion compensation system configured to determine at least one of a plate rotation speed and a plate rotation angle based on the relative dynamics between the imaging system and the target area and the optical properties of one or more glass plates, and to control one or more first drive units to rotate one or more glass plates around one or more predetermined axes based on the corresponding plate rotation speed and plate rotation angle.

[0358] (101) The system according to (100), wherein the image sensor is exposed to the imaging beam in synchronization with the movement of one or more glass plates.

[0359] (102) The motion compensation system according to any one of (100) to (101), wherein the motion compensation system is configured to move one or more glass plates in succession when an image is taken by a camera.

[0360] (103) The system according to any one of (100) to (102), wherein the scanning axis of one or more second drive units is selected from substantially perpendicular to the optical axis of the camera and substantially parallel to the optical axis of the camera.

[0361] (104) The motion compensation system according to any one of (100) to (103), wherein the system is configured to acquire a region of interest in each of the captured images and to use the region of interest to estimate the pixel velocity.

[0362] (105) The motion compensation system according to any one of (100) to (104), wherein the motion compensation system is configured to estimate at least one of motion pixel velocity and attitude rate pixel velocity and to control one or more first drive units based on the motion pixel velocity and attitude rate pixel velocity.

[0363] (106) The attitude rate pixel velocity is the yaw rate pixel velocity in any one of the systems described in (100) to (105).

[0364] (107) The motion pixel velocity is the forward motion pixel velocity in any one of the systems described in (100) to (106).

[0365] (108) The motion compensation system according to any one of (100) to (107), wherein the motion compensation system is configured to control one or more first drive units based on at least one of the motion of the imaging system with respect to the area of ​​interest, the scanning angle, the projection geometry, the alignment of one or more glass plates, the properties of one or more glass plates, the optical properties of one or more glass plates, the alignment of the imaging system with respect to the flight path, and the rate of change of attitude of the imaging system with respect to the area of ​​interest.

[0366] (109) An imaging method comprising: reflecting an imaging beam from a target area to an image sensor of a camera using at least one mirror surface of a scanning mirror structure to capture a set of images along a scanning path of the target area, wherein the camera comprises a lens and an image sensor; capturing an image from the imaging beam from the target area reflected by at least one mirror surface using the image sensor of the camera; positioning one or more glass plates between the image sensor of the camera and the lens; determining a plate rotation speed and plate rotation angle based on one of the characteristics of the camera, the characteristics and positioning of one or more glass plates, and the relative dynamics between imaging and the target area; and rotating one or more glass plates around one or more predetermined axes based on the corresponding plate rotation speed and plate rotation angle.

[0367] (110) The method according to (109), wherein the image sensor is exposed to the imaging beam in synchronization with the movement of one or more glass plates.

[0368] (111) The method according to any one of (109) to (110), comprising moving one or more glass plates in sequence when taking an image with a camera.

[0369] (112) The method according to any one of (109) to (111), comprising obtaining a region of interest in each of the captured images and estimating the pixel velocity using the region of interest.

[0370] (113) The method according to any one of (109) to (112), comprising estimating at least one of motion pixel velocity and attitude rate pixel velocity, and controlling one or more first drive devices based on the motion pixel velocity and attitude rate pixel velocity.

[0371] (114) The method according to any one of (109) to (113), wherein determining at least one of the plate rotation speed and plate rotation angle is based on at least one of the camera movement with respect to the area of ​​interest, scanning angle, projection geometry, alignment of one or more glass plates, properties of one or more glass plates, optical properties of one or more glass plates, alignment with respect to the flight path, and rate of change of the camera's attitude with respect to the area of ​​interest. [Explanation of symbols]

[0372] 110 Flying Vehicles 111 Scan Patterns 112 scan patterns 113 Scan Patterns 114 scan patterns 115 Projection Geometry 116 grid lines 117 grid lines 118 grid lines 119 grid lines 121 Scan Patterns 122 scanning patterns 123 Scan Patterns 124 scan patterns 125 frame 126 frames 127 frames 128 frames 130 scan patterns 131 scan patterns 132 scan patterns 133 Scan Patterns 135 scan patterns 136 scan patterns 137 scan patterns 138 scanning patterns 139 scan patterns 140 scan patterns 141 scan patterns 142 scan patterns 150 scan patterns 151 scan patterns 152 scan patterns 153 scan patterns 155 scan patterns 156 scan patterns 157 scan patterns 160 scan patterns 161 scan patterns 162 scan patterns 163 scan patterns 165 scan patterns 170 scan patterns 171 scan patterns 172 scan patterns 173 scan patterns 175 scan patterns 176 scan patterns 177 scan patterns 178 scan patterns 179 scan patterns 180 scan patterns 210 Flightline Flightline 211 Flightline 212 Flightline 213 214 Flightline 215 Flightline 220 Turning Path 221 Turning Path 222 Turning Path 223 Turning Path 224 Turning Path 225 Turning path 226 Flightline spacing 230 View direction 231 View direction 232 View direction 233 View direction 234 View direction 235 View direction 236 Circle in the line of sight at a fixed elevation angle 237 Circle in the line of sight at a fixed elevation angle 238 Circle in the line of sight at a fixed elevation angle 240 lenses 241 Optical Plate 242 Optical Plates 243 Sensors 244 Optical Plate 245 Optical Plate 246 Optical Plates 250 Cameras 251 areas 252 Mirror 253 Investigation hole 254 Placement 255 beams 256 Mirror 257 beams 258 Placement 259 beams 260 Low reflective material 261 Beam 262 beams 263 Beam 265 Low reflective material 266 Mirror 271 Sensor placement 272 Sensor placement 273 Sensor placement 274 Sensor placement 275 Sensor placement 276 Sensor placement 277 Sensor placement 278 Sensor placement 279 Sensor placement 281 rotations 282 rotations 283 rotations 284 rotations 290 Rays 291 Optical Plate 292 Front 293 Rear 294 Curved paths in the line of sight within a hemisphere 295 Curved paths in the line of sight within a hemisphere 296 Curved paths in the line of sight within a hemisphere 297 Curved paths in the line of sight within a hemisphere 300 Scanning Camera System 301 Scanning drive unit 302 Scanning drive unit 303 Scanning drive unit 305 Survey hole 310 Camera 311 Camera 312 Scanning mirror structure 313 Scanning drive unit 314 Mirror surface 315 Mirror surface 316 Rotation axis 317 Low reflective material 321 Camera 322 Scanning drive unit 323 Primary Mirror 324 Secondary Mirror 325 Camera 326 Scanning drive unit 327 Primary Mirror 328 Secondary Mirror 350 Scanning drive unit 351 Scanning drive unit 352 Scanning drive unit 353 Scanning drive unit 354 Scanning Camera System 355 Scanning Camera System 356 Scanning drive unit 357 Primary Mirror 358 Secondary Mirror 360° Scanning Drive Unit 361 Scanning drive unit 362 Scanning drive unit 363 Scanning drive unit 364 Scanning Camera System 365 Camera 366 Scanning drive unit 367 Primary Mirror 368 Secondary Mirror 369 Cameras 370 Scanning drive unit 371 Scanning drive unit 372 Scanning drive unit 373 Scanning drive unit 375 Camera 376 Scanning drive unit 377 Primary Mirror 378 Secondary Mirror 379 Scanning Camera System 380 Scanning drive unit 381 Scanning Camera System 382 Scanning Camera System 383 Primary Mirror 384 Secondary Mirror 385 Scanning drive unit 386 Scanning drive unit 387 Scanning drive unit 388 Scanning drive unit 389 Scanning drive unit 390 Scanning drive unit 391 Scanning Camera System 392 Scanning Camera System 393 Scanning Camera System 394 Scanning Camera System 395 Scanning Camera System 401 Autopilot 402 Pilot Display 403 Pilot Input 404 GNSS receiver 405 System Control Unit 406 Data Storage Devices 407 Stabilization Platform 408 Scanning Camera System 409 IMU 410 Camera 411 Scanning drive unit 412 Scanning drive unit 413 Scanning mirror 414 Camera 415 Motion Compensation Unit 416 Camera 417 Motion Compensation Unit 430 scanning angle 431 Mirror drive mechanism 432 Mirror control device 433 Mirror Sensor 434 Scanning drive unit parameters 435 Motion Compensation Data 436 IMU attitude data 437 Mirror control data 438 Focal Data 439 pixel data 440 ROI Pixel Velocity Estimator 450 Geometry Estimator Modules 451 Projection Geometry 452 Forward Motion Pixel Velocity Estimator 453 forward motion pixel speed 454 Attitude Rate Pixel Velocity Estimator 455 Attitude Rate Pixel Speed 456 Attitude Rate Pixel Velocity Estimator 457 ROI Pixel Speed 458 Motion Compensation Control Device 459 Motion Compensation Sensor 460 Motion Compensation Drive System 461 Motion Compensation Calibration Data 462 Ground speed 463 Latitude / Longitude Data 464 Advanced Data 465 DEM data 466 IMU attitude data 467 SDU Geometry Data 468 IMU Attitude Rate 469 ROI images 470 Gimbal Angle 471 Survey hole geometry 472 Mirror Data 473 Caliber erosion data 474. Caliber erosion analysis process 475 Processing Steps 476 Refinement Step 477 Camera attitude, position, and additional data 478 3D surface reconstruction 479 Orthomosaic generation 480 Vignetting compensation 482 Orthomosaic 501 Scanning drive unit 502 Scanning mirror structure 503 Scanning drive unit 504 Mirror surface 505 Mirror surface 506 Camera 507 Camera 508 Camera 509 Scanning drive unit 510 Scanning Camera System 511 Scanning drive unit 512 Scanning mirror structure 513 Scanning drive unit 514 Mirror surface 515 Mirror surface 516 Camera 517 Camera 518 Camera 519 Camera 521 Scanning drive unit 522 Scanning mirror structure 523 Scanning drive unit 524 Mirror surface 525 Mirror surface 526 Camera 527 Camera 528 Camera 529 Camera 530 Scanning drive unit 531 Scanning Camera System 532 Scanning Camera System 535 Scanning drive unit 536 Scanning Camera System 537 Scanning Camera System

Claims

1. A first camera configured to capture a first set of oblique images along a first scanning path over a target area, A second camera configured to capture a second set of oblique images along a second scanning path on the target region, A scanning mirror structure having at least one mirror surface, The device comprises a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, The first camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera. The second camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the second imaging beam reflected from the scanning mirror structure onto the image sensor of the second camera. At least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, change according to the scanning angle. The image sensor of the first camera captures a first set of oblique images along the first scanning path by sampling the first imaging beam at a first value of the scanning angle. The image sensor of the second camera captures a second set of oblique images along the second scanning path by sampling the second imaging beam at a second value of the scanning angle. An imaging system in which the first scanning path and the second scanning path are curved.

2. The at least one mirror surface includes a first mirror surface and a second mirror surface substantially opposite the first mirror surface. The imaging system according to claim 1, wherein the first imaging beam is reflected from the first mirror surface, and the second imaging beam is reflected from the second mirror surface.

3. The imaging system according to claim 1, wherein the first scanning angle with respect to the first camera is the same as the first scanning angle with respect to the second camera.

4. The imaging system according to claim 1, wherein the image sensor of the first camera and the image sensor of the second camera simultaneously capture images from the first set of oblique images and the second set of oblique images.

5. The external shape of the at least one mirror surface is at least partially, One or more predetermined orientations of the image sensor of the first camera and one or more predetermined orientations of the image sensor of the second camera, and Setting the scanning angle of the scanning mirror structure The imaging system according to claim 1, determined based on at least one of the following.

6. The imaging system according to claim 1, wherein the scanning mirror structure is symmetrical with respect to the scanning axis.

7. The imaging system according to claim 1, wherein the scanning angle is the inclination angle of the scanning mirror structure.

8. The imaging system according to claim 7, wherein the step of the inclination angle is determined based on the size of the image sensor and the focal lengths of the first camera and the second camera.

9. The first camera and the second camera are inclined at a predetermined angle toward the scanning mirror structure, The imaging system according to claim 1, wherein the predetermined angle is substantially 45 degrees.

10. The imaging system according to claim 1, wherein the first scanning path and the second scanning path are symmetrical.

11. A first camera configured to capture a first set of oblique images along a first scanning path over a target area, A second camera configured to capture a second set of oblique images along a second scanning path on the target region, A scanning mirror structure having at least one mirror surface, The device comprises a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, The first camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera. The second camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the second imaging beam reflected from the scanning mirror structure onto the image sensor of the second camera. At least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, change according to the scanning angle. The image sensor of the first camera captures a first set of oblique images along the first scanning path by sampling the first imaging beam at a first value of the scanning angle. The image sensor of the second camera captures a second set of oblique images along the second scanning path by sampling the second imaging beam at a second value of the scanning angle. An imaging system in which the azimuth angle of the first camera is substantially 180 degrees from the azimuth angle of the second camera.

12. A first camera configured to capture a first set of oblique images along a first scanning path over a target area, A second camera configured to capture a second set of oblique images along a second scanning path on the target region, A scanning mirror structure having at least one mirror surface, The device comprises a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, The first camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera. The second camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the second imaging beam reflected from the scanning mirror structure onto the image sensor of the second camera. At least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, change according to the scanning angle. The image sensor of the first camera captures a first set of oblique images along the first scanning path by sampling the first imaging beam at a first value of the scanning angle. The image sensor of the second camera captures a second set of oblique images along the second scanning path by sampling the second imaging beam at a second value of the scanning angle. A third camera configured to capture vertical images, An imaging system further comprising: at least one mirror configured to direct a third imaging beam, corresponding to the vertical image, toward the at least one third camera.

13. A first camera configured to capture a first set of oblique images along a first scanning path over a target area, A second camera configured to capture a second set of oblique images along a second scanning path on the target region, A scanning mirror structure having at least one mirror surface, The device comprises a drive unit coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, The first camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the first imaging beam reflected from the scanning mirror structure onto the image sensor of the first camera. The second camera has an optical axis set at an oblique angle with respect to the scanning axis, and includes a lens that focuses the second imaging beam reflected from the scanning mirror structure onto the image sensor of the second camera. At least one of the elevation angle and azimuth angle of the first imaging beam, and at least one of the elevation angle and azimuth angle of the second imaging beam, change according to the scanning angle. The image sensor of the first camera captures a first set of oblique images along the first scanning path by sampling the first imaging beam at a first value of the scanning angle. The image sensor of the second camera captures a second set of oblique images along the second scanning path by sampling the second imaging beam at a second value of the scanning angle. A third camera configured to capture a third set of images, An imaging system further comprising: a second scanning mirror structure configured to direct a third imaging beam, corresponding to a third set of images, so that it is received by the third camera.

14. A fourth camera configured to capture a fourth set of images, The imaging system according to claim 13, further comprising: a third scanning mirror structure configured to direct a fourth imaging beam corresponding to a fourth set of images so as to be received by the fourth camera.

15. A step of capturing a first set of oblique images along a first scanning path of a target area by using a scanning mirror structure having at least one mirror surface to reflect a first imaging beam from a target area onto a first image sensor of a first camera, wherein the first camera includes a first lens that focuses the first imaging beam onto the first image sensor. A step of using the scanning mirror structure to reflect a second imaging beam from the target area onto a second image sensor of a second camera to capture a second set of oblique images along a second scanning path of the target area, wherein the second camera includes a second lens that focuses the second imaging beam onto the second image sensor. A step of rotating the scanning mirror structure around the scanning axis based on the scanning angle, wherein at least one of the elevation angle and azimuth angle of the first imaging beam and the second imaging beam changes according to the scanning angle. The steps include setting the optical axes of the first camera and the second camera at an oblique angle with respect to the scanning axis, The step includes sampling the first imaging beam and the second imaging beam at the value of the scanning angle, An imaging method in which the first scanning path and the second scanning path are curved.

16. An imaging system, A camera configured to capture an image of a target region from an imaging beam from the target region, the camera comprising an image sensor and a lens, One or more glass plates positioned between the image sensor and the lens of the camera, One or more first drive devices coupled to each of the one or more glass plates, A scanning mirror structure having at least one mirror surface, A second drive device coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, Based on the relative dynamics between the imaging system and the target region and the optical properties of the one or more glass plates, at least one of the plate rotation speed and plate rotation angle is determined. An imaging system comprising: a motion compensation system configured to control one or more first drive devices to rotate one or more glass plates around one or more predetermined axes based on at least one of a corresponding plate rotation speed and plate rotation angle.

17. The imaging system according to claim 16, wherein the image sensor is exposed to the imaging beam in synchronization with the movement of the one or more glass plates.

18. The motion compensation system is configured to continuously move one or more glass plates when the camera captures an image, according to claim 16.

19. The scanning axis of the one or more first drive devices is The optical axis of the camera is substantially perpendicular to the optical axis, and The imaging system according to claim 16, wherein the system is substantially parallel to the optical axis of the camera, or the system is substantially parallel to the optical axis of the camera, as selected from either of these two options.

20. The motion compensation system described above is: Estimate at least one of the motion pixel velocity and the attitude rate pixel velocity, The imaging system according to claim 16, configured to control one or more first drive devices based on one of the motion pixel velocity and the attitude rate pixel velocity.

21. The imaging system according to claim 20, wherein the attitude rate pixel velocity is the yaw rate pixel velocity.

22. The imaging system according to claim 20, wherein the motion pixel velocity is the forward motion pixel velocity.

23. The motion compensation system controls the one or more first drive devices. The movement of the imaging system with respect to the target region, Scanning angle, Projected external shape, Alignment of the one or more glass plates, Characteristics of the one or more glass plates, Optical properties of one or more glass plates, Alignment of the imaging system with respect to the flight path, and The imaging system according to claim 16, configured to perform the imaging based on at least one of the rates of change of the orientation of the imaging system with respect to the target region.

24. A step of capturing a set of images along a scanning path of a target area by using at least one mirror surface of a scanning mirror structure to reflect an imaging beam from a target area to the image sensor of a camera, wherein the camera comprises a lens and an image sensor. The steps include: using the image sensor of the camera to capture an image with the imaging beam from the target area reflected by the at least one mirror surface; The steps include positioning one or more glass plates between the image sensor of the camera and the lens, A step of determining the plate rotation speed and plate rotation angle based on one of the characteristics of the camera, the characteristics and positioning of the one or more glass plates, and the relative dynamics between the imaging beam and the target area, The step of rotating one or more glass plates around one or more predetermined axes based on corresponding plate rotation speeds and plate rotation angles, At least one of the plate rotation speed and the plate rotation angle is The movement of the camera with respect to the target region, Scanning angle, Projected external shape, Alignment of the one or more glass plates, Characteristics of the one or more glass plates, The optical properties of the one or more glass plates, Alignment regarding flight paths, and An imaging method determined based on at least one of the rates of change of the camera's posture with respect to the target area.

25. An imaging system, A camera configured to capture an image of a target region from an imaging beam from the target region, the camera comprising an image sensor and a lens, One or more glass plates positioned between the image sensor and the lens of the camera, One or more first drive devices coupled to each of the one or more glass plates, A scanning mirror structure having at least one mirror surface, A second drive device coupled to the scanning mirror structure and configured to rotate the scanning mirror structure around the scanning axis based on the scanning angle, Based on the rate of change in the orientation of the imaging system with respect to the target region and the optical properties of the one or more glass plates, at least one of the plate rotation speed and plate rotation angle is determined. An imaging system comprising: a motion compensation system configured to control one or more first drive devices to rotate one or more glass plates around one or more predetermined axes based on at least one of a corresponding plate rotation speed and plate rotation angle.

Images