System and method for registering terrain data and image data

Abstract

A system and method for registering terrain data and image data are provided, the method comprising the steps of receiving terrain data and image data. The method further comprises the steps of determining a position of a light source based on the image data. The method further comprises the steps of creating a hillshade representation of the terrain data based on the terrain data and the position of the light source. The method further comprises the steps of identifying a portion of the hillshade representation and a portion of the image data that correspond to each other. The method further comprises the steps of comparing the portion of the hillshade representation with the portion of the image data. The method further comprises the steps of determining a vector control between the portion of the hillshade representation and the portion of the image data based on the comparison. The method further comprises the steps of applying the vector control to the image data to generate updated image data.

Description

Technical Field

[0001] The present disclosure is directed to systems and methods for co-registering terrain data and image data. Background Art

[0002] Terrain data and image data are different types of data that can represent a target area. As used herein, "terrain data" refers to elevation information in a gridded (e.g., image) format where each pixel corresponds to a point on a coordinate system. For example, the point may include an X value, a Y value, and / or a Z value in a Cartesian coordinate system. Terrain data can be captured using LIDAR, radar, stereo image triangulation, ground surveying, or structure from motion image processing techniques.

[0003] As used herein, "image data" refers to an electro-optical image (e.g., a picture) captured / collected by a source such as a satellite or a camera on an aircraft. The image data may be in the visible spectrum or the infrared spectrum. In a specific example, the target area may include a portion of the Earth's surface. As will be apparent, the portion of the Earth's surface may include varying shapes and elevations, such as mountains, trees, buildings, etc.

[0004] Both topographic data and image data can include multiple pixels. However, pixels in topographic data and image data are often misaligned. For example, a first pixel in the topographic data corresponding to a particular point on a mountain range may be misaligned with a corresponding first pixel in the image data corresponding to the same point on the mountain range. The topographic data and / or image data can be shifted so that the first pixels are aligned; however, at that time, a second pixel in the topographic data corresponding to the particular point on the mountain range may be misaligned with a corresponding second pixel in the image data corresponding to the same point on the mountain range. Therefore, it may be difficult to align corresponding pairs of pixels. If pixels are misaligned when combining topographic data and image data to generate an image (e.g., a map), such misalignment may reduce the quality and accuracy of the image. Summary of the Invention

[0005] A method for aligning terrain data and image data is disclosed. The method includes the following steps: receiving terrain data and image data. The method also includes the following steps: determining the position of a light source based on the image data. The method also includes the following steps: creating a hillshade representation of the terrain data based on the terrain data and the position of the light source. The method also includes the following steps: identifying a portion of the hillshade representation and a portion of the image data that correspond to each other. The method also includes the following steps: comparing the portion of the hillshade representation with the portion of the image data. The method also includes the following steps: determining a vector control between the portion of the hillshade representation and the portion of the image data based on the comparison. The method also includes the following steps: applying the vector control to the image data to generate updated image data.

[0006] In another implementation, the method includes the steps of receiving terrain data and receiving image data. The image data is captured by a camera on an aircraft or satellite. The method further includes the steps of determining the position of the sun at the time the image data was captured based on shadows in the image data. The position of the sun includes its azimuth and altitude. The method further includes the steps of creating a hillshade representation of the terrain data based on the terrain data and the position of the sun. The method further includes the steps of identifying a portion of the hillshade representation and a portion of the image data that correspond to each other. The method further includes the steps of comparing the portion of the hillshade representation with the portion of the image data using an image matching technique, a pattern matching technique, or an object matching technique that outputs a plurality of coordinate pairs. Each coordinate pair includes a first pixel in the image data and a second pixel in the hillshade representation. The first pixel and the second pixel both correspond to the same point on the surface of the Earth. The first pixel and the second pixel are misaligned. The method further includes the steps of determining a vector control for each coordinate pair. The vector control includes a distance and a bearing from the first pixel to the second pixel. The method further includes applying the vector control to the image data to move the first pixel along the orientation by the distance to become aligned with the second pixel, thereby generating updated image data.

[0007] A computing system is also disclosed. The computing system includes one or more processors and a memory system. The memory system includes one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform operations. The operations include receiving terrain data and receiving image data. The image data is captured by a camera on an aircraft or satellite. The operations also include determining the position of the sun at the time the image data was captured based on shadows in the image data. The sun's position includes its azimuth and altitude. The operations also include creating a hillshade representation of the terrain data based on the terrain data and the sun's position. The operations also include identifying a portion of the hillshade representation and a portion of the image data that correspond to each other. The operations also include comparing the portion of the hillshade representation with the portion of the image data using an image matching technique, a pattern matching technique, or an object matching technique that outputs a plurality of coordinate pairs. Each coordinate pair includes a first pixel in the image data and a second pixel in the hillshade representation. The first pixel and the second pixel both correspond to the same point on the Earth's surface. The first pixel and the second pixel are not aligned. The operations further include determining a vector control for each coordinate pair. The vector control includes a distance and an orientation from the first pixel to the second pixel. The operations further include applying the vector control to the image data to move the first pixel by the distance along the orientation to become aligned with the second pixel, thereby generating updated image data.

[0008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present teachings, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate various aspects of the present teachings and, together with the description, serve to explain the principles of the present teachings.

[0010] Figure 1 A flow chart illustrating a method of registering terrain data and image data according to an implementation is illustrated.

[0011] Figure 2A An example of terrain data according to an implementation is illustrated.

[0012] Figure 2B An example of image data according to an implementation is illustrated.

[0013] Figure 3 Topographic data is illustrated alongside (ie, compared to) image data according to an implementation.

[0014] Figure 4 A hillshade representation of terrain data according to an implementation is illustrated.

[0015] Figure 5A illustrates a portion of a hillshade representation according to an implementation, and Figure 5B Corresponding (eg, overlapping) portions of image data according to an implementation are illustrated.

[0016] Figure 6 The portion of the hillshade is illustrated juxtaposed with (ie, compared to) the portion of the image data, according to an implementation.

[0017] Figure 7 Illustrated is topographic data (from a) side-by-side with (i.e., compared to) updated image data according to one implementation. Figure 2A ).

[0018] Figure 8 1 illustrates a hillshade representation (from a 3D image) alongside (i.e., compared to) updated image data according to one implementation. Figure 4 ).

[0019] Figure 9 A plot of vector control values used to generate updated image data according to an implementation is illustrated.

[0020] Figure 10 A schematic diagram illustrating a computing system for performing at least a portion of the method according to an implementation is illustrated.

[0021] It should be noted that some details of these figures have been simplified and drawn to facilitate understanding rather than maintaining strict structural accuracy, detail, and scale. DETAILED DESCRIPTION

[0022] The present teachings will now be described in detail, with examples of the present teachings being illustrated in the accompanying drawings. Like reference numerals are used throughout the drawings to designate identical elements. In the following description, the accompanying drawings, which form a part of this description, are described, and in the accompanying drawings, specific examples in which the present teachings may be practiced are shown by way of illustration. Therefore, the following description is merely illustrative.

[0023] Figure 1 A flow chart illustrating a method 100 for registering terrain data and image data according to one implementation is provided. As used herein, "register" refers to aligning or matching pixels in two images (e.g., terrain data and image data). Method 100 may include the following steps: At 102, terrain data and image data are captured or received. Figure 2Aillustrates an example of terrain data 200 according to an implementation, and Figure 2B An example of image data 250 according to one implementation is illustrated. Terrain data 200 and image data 250 may both represent the same target area. For example, terrain data 200 and image data 250 may both represent the same portion of the Earth's surface. As shown, the area represented in terrain data 200 and image data 250 is a mountainous area. In the specific example shown, terrain data 200 includes a ridge 210 having one or more arms 211 to 216, and image data 250 also includes the same ridge 260 having one or more arms 261 to 266.

[0024] Figure 3 Terrain data 200 is illustrated alongside (i.e., compared to) image data 250 according to one implementation. As shown, there is a misalignment 300 between the terrain data 200 and the image data 250. In this example, the misalignment 300 between ridge 210 in the terrain data 200 and ridge 260 in the image data 250 may be approximately 800 meters. Misalignment 300 may negatively impact the appearance and accuracy of an orthoimage generated using the terrain data 200 and the image data 250. In turn, the negative impact on the appearance and accuracy of the orthoimage may negatively impact any spatial data derived from the orthoimage source. In other words, an orthoimage can be created by correcting the image data 250 against the terrain data 200, or by correcting the terrain data 200 against the image data 250. The actual quality of the orthoimage may not yet be assessed beyond spatial correction and may depend on the transformation used to distort the image once vector control has been created, as discussed below. An orthophoto is a remote sensing image, such as an aerial or satellite photograph, that has been corrected to remove distortion caused by topographic relief (e.g., topography) and collection parameters (e.g., sensor lens distortion, collection tilt). Orthophotos preserve scale to enable accurate distance measurements.

[0025] Back to Figure 1 The method 100 may further include the following steps: at 104, determining a position of a light source (e.g., the sun) based on the image data 250. More specifically, the method may include the following steps: determining the sun azimuth, the sun altitude (also known as the sun elevation), the sun zenith, or a combination thereof based at least in part on metadata of the image data 250.

[0026] The method 100 may further include the following steps: at 106 , creating a hill shadow using the terrain data 200 and the position of the sun. Figure 4 A hillshade 400 is illustrated, created using terrain data 200 and the position of the sun, according to one implementation. Hillshade 400 is a two-dimensional (2D) or three-dimensional (3D) representation of terrain data 200 that takes the position of the sun into account for shadows and shading. More specifically, hillshade 400 simulates a determined sun position to create shadows and shading, thereby adding a relief-like appearance to terrain data 200. To match terrain data 200 to image data 250 as closely as possible, hillshade 400 may be created using sun azimuth, sun altitude, and / or sun zenith.

[0027] Back to Figure 1 The method 100 may further include determining or identifying, at 108 , a portion of the hillshade 400 that corresponds to a portion of the image data 250 . Figure 5A illustrates a portion 500 of a hillshade 400 according to an implementation, and Figure 5B 2. The corresponding (eg, overlapping) portion 550 of the image data 250 is illustrated according to an implementation. The step of determining the portion 500 of the hillshade 400 corresponding to the portion 550 of the image data 250 may first include: Figure 4 ) around the image data 250 and placing boundary coordinates 270 (see Figure 2B ).

[0028] A portion 500 of hillshade 400 (e.g., within bounding coordinates 470) and a portion 550 of image data 250 (e.g., within bounding coordinates 270) that correspond to (e.g., overlap) each other are then determined / identified. This determination / identification can be performed by calculating spatial overlap (also known as clipping). In at least one implementation, even if portions 500 and 550 overlap, one or more pixels of portion 500 of hillshade 400 may be misaligned with one or more pixels of portion 550 of image data 250.

[0029] Then, if Figure 5A As shown, boundary coordinates 510 may be placed around portion 500 of hillshade 400, and as Figure 5B As shown, boundary coordinates 560 may be placed around portion 550 of image data 250. The data at boundary coordinates 510 ( Figure 5A Center) and boundary coordinates 470( Figure 4Similarly, the remaining portion of the hill shadow 400 between the boundary coordinates 560 ( Figure 5B Center) and boundary coordinates 270( Figure 2B The remaining portion of the image data 250 between .

[0030] Back to Figure 1 The method 100 may further include the following steps: at 110 , comparing the portion 500 of the hillshade 400 with the portion 550 of the image data 250 . Figure 6 Portion 500 of hillshade 400 is illustrated alongside (ie, compared to) portion 550 of image data 250 , according to an implementation. Figure 6 Except that the terrain data 200 has been replaced by the mountain shadow 400, the other parts are similar to Figure 3 .therefore, Figure 6 Can be compared Figure 3 Misalignment 300 is more clearly shown. Comparison of portion 500 of hillshade 400 with portion 550 of image data 250 (or vice versa) may be performed using image matching techniques, pattern matching techniques, or object matching techniques.

[0031] The matching technique can output one or more coordinate pairs. As used herein, a coordinate pair refers to a first coordinate in portion 500 of the hillshade 400 and a second coordinate in portion 550 of the image data 250. The first coordinate and the second coordinate can both correspond to the same 2D or 3D point in the target area (e.g., the same point on the surface of the Earth). The coordinate pair can be in geographic space or image space. If the coordinate pair is in image space (e.g., image coordinates), they can be converted to geographic coordinates. In one example, the coordinate pair can be or include an arrow (e.g., an arrow from a first coordinate to a second coordinate, or an arrow from a second coordinate to a first coordinate).

[0032] Method 100 may also include determining, at 112, a vector control between portion 500 of hillshade 400 and portion 550 of image data 250 based at least in part on the comparison. The vector control may represent a distance and / or direction (i.e., bearing) between a first coordinate and a second coordinate in each coordinate pair. For example, a first vector control may represent a distance and direction from a coordinate in portion 500 of hillshade 400 to a corresponding coordinate in portion 550 of image data 250. The vector control may be created using GeoGPM.

[0033] Table 1 below shows Figure 5A and Figure 5BThe coordinates in the coordinates used to determine some coordinates of vector control. In Table 1:

[0034] Starting image X and Y: refers to the starting image (e.g. Figure 5A ). For example, a starting image X, Y of 200, 200 refers to a point in the starting image that is 200 pixels / column from the left and 200 pixels / row from the top.

[0035] Target image X and Y: refers to the target image (e.g. Figure 5B ) in the target image. For example, the point at 200, 200 in the starting image is at 213, 198 in the target image. In other words, if Figure 5A and Figure 5B Overlap, then Figure 5A The points mentioned in Figure 5B The corresponding points in the starting image are not aligned. Therefore, the vector control of these coordinates can indicate that the point in the starting image should be moved 13 columns / pixels to the right and 2 rows / pixels down to become aligned with the point in the target image. The starting image X and Y and the target image X and Y are in image space.

[0036] Starting Geo X and Y: These are the same as Starting Image X and Y, but in geographic space, as opposed to image space.

[0037] Target Geo X and Y refer to the same things as target Image X and Y, but in geographic space, as opposed to image space.

[0038]

[0039] The method 100 may further include the step of applying vector control to the image data 250 at 114 to generate updated image data. Figure 7 The terrain data 200 is illustrated alongside (ie, compared to) updated image data 700 according to an implementation. Figure 8 A hillshade 400 is illustrated alongside (i.e., compared to) updated image data 700 according to one implementation. The updated image data 700 can be used to generate a map. Instead of, or in addition to, applying vector control to the image data 250, vector control can also be applied to the terrain data 200 to generate updated terrain data (or updated hillshade 400).

[0040] Applying vector control can co-register (and / or geo-register) the terrain data 200 and / or the image data 250, which can reduce or eliminate the need for Figure 3 and Figure 6 300 as seen in the figure. As used herein, "geo-referencing" refers to aligning or matching pixels in a control image (e.g., terrain data or image data) to the correct place / location on the surface of the earth. In other words, applying vector controls can transform different data sets (e.g., terrain data 200 and image data 250) into a single common coordinate system in which pixels are aligned to relevant locations in image space and / or geographic space. For example, vector controls can be used to adjust image data 250 to terrain data 200 (or hillshade 400), or vector controls can be used to adjust terrain data 200 (or hillshade 400) to image data 250. Vector controls can be applied using a transformation to align pixels. The transformation can be or include a polynomial transformation. The transformation can be a first-order transformation, a second-order transformation, a third-order transformation, a fourth-order transformation, a fifth-order transformation, or a higher-order transformation. Instead of using a transformation, or in addition to using a transformation, vector controls can also be applied using a warp to align pixels.

[0041] The method 100 may further include the step of generating, at 116 , a plot including the vector control. Figure 9 A plot 900 including distance and bearing values for various vector controls is illustrated according to an implementation. For example, various points (e.g., point 910) in the plot 900 represent Figure 5A The coordinates in Figure 5B The distance and bearing of the corresponding coordinates in (or from Figure 5B The coordinates in Figure 5A The distance and bearing of the corresponding coordinates in ). As mentioned above, Figure 5A and Figure 5B The coordinates in the graph 900 may all correspond to the same point on the surface of the Earth (e.g., 2D or 3D coordinates). Thus, the points in the plot 900 may represent Figure 5A and Figure 5B These range and bearing values can be plotted to represent statistics such as the 90% circular error probability (CEP) (referred to as CEP90), the mean bearing, and the variance.

[0042] When the right Figure 5A The coordinates in Figure 5B When compared to the coordinates in , multiple points in plot 900 (e.g., including point 910) provide the following exemplary statistics:

[0043] CE90: 259.8 meters

[0044] Average bearing: 26.8 degrees

[0045] Azimuth range: 19.1 degrees to 32.7 degrees

[0046] Distance range: 89.1m to 317.1m

[0047] Azimuth difference: 13.6 degrees

[0048] Distance difference: 228.0 meters

[0049] Although plot 900 illustrates the CEP of portion 550 of image data 250 compared to portion 500 of hillshade 400, in other implementations, plot 900 may illustrate image-to-image CEP, image-to-terrain CEP, terrain-to-image CEP, and / or terrain-to-terrain CEP. Additionally, plot 900 may illustrate data that enables a user to quickly visualize the type of shift between the sources (e.g., terrain data 200 and image data 250). For example, if the points are clustered in a single quadrant, this may indicate a shift (e.g., misalignment) between the sources. However, if the points are distributed across multiple quadrants of plot 900, this may indicate a scaling and / or rotation issue.

[0050] At least a portion of method 100 is not a mental process and cannot be performed in the human brain. For example, in one implementation, one or more of the steps (e.g., all of the steps) are performed by a computing system such as the computing system described below.

[0051] Figure 10A schematic diagram of a computing system 1000 for performing at least a portion of method 100 according to one implementation is illustrated. Computing system 1000 may include a computer or computer system 1001A, which may be a standalone computer system 1001A or an arrangement of distributed computer systems. Computer system 1001A includes one or more analysis modules 1002 configured to perform various tasks according to some implementations, such as one or more methods disclosed herein. To perform these various tasks, analysis module 1002 performs the tasks independently or in cooperation with one or more processors 1004, which are connected to one or more storage media 1006. Processor 1004 is also connected to a network interface 1007 to enable computer system 1001A to communicate with one or more additional computer systems and / or computing systems (such as 1001B, 1001C and / or 1001D) via a data network 1009 (it should be noted that computer systems 1001B, 1001C and / or 1001D may or may not share the same architecture as computer system 1001A and may be located in different physical locations, for example, computer systems 1001A and 1001B may be located in such a processing facility that simultaneously communicates with one or more computer systems (such as 1001C and / or 1001D) located in one or more data centers and / or in different countries on different continents).

[0052] A processor may include a microprocessor, a microcontroller, a processor module or subsystem, a programmable integrated circuit, a programmable gate array, or another control or computing device.

[0053] The storage medium 1006 may be implemented as one or more computer-readable or machine-readable storage media. Figure 10 In the example implementation of , the storage medium 1006 is depicted as being within the computer system 1001A, but in some implementations, the storage medium 1006 can be distributed within and / or across multiple internal and / or external enclosures of the computing system 1001A and / or additional computing systems. The storage medium 1006 can include one or more different forms of memory, including: semiconductor memory devices such as dynamic or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; magnetic disks such as fixed disks, floppy disks, and removable disks; other magnetic media including magnetic tape; magnetic media such as compact disks (CDs) or digital video disks (DVDs), or other types of storage devices. It should be noted that the instructions discussed above may be provided via one computer-readable or machine-readable storage medium, or may be provided via multiple computer-readable or machine-readable storage media distributed across a large system, possibly with multiple nodes. Such computer-readable or machine-readable storage media are considered part of an article (or article of manufacture). An article or article of manufacture may refer to any manufactured single component or multiple components. The storage medium may be located in a machine that runs the machine-readable instructions, or may be located at a remote site from which the machine-readable instructions may be downloaded over a network for execution.

[0054] In some implementations, the computing system 1000 includes one or more registration modules 1008 that can perform at least a portion of the above-described method 100. It should be understood that the computing system 1000 is only one example of a computing system, and the computing system 1000 can have more or fewer components than shown, can be combined with other components, and can be used to perform the above-described method 100. Figure 10 Additional components not depicted in the example implementation of , and / or computing system 1000 may have Figure 10 Different configurations or arrangements of the components depicted in the FIG. can be implemented using hardware, software, or a combination of hardware and software Figure 10 The various components shown may be implemented using one or more signal processing and / or application specific integrated circuits. Figure 10 Various components shown.

[0055] In addition, the steps of the processing method described herein can be implemented by running one or more functional modules in an information processing device such as a general-purpose processor or a dedicated chip (such as an ASIC, FPGA, PLD or other suitable device). These modules, combinations of these modules and / or combinations of these modules with general-purpose hardware are all included in the scope of protection of the present invention.

[0056] Furthermore, the present disclosure includes implementations according to the following clauses:

[0057] 1. A method for aligning terrain data and image data, the method comprising the following steps: receiving terrain data and image data; determining a position of a light source based on the image data; creating a hillshade representation of the terrain data based on the terrain data and the position of the light source; identifying a portion of the hillshade representation and a portion of the image data that correspond to each other; comparing the portion of the hillshade representation with the portion of the image data; determining a vector control between the portion of the hillshade representation and the portion of the image data based on the comparison; and applying the vector control to the image data to generate updated image data.

[0058] 2. The method of clause 1, wherein the light source is the sun, the sun casting a shadow in the image data.

[0059] 3. The method according to clause 2, wherein the position of the light source comprises: solar azimuth, solar altitude, solar zenith, or a combination thereof.

[0060] 4. The method of clause 1, wherein comparing the portion of the hillshade representation to the portion of the image data is performed using an image matching technique that outputs coordinate pairs.

[0061] 5. The method of clause 4, wherein the coordinate pair comprises: a first pixel in the hillshade representation and a second pixel in the image data, wherein the first pixel and the second pixel correspond to the same point on the surface of the Earth.

[0062] 6. The method of clause 5, wherein the coordinate pairs are in image space, and the method further comprises the step of converting the coordinate pairs into geographic space.

[0063] 7. The method of clause 5, wherein the first pixel and the second pixel are misaligned.

[0064] 8. The method of clause 7, wherein the vector control comprises: a distance between the first pixel and the second pixel; and a bearing from the first pixel to the second pixel.

[0065] 9. The method of clause 8, wherein applying the vector control comprises moving the first pixel along the direction by the distance to become aligned with the second pixel.

[0066] 10. The method of clause 9, further comprising the step of generating a plot showing the distance and the bearing.

[0067] 11. A method for registering terrain data and image data, the method comprising the following steps: receiving terrain data; receiving image data, wherein the image data is captured by a camera on an aircraft or a satellite; determining the position of the sun when the image data was captured based on shadows in the image data, wherein the position of the sun includes the azimuth and altitude of the sun; creating a hillshade representation of the terrain data based on the terrain data and the position of the sun; identifying a portion of the hillshade representation and a portion of the image data that correspond to each other; and using an image matching technique, a pattern matching technique, or an object matching technique that outputs a plurality of coordinate pairs to match the hillshade representation to the image data. The portion of the hillshade representation is compared to the portion of the image data, wherein each coordinate pair includes: a first pixel in the image data and a second pixel in the hillshade representation, wherein the first pixel and the second pixel both correspond to the same point on the surface of the Earth, and wherein the first pixel and the second pixel are not aligned; determining a vector control for each coordinate pair, wherein the vector control includes a distance and an orientation from the first pixel to the second pixel; and applying the vector control to the image data to move the first pixel by the distance along the orientation to become aligned with the second pixel, thereby generating updated image data.

[0068] 12. The method of clause 11, further comprising the step of generating a plot showing the range and the orientation of each vector control.

[0069] 13. The method of clause 12, determining a circular error probability for the vector control based at least in part on the plot.

[0070] 14. The method of clause 12, wherein the terrain data is captured by a source, and further comprising the step of determining that the source and camera are misaligned based at least in part on the vector controls being in the same quadrant in the plot.

[0071] 15. A method according to clause 12, wherein the terrain data is captured by a source and the method further comprises the following steps: at least in part based on the vector control being positioned in at least two different quadrants of the plot, determining that the zoom of the source is not equal to the zoom of the camera, the rotation angle of the source is not equal to the rotation angle of the camera, or both.

[0072] 16. A computing system, the computing system comprising: one or more processors; and a memory system, the memory system comprising one or more non-transitory computer-readable media storing instructions, which, when executed by at least one of the one or more processors, cause the computing system to perform operations, the operations comprising: receiving terrain data; receiving image data, wherein the image data is captured by a camera on an aircraft or a satellite; determining a position of the sun at the time the image data was captured based on shadows in the image data, wherein the position of the sun includes an azimuth and an altitude of the sun; creating a hillshade representation of the terrain data based on the terrain data and the position of the sun; and performing a multi-processing operation on a portion of the hillshade representation and the image data that corresponds to each other. The method comprises the steps of identifying a portion of the hillshade representation and comparing the portion of the image data using an image matching technique, a pattern matching technique, or an object matching technique that outputs a plurality of coordinate pairs, wherein each coordinate pair comprises: a first pixel in the image data and a second pixel in the hillshade representation, wherein the first pixel and the second pixel both correspond to the same point on the surface of the earth, and wherein the first pixel and the second pixel are not aligned; determining a vector control for each coordinate pair, wherein the vector control comprises a distance and an orientation from the first pixel to the second pixel; and applying the vector control to the image data to move the first pixel along the orientation by the distance to become aligned with the second pixel, thereby generating updated image data.

[0073] 17. The computing system of clause 16, wherein the operations further comprise generating a plot showing the range and the orientation of each vector control.

[0074] 18. The computing system of clause 17, wherein the operations further comprise determining a circular error probability for the vector control based at least in part on the plot.

[0075] 19. The computing system of clause 17, wherein the terrain data is captured by a source, and the method further comprises the step of determining that the source and camera are misaligned based at least in part on the vector controls being positioned in the same quadrant in the plot.

[0076] 20. A computing system according to clause 17, wherein the terrain data is captured by a source and the method further comprises the following steps: at least in part based on the vector control being positioned in at least two different quadrants of the plot, determining that the zoom of the source is not equal to the zoom of the camera, the rotation angle of the source is not equal to the rotation angle of the camera, or both.

[0077] Although the numerical ranges and parameters setting forth the broad scope of the present disclosure are approximations, the numerical values set forth in the specific examples are reported as precisely as possible. However, any numerical value inherently contains certain errors necessarily resulting from the standard deviation found in their respective testing measurements. Furthermore, it is to be understood that all ranges disclosed herein encompass any and all subranges subsumed therein.

[0078] Although this teaching has been illustrated with reference to one or more implementations, changes and / or modifications may be made to the illustrated examples without departing from the spirit and scope of the appended claims. In addition, although the specific features of this teaching have been disclosed for only one implementation in several implementations, for any given or specific function, one or more other features that may be desired and advantageous may be combined with this feature and other implementations. As used herein, the terms "one (a and an)" and "the" may refer to one or more elements or one or more parts of an element. As used herein, the terms "first" and "second" may refer to two different elements or two different parts of an element. As used herein, the term "at least one of A and B" about enumerating an item (for example, such as A and B) means independent A, independent B or A and B. Those skilled in the art will recognize that these and other variations are all possible. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or variations thereof are used in any detailed description and / or claims, such terms are intended to be inclusive in a manner similar to the term "comprising." Furthermore, in the discussion and claims herein, the term "about" indicates that the listed values may vary slightly as long as such variations do not cause the process or structure to be unsuitable for its intended purpose as described herein. Finally, "exemplary" indicates that the description is used as an example and does not imply that the description is ideal.

[0079] It should be clear that variations of the above-disclosed and other features and functions, or alternatives of these features and functions, can be combined into many other different systems or applications. Those skilled in the art may subsequently make various alternatives, modifications, changes or improvements that are not currently foreseen or anticipated herein, and these alternatives, modifications, changes or improvements will also be covered by the appended claims.

Claims

1. A method for registering terrain data and image data, the method comprising the following steps: receiving the terrain data and the image data; determining a position of a light source based on the image data; creating a hillshade representation of the terrain data based on the terrain data and the position of the light source; identifying a portion of the hillshade and a portion of the image data that correspond to each other; comparing the portion of the hillshade representation to the portion of the image data; determining a vector control between the portion of the hillshade and the portion of the image data based on the comparison; applying the vector control to the image data to generate updated image data; as well as Generate a plot showing the distance and bearing controlled by each vector, wherein comparing the portion of the hillshade representation with the portion of the image data is performed using at least one of the following of an output coordinate pair: an image matching technique, a pattern matching technique, and an object matching technique, wherein the coordinate pair comprises: a first pixel in the hillshade representation and a second pixel in the image data, wherein the first pixel and the second pixel correspond to a same point on the surface of the Earth, wherein the first pixel and the second pixel are not aligned, The vector control includes: a distance between the first pixel and the second pixel; and a direction from the first pixel to the second pixel, wherein applying the vector control to the image data comprises: moving the first pixel along the orientation by the distance to become aligned with the second pixel and generating updated image data, wherein the distance and the bearing are represented by points in the plot, Wherein, the terrain data is captured by a source, The method further includes determining, based at least in part on the vector control being positioned in at least two different quadrants of the plot, that the source zoom is not equal to the camera zoom, the source rotation angle is not equal to the camera rotation angle, or both.

2. The method according to claim 1, wherein The light source is the sun, which casts shadows in the image data.

3. The method according to claim 2, wherein: The position of the light source includes: solar azimuth, solar altitude, solar zenith, or a combination of these.

4. The method according to claim 1, wherein The coordinate pairs are in image space, and the method further comprises the step of converting the coordinate pairs into geographic space.

5. The method according to claim 1, wherein Based at least in part on the plot, a circular error probability for the vector control is determined.

6. The method according to claim 1, wherein The terrain data is captured by a source, and the method further comprises the following steps: The source and the camera are determined to be misaligned based at least in part on the vector controls being in the same quadrant in the plot.

7. The method according to claim 1, wherein The image data is captured by the camera on an aircraft or a satellite.

8. A computing system, comprising: one or more processors; as well as A memory system comprising one or more non-transitory computer-readable media storing instructions that, when executed by at least one of the one or more processors, cause the computing system to perform the method of any one of claims 1 to 7.

Images