High resolution scanned image stitching method, system and computer program product

Abstract

A computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection is disclosed. The method includes providing a plurality of partial digital images of the infrastructure surface. The method includes extracting global positioning system metadata from data corresponding to the partial digital images. The method includes determining feature descriptions of features in one or more of the partial digital images. The method includes performing a scheduled processing sequence of the partial digital images based on the extracted global positioning system metadata, including determining an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally locate each of the partial digital images such that a full view image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

Description

BACKGROUND

[0001] The present disclosure relates generally to a method for image stitching of a plurality of digital images, and more particularly, to a computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection. The present disclosure also relates to an image stitching system of image stitching of a plurality of digital images of an infrastructure surface for defect detection, and a computer program product.

[0002] Civil infrastructure around the world is becoming older and older, and its reliability can weaken over time. Continuous human-based inspection of civil infrastructure components such as buildings, bridges, roads, pipelines, power grid towers, wind turbines, solar cell systems, dams, etc. is expensive and often also dangerous. Therefore, in recent years, automated methods for monitoring the structural health of civil infrastructure have gained popularity. The availability of autonomous drones to easily access and scan the surface of a wall, bridge, or street with high-quality image material can facilitate the evaluation of the surface of a structure by locating, identifying, and assessing defects (e.g., cracks, rust, or algae). However, the interpretation of defects can be improved by relying at least on high-resolution close-ups of the defects, as well as on their location and size, to better assess the impact on the overall structure.

[0003] In order to seriously inspect large infrastructure components, one digital image of the surface is not enough. Indeed, multiple digital images of the larger surface area of interest can be required. In order to reconstruct an accurate full view image of the infrastructure component, this plurality of digital images needs to be mapped to each other. Some known methods that attempt to solve this problem are based on creating a panoramic scene from a single photograph. However, these methods are manual and time-consuming and often result in inconsistencies. Unfortunately, such problems also arise in known techniques currently available for seamlessly mapping multiple digital images. In addition, these known techniques can have shortcomings in the accuracy and speed of feature detection during the image stitching process, which can result in inaccuracies in potential detected defects.

[0004] Therefore, it is desirable to overcome the limitations of existing image stitching techniques and provide an image stitching technique for delivering accurate results for civil infrastructure health monitoring based on deterministic algorithm improvements. SUMMARY

[0005] According to one aspect of the present disclosure, a computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection can be provided. The method can include providing a plurality of partial digital images of the infrastructure surface. The method can include extracting global positioning system metadata from data corresponding to the partial digital images. The method can include determining feature descriptions of features in one or more of the partial digital images. The method can include performing a scheduled processing sequence for the partial digital images based on the extracted global positioning system metadata, including using the feature descriptions of adjacent partial digital images to determine an affinity matrix to incrementally locate each of the partial digital images such that a full view image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

[0006] According to another aspect of the present disclosure, an image stitching system of image stitching of a plurality of digital images of an infrastructure surface for defect detection can be provided. The system can include a memory for storing program code portions, the memory coupled to a processor, which when executing the program code portions can enable the processor to receive a plurality of partial digital images of an infrastructure surface. The processor can also be enabled to extract global positioning system metadata from data corresponding to the partial digital images. The processor can also be enabled to determine feature descriptions of features in one or more of the partial digital images when executing the program code portions. The processor can also be enabled to perform a scheduled processing sequence for the plurality of partial digital images based on the extracted global positioning system metadata, including using the feature descriptions of adjacent partial digital images to determine an affinity matrix to incrementally locate each of the partial digital images such that a full view image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

[0007] The methods and systems of image stitching of a plurality of digital images of an infrastructure surface for defect detection disclosed herein can provide a number of advantages, technical effects, contributions, and / or improvements.

[0008] For example, embodiments of the methods and systems disclosed herein can include using an innovative algorithm to estimate a robust and accurate transformation matrix of scans of a planar service, such as a civil infrastructure component.

[0009] Additionally, embodiments of the methods and systems disclosed herein can use global positioning system (GPS) data to initially place partial digital images to define an error- resistant processing order. For example, according to at least some embodiments of the disclosure, GPS data can be used to define a schedule for processing partial digital images. In such embodiments, the disclosed methods and systems facilitate computation speed, as they can scale linearly with the number of images. Thus, in such embodiments, results can be better than those obtained using traditional techniques, e.g., techniques that include little or no non-overlapping areas in the overall image. This can be based primarily on the use of GPS coordinates in combination with visual alignment, as pure visual methods are often difficult to align images with similar, repetitive, homogenous structures, such as dozens or hundreds of identical windows in a high-rise building.

[0010] Further, embodiments of the methods and systems disclosed herein can be resilient against the effects of outlier images that can exist in the captured plurality of partial digital images. Such outlier images can occur if a camera on a flying object, e.g., a drone, is used. For example, the camera can be out of focus or can not have a view normal to the surface. For example, the camera can not be able to orient itself with a view normal to the surface if there is an obstacle that visually blocks a portion of the surface. Additionally, reflections can occur on the captured digital surface, or a drop-off can occur caused by camera movement. Embodiments of the methods and systems disclosed herein can be resilient to all of these issues, and can self-correct for such errors.

[0011] Resilience against outlier images that can occur during the capture of partial digital images can be addressed primarily through feature detection, feature matching, and transformation matrix estimation aspects according to features of at least some embodiments of the methods and systems disclosed herein. Additionally, embodiments of the methods and systems disclosed herein can include verification, which can check properties of the estimated matrix in order to accept or discard it.

[0012] To determine a processing schedule for fast accumulation stitching, which can also be referred to as a stitching sequence, embodiments of the methods and systems disclosed herein can rely on the fact that images with a small Euclidean distance between GPS locations can have a high overlap. Further, in embodiments of the methods and systems disclosed herein, the beginning of the stitching process follows a heuristic approach, as it starts in the middle or center of the plurality of captured partial digital images for a particular scene, e.g., from a GPS-based context. This strategy can provide the most natural view, and can minimize potential error propagation paths.

[0013] Further embodiments of the disclosure applicable to the methods and systems disclosed herein can facilitate additional benefits.

[0014] According to an optional embodiment, the method can further comprise rendering the overview image and mapping the at least one identified defect, the at least one annotation or the at least one measurement onto the overview image. For example, the defects can be detected according to various methods by a neural network system such as a deep neural network. Based thereon, the related annotations or measurements determined according to the detected elements in the overview image can also be shown as overlays to the overview image. As an example, the size of the defects or the number of defects (e.g. per region) or the average distance between defects of a predetermined size can be determined and can also be displayed in the rendered overview image.

[0015] According to an optional embodiment, the method can further comprise performing at least one of scene registration, image warping (or unwarping) and pixel-use comparison for a time evolution assessment of multiple partial digital images of the infrastructure surface having different timestamps. This feature can facilitate determining the evolving changes of defects and / or cracks in civil infrastructure elements. Scene registration and additional activities in the present context can be more suitable than single image comparisons such as those performed by conventional techniques.

[0016] According to another optional embodiment, the method can further comprise generating the multiple partial digital images using a camera of an unmanned vehicle. Such embodiments facilitate capturing high building facades or bridge pillars, for example, on a river. The unmanned vehicle (e.g. UAV) can be an unmanned aerial vehicle, drone, helicopter, model airplane, airship, balloon, camera mounted on a vehicle sliding along the surface, such as a lift for building cleaning, etc. However, the UAV should be maneuverable by a remote control or an automatic control and positioning system.

[0017] According to another optional embodiment, determining the feature description can comprise using a SIFT (Scale-Invariant Feature Transform) method. This method can be used to determine local features in an image. Thus, if different captured images are generated at different viewing angles, the local features are still detectable and matchable if they are visible in different images, in particular in partially overlapping images.

[0018] According to another optional embodiment, determining the feature description can comprise detecting at least one point of interest in the captured partial images. In such embodiments, the SIFT method can be supported and the stitching process can be accelerated.

[0019] According to another optional embodiment, determining the affinity matrix can include using a RANSAC method (e.g., known as Random Sample Consensus) to incrementally locate each of the partial digital images. The RANSAC method is a robust estimation procedure that can use a minimal set of randomly sampled correspondences in the images to estimate the image transformation parameters and find the solution with the best consensus, regardless of whether the existing data (e.g., the captured partial images) matches.

[0020] According to another optional embodiment, incrementally locating each of the partial digital images can include determining a transformation matrix H i to stitch the current partial digital image I i to the already existing intermediate stitched image S i In such an embodiment, the already existing intermediate stitched image S i may be the result of a previous iteration or loop of the image stitching of the captured partial digital images. As such, the general perspective transformation matrix can be a 3x3 matrix, while an affine transformation is a special case of perspective, where the third row is [0, 0, 1]. An affine transformation matrix is a combination of rotation, shear, translation, and scaling (further details discussed below).

[0021] According to another optional embodiment, scheduling the processing sequence for the images can include ordering the partial digital images with increasing distance (such as Euclidean distance) from a center image of the infrastructure surface, and starting the incremental location of each of the partial digital images from the center image. This approach provides a more robust method for the solid stitching process, which is more accurate than, for example, detector-based algorithms, which are not resilient to increasing variables.

[0022] According to another optional embodiment, providing the partial digital images can include capturing the partial digital images at an angle that is parallel or nearly parallel to the surface of interest of the infrastructure. In other words, the camera-to-drone perspective can be perpendicular or nearly perpendicular to the surface of interest. This can reduce the need for significant transformation (e.g., de-skewing) of the partial digital images before they can be used in the stitching process.

[0023] According to another optional embodiment, providing the partial digital images can include capturing the partial digital images such that the image plane is not parallel to the captured surface, and digitally de-skewing the partial digital images in order to align the partial digital images parallel to the captured surface. In such an embodiment, digital images can also be used where the perspective of the camera is not perpendicular to the infrastructure surface, for example, in cases where the surface cannot be clearly seen due to visual obstructions.

[0024] According to another optional embodiment, providing the partial digital images can comprise capturing the partial digital images on a virtual regular grid of the infrastructure surface. In such embodiments, the required transformations before stitching the partial digital images together into the full view image can be reduced. Thus, optical inconsistencies in the final full view image can be significantly reduced or completely avoided.

[0025] Furthermore, embodiments of the present disclosure can take the form of an associated computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. BRIEF DESCRIPTION OF DRAWINGS

[0026] It should be noted that embodiments of the present disclosure are described with reference to different subject matters. In particular, some embodiments are described with reference to method type claims whereas other embodiments are described with reference to device type claims. However, a person skilled in the art will gather from the above and the following description that, unless otherwise indicated, any aspect or feature described as pertaining to a particular subject matter can be implemented with any or all of the features of another subject matter, and in any combination thereof. In particular, a person skilled in the art will gather from the above and the following description that, unless otherwise indicated, any combination of features described as pertaining to a method type subject matter can be implemented with a device type subject matter, and vice versa.

[0027] The above aspects and other aspects of the present disclosure are apparent from the examples of embodiments to be described hereinafter and the explanations thereof, which are not limited to the examples of embodiments but are illustrated by way of example in conjunction therewith. Preferred embodiments of the present disclosure will be described, by way of example only, with reference to the accompanying drawings.

[0028] Figure 1 A block diagram depicting an embodiment of a computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection according to embodiments of the present disclosure is depicted.

[0029] Figure 2 A schematic diagram depicting an embodiment of an implementation of the present disclosure according to embodiments of the present disclosure is depicted.

[0030] Figure 3 A schematic diagram depicting a full view of an implementation of the present disclosure according to embodiments of the present disclosure is depicted.

[0031] Figure 4 A schematic diagram depicting an illustrative embodiment of a movement of a camera for stitching and a sequence of images according to embodiments of the present disclosure is depicted.

[0032] Figure 5A An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0033] Figure 5B An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0034] Figure 5C An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0035] Figure 5D An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0036] Figure 5E An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0037] Figure 5F An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0038] Figure 5G An image is shown after execution of a portion of the process of stitching together captured partial digital images, according to embodiments of the disclosure.

[0039] Figure 6 An illustrative example of captured images that are time related and compared in single image mode, according to embodiments of the disclosure, is shown.

[0040] Figure 7 A block diagram of multiple captured images that depict the evolution of a defect using scene comparison to determine the defect, according to embodiments of the disclosure, is depicted.

[0041] Figure 8 A block diagram of an embodiment of an image stitching system that performs image stitching of multiple digital images of an infrastructure surface for defect detection, according to embodiments of the disclosure, is depicted.

[0042] Figure 9 A schematic diagram of an embodiment of a computing system that includes a system according to Figure 8 , according to embodiments of the disclosure, is depicted. DETAILED DESCRIPTION

[0043] In the context of the present specification, the following conventions, terms and / or expressions can be used:

[0044] The term "image stitching" can refer to the process of combining multiple taken digital images with possibly overlapping areas to provide a segmented panoramic or overview image in high resolution form.

[0045] The term "infrastructure surface" can refer to the surface of any man-made civil infrastructure, such as buildings, bridges, pipes, pillars of wind energy generators, pillars of electric energy transmission cables, roads and similar structures.

[0046] The term "partial digital image" can refer to a digital image, for example, captured by a digital camera of an unmanned flying object, which can be stitched together with other images to build an overview image having the same or similar resolution as the partial digital images.

[0047] The term "global positioning system" (GPS) can refer to a global satellite-based radio navigation system, which is based on the global navigation satellite system (GNSS) to provide GPS receivers with geolocation and time information. The GPS receiver can be part of the camera capturing the partial digital images, so that GPS data, for example, latitude data, longitude data and altitude data (also referred to as height data), can be integrated as metadata into the dataset of the captured digital images.

[0048] The term "processing sequence" can refer to the process according to which partial digital images are determined to be processed and stitched together. The stitching process can in particular start at a starting image, which can be located in the middle of the overview image that shall be produced. Other images to be stitched to the starting image follow a rule of increasing distance from the center image. After two or more images have been stitched together, they can be denoted as "intermediate images".

[0049] The term "feature description" can refer to a specific marker for identifying a detected feature in a digital image.

[0050] The term "affinity matrix" can refer to a mathematical matrix, in which related features in adjacent digital images can be related to each other, so that these related features are overlaid in a seamless manner during the stitching process.

[0051] The term "overview image" can refer to a plurality of captured partial digital images stitched together in a seamless manner according to the methods and systems disclosed herein. The overview image can thus have the same resolution (e.g., pixels per inch) as each of the partial digital images.

[0052] The term "identified defect" can refer to a mechanical or other inconsistency in a digitally captured image. Such a defect can involve a crack or other problematic area in a civil infrastructure component that can eventually be the cause of a functional defect in the relevant infrastructure component. To identify a defect, various trained neural network systems can use one or more captured partial digital images as input.

[0053] The term "scene registration" can refer to the registration of a full view image produced from a plurality of captured partial digital images that have been stitched together. The registered scene at a particular time (specifically, the time with timestamp X) can be used as a reference for comparison to another scene at a later time (specifically, the later time with timestamp Y).

[0054] The term "image warping" can generally refer to a process of digitally manipulating a digital image. Shapes and forms depicted in the image undergo one or more transformations and thus can be significantly distorted if compared to the captured image. The term "image unwarping" can be used herein to refer to a transformation of an image that can not have been captured under ideal conditions such that the perspective would be perpendicular to the captured surface.

[0055] The term "pixel-use comparison" can refer to a comparison process of digital images using techniques based on comparison of corresponding pixels in the digital images.

[0056] The term "SIFT" can refer to a known feature detection algorithm for detecting and describing local features and images. The acronym SIFT stands for Scale-Invariant Feature Transform. This technique is useful for object recognition, especially for image stitching. SIFT keypoints of an object can first be extracted from a set of reference images and stored in a database. Subsequently, the object can be identified in a new image by individually comparing each feature from the new image to the images in the database and finding candidate matching features based on the Euclidean distance of their feature vectors. As an example, 128 floating point value vectors can be used here.

[0057] The term "transformation matrix" can refer to a mathematical matrix used to transform another matrix into another representation. For example, a transformation matrix can be used to transform a matrix describing a digital image or portion thereof into another representation.

[0058] Detailed descriptions of the drawings are provided herein. All illustrations are schematic. First, a block diagram of an embodiment of a computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection is presented. Further, additional embodiments are described as well as embodiments of an image stitching system of image stitching of a plurality of digital images of an infrastructure surface for defect detection.

[0059] Before undertaking a detailed description of the present disclosure, a general description of the general concepts of the present disclosure is presented below.

[0060] First, a substantially flat and stationary surface of the surface of interest is assumed to be captured as a plurality of digital images. The overall complex structure is decomposed into a plurality of simple scenes. For example, each face of a rectangular bridge column or bridge tower is treated individually. Basic elements such as a facade of a building, a hospital room, and a street surface are simple examples of a face. Each face typically defines a plane of interest, which the face is assumed to lie in. During the scanning process, a flying object, such as a drone, moves along parallel planes in order to keep the heading of the camera facing the plane of interest in a perpendicular manner.

[0061] A mission plan is executed such that the path of the drone is controlled along a regular grid pattern to scan the surface of interest. Images are captured at regular intervals and are automatically related to GPS metadata, which is enhanced by real-time kinematic techniques (ITK) that rely on reference base station signals. The raw images with GPS annotations are the only input data considered for subsequent processing.

[0062] Image sensor resolution and the distance of the drone from the surface of interest determine the ground sampling distance. The mission plan ensures a minimum target ground sampling distance. Since the drone moves during image capture, the flight speed can be limited to limit the maximum image sensor exposure to minimize motion blur visible in the collected data. For example, the flight speed can be limited to 0.4 meters per second or similar.

[0063] For feature matching, at iteration i, feature matching is performed between the intermediate stitched image S i and the corresponding current image I i The features of the considered image pair can be matched by, for example, the following procedure. According to at least some embodiments of the present disclosure, a k-NN matching algorithm based on the Euclidean distance between features, with k = 2 corresponding to each of the first two features from S i may be used. Additionally, Lowe's ratio test can be used to prune false positive matches.

[0064] In a further operation, using the matches of the previous operation, an estimation of the affine transformation matrix is performed in order to transform I i into S iSystem's accordion. According to at least one embodiment of the present invention, the RANSAC (Random Sample Consensus) algorithm can be used to perform this operation. RANSAC is a robust estimation procedure that uses a minimal random sample of corresponding pairs to estimate image transformation parameters and find the solution with the best consensus with the data. It removes outliers and returns the best transformation matrix. At least three non-collinear points can need to be matched to be able to estimate a fine transformation matrix.

[0065] A general perspective transformation matrix is a 3x3 matrix, while an affine transformation matrix is a special case of perspective, where the third row is [0, 0, 1]. An affine transformation matrix is a combination of rotation, shear, translation, and scaling, which can be represented as:

[0066]

[0067] where θ is the rotation angle of the image, t x and t y are translation factors, s x and s y are scaling factors, and m x and m y are shear factors in x and y directions, respectively.

[0068] Assuming ideal setup (as described above) only transformations between images can be expected. Thereafter, the ideal transformation matrix is constructed as follows:

[0069]

[0070] However, to address drone and camera positioning distortions, rotation, shear, and scaling to a certain threshold are also allowed. Transformations that exceed the threshold are marked as outliers and are excluded from further processing to avoid error propagation.

[0071] This technique is effective for eliminating some typical errors in a single image, such as out-of-focus, having motion blur, or key points that cause erroneous matching.

[0072] The figures described in the context of the above discussion are described below.

[0073] Figure 1A block diagram depicting a preferred embodiment of a computer-implemented method 100 of image stitching of a plurality of digital images of a surface of infrastructure for defect detection is depicted. The plurality of digital images can result from a high resolution regular scan of a surface of infrastructure for defect detection. The infrastructure can be, for example, a building, a bridge or another civil infrastructure. The plurality of digital images can also result from a scan for positioning and evolution-based severity assessment. At operation 102 of the method 100, a plurality of partial digital images of the surface of the infrastructure are provided. According to at least some embodiments of the present disclosure, the partial digital images also include GPS metadata.

[0074] At operation 104, the method 100 also includes extracting global positioning system metadata from the data of the partial digital images. According to at least some embodiments of the present disclosure, the metadata is extracted, for example, from a GPS or GNSS system, for example, Galileo, Navstar, GLONASS or Beidou.

[0075] At operation 106, the method 100 also includes scheduling a processing sequence for the images based on the extracted global positioning system metadata.

[0076] At operation 108, the method 100 also includes determining a feature description in one or more of the partial digital images. According to at least some embodiments of the present disclosure, the feature description can be determined by using, for example, 128-dimensional floating vectors from a SIFT method or other method.

[0077] At operation 110, the method 100 also includes executing the scheduled processing sequence. By executing the scheduled processing sequence, an affinity matrix of the feature descriptions of adjacent partial digital images can be determined to incrementally locate each partial digital image. In this way, by digitally stitching together the plurality of partial digital images, a full view image of the surface of the infrastructure can be produced step by step.

[0078] Figure 2 A schematic diagram illustrating an illustrative embodiment 200 of the present disclosure is shown. In the illustrative embodiment 200, the infrastructure object of interest can be a bridge 202, a building 204, a pole for a land cable for electricity, a pole for a wind energy generator or another similar structure. According to the illustrative embodiment 200, a camera mounted on an aerial vehicle can capture an image 206 having a given resolution. In the captured image 206, certain areas or sub-images 208 can be of more interest due to potential presence of damage, such as small cracks, surface scratches or similar defects. However, the resolution of the captured image can not be high enough to enable identification of the areas of interest. Accordingly, the area of the image 206 is scanned to produce a plurality of partial digital images (not shown here).

[0079] In some embodiments of the present disclosure, such as in the case in this illustrative example, it is necessary to span up to four orders of magnitude in resolution to accurately detect and locate defects. For example, a region of interest of the bridge 202 can have a width of approximately 10 meters. This region can be represented by a full view image 206 taken by a camera. However, a defect crack in this region of interest can have a width of less than about 1 millimeter. Thus, the full view image 206 typically has too low of a resolution to accommodate both the width of the region of interest (about 10 meters) and the width of the crack (less than about 1 millimeter).

[0080] Figure 3 An illustrative embodiment of a schematic diagram showing a full view 300 of an embodiment of the present disclosure is shown. A camera 302 is mounted on a flying object 304, and the camera 302 can capture a plurality of partial digital images 306. According to at least one embodiment of the present disclosure, the flying object 304 is a drone. Each digital image of the plurality of partial digital images 306 includes metadata, such as GPS data. The GPS data includes coordinates, such as latitude, longitude, and altitude (which can also be referred to as elevation). According to at least some embodiments of the present disclosure, the metadata of each digital image of the plurality of digital images 306 can also include a relevant time stamp. Using current technology, the GPS data can be accurate to ±5 centimeters. However, this is not accurate enough for high quality stitching of the partial digital images 306 to enable detection of infrastructure defects.

[0081] Based on the captured images 306 and the GPS data, a stitching order 308 is determined. Subsequently, the captured images 306, along with the stitching order 308, are used as inputs to a system 310 (implementing one or more algorithms) that includes SIFT interest point detection and description 312, interest point matching 314, transformation matrix estimation 316, and defect localization and / or measurement 318. The system 310 outputs a rendered fully stitched image 320, which includes defect localization and / or measurement data. The fully stitched image 320 can also be referred to as a resulting full view image.

[0082] Figure 4 An illustrative embodiment of a schematic diagram 400 showing movement of a camera for stitching and a sequence of images is shown. In the diagram 400, a cube 402 can represent an illustrative object of interest, such as a tower of a bridge. The camera 302 of the flying object 304 can fly along a path 404 in front of a surface of interest 406 of the tower. Assuming that the flying object 304 and the camera 302 operate in a typical manner, the camera 302 captures images from a relatively short distance away from the tower. In this illustrative embodiment, by capturing these images, the camera 302 provides a plurality of partial digital images, and the plurality of partial digital images are stored in a memory system 408.

[0083] Based on the GPS data associated with each of the captured images, a graph 412 can be formed in which each image is represented by a point. According to at least some embodiments of the present disclosure, the graph 412 is oriented such that the position in the relative longitude direction is on the x-axis and the position in the relative latitude direction is on the y-axis, and the points are organized on the graph 412 in a manner that indicates their position relative to one another. In the context of the graph 412, the partial digital images are ordered according to increasing geometric distance from the central image 410, which is determined to be at or near the middle or center of the overview image. The geometric distance can also be referred to as the physical distance or the geographic distance.

[0084] According to at least some embodiments of the present disclosure, the points representing the images can be connected by lines, which can indicate the order in which the images were captured. According to at least some embodiments of the present disclosure, the points representing the images can be labeled with a number adjacent to the point. In such embodiments, the number can represent the determined or scheduled order of the stitching process. As Figure 4 shown, according to at least some embodiments of the present disclosure, the order in which the images were captured is generally not the same as the order in which the images are stitched together to produce the overview image.

[0085] Figures 5A-5G A series of results are shown that demonstrate the performance of various operations of the processes described herein for stitching together captured partial digital images. In Figure 5A , each of the points 502 represents a reference point (e.g., center, upper left corner, middle of upper edge, etc.) of a corresponding one of the captured partial digital images 504 based on GPS metadata. However, in Figure 5A , the images 504 are only loosely positioned because the GPS metadata is not precise enough for proper matching and stitching.

[0086] As already mentioned in the context of Figure 4 , the stitching process starts in the middle. This is shown in Figure 5B , in which the central image 504a is highlighted. As Figure 5C shown, a second partial image 504b is stitched to the left of the central image 504a, which is the first image addressed in Figure 5B . As Figure 5D and Figure 5E shown, additional images 504c and 504d are added. In each iteration step, until all available partial digital images have been stitched together, the number of partial digital images that have been stitched together refers to the “so far” or “in progress” intermediate stitched image. These intermediate stitched images are referred to herein with the indicator “S i ”.

[0087] Figure 5FThe result of the progression of the iteration of the stitching process after more partial digital images have been stitched to the intermediate stitched image 506 is illustrated. Figure 5G The result of all available partial digital images having been stitched together to produce a full view image 508 as a result of the process described herein is illustrated.

[0088] Figure 6 A flow 600 of captured images related to data representative of the time at which the captured images were captured and the three-dimensional spatial conditions at which the captured images were captured is illustrated. In this illustrative example, an image 602 is captured with a timestamp X and illustrates a defect 604 in the surface and another artifact 606 in the surface. An image 608 is captured with a timestamp Y, which is later than timestamp X. The identified defect 604 has changed in image 608 relative to image 602. The perspective of the images is also different. More specifically, the perspective of image 608 is rotated clockwise relative to the perspective of image 602. Additionally, the right tail of defect 604 is larger in image 608 than in image 602. Conversely, artifact 606 appears to have not changed in image 608 relative to image 602.

[0089] For the purposes of this illustrative example, the time elapsed between timestamp X and timestamp Y is relatively large. In order to provide a trustworthy assessment of the infrastructure components, it is important to be able to observe defects or potential defects over a long period of time. An assessment of the evolution of a defect allows for a better assessment of its severity. However, comparing cracks over a long period of time is difficult due to the need for accurate positioning, the need to compare the same defect over multiple points in time, and the need for precise measurement of crack changes. A simple solution for accurate observation using a fixed camera is often not feasible because the scanning area is too large or the image material originates from a drone that can be difficult to assess.

[0090] Image 610 illustrates the result of images 602 and 608 overlaid on each other. Image 610 thus enables a direct comparison, as well as image registration, warping (or de-warping), and finally a direct comparison. However, image registration can require highly visible reference points that can be difficult to access without any markers. For example, concrete is a self-similar material, which makes it difficult to locate reliable matching points. Thus, it can be difficult to achieve the desired result illustrated in image 612, which depicts a correctly matched image.

[0091] Thus, for the methods disclosed herein, the scene comparison uses the image stitching process disclosed herein, rather than using a single image comparison. To this end, Figure 7 A related illustrative diagram 700 is illustrated.

[0092] In FIG. 700, a plurality of images 702 of a scene are captured at a first time with a timestamp X, and a plurality of images 704 of the same scene are captured at a second time with a timestamp Y. As described above, the timestamp Y occurs later than the timestamp X. At operation 706, the respective sets of images of the same scene are stitched together using the methods described above, resulting in respective generated overview images 708 and 710.

[0093] FIG. 700 also includes a partial flowchart 716 illustrating two operations performed using the overview images 708 and 710. First, at operation 712, scene registration is performed. Then, at operation 714, image registration, refinement, and comparison are performed. Figure 7 The techniques illustrated in FIG. 700 enable high-quality assessment of crack evolution because the scene likely includes highly visible reference points (such as boundaries or lines), and the accuracy of the registration depends on the image stitching accuracy of each scene's images. Thus, automated detection of unique growth defects in contrast to new defects is facilitated.

[0094] Figure 8 An illustrative embodiment of an image stitching system 800 that stitches a plurality of digital images of an infrastructure surface for defect detection is shown. The system 800 includes a memory 802 for storing portions of program code. The memory 802 is coupled to a processor 804 that, when executing the portions of program code, is capable of receiving a plurality of partial digital images of an infrastructure surface, extracting global positioning system metadata from data of the partial digital images, and scheduling a processing sequence for the images based on the extracted global positioning system metadata. According to at least some embodiments of the present disclosure, the processor 804 can use a receiver 806 to receive the plurality of partial digital images of the infrastructure surface. According to at least some embodiments of the present disclosure, the processor 804 can use a GPS extraction unit 808 to extract the global positioning system metadata from the partial digital images. According to at least some embodiments of the present disclosure, the processor 804 can use a scheduler 810 to schedule the processing sequence for the images.

[0095] The processor 804 is also enabled, when executing the portions of program code, to determine a feature description in one or more of the partial digital images and to execute the scheduled processing sequence to determine an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally locate each partial digital image. In this way, a stepwise overview image of the infrastructure surface is produced by digitally stitching together the plurality of partial digital images. According to at least some embodiments of the present disclosure, the processor 804 can use a determination module 812 to determine the feature description in one or more of the partial digital images. According to at least some embodiments of the present disclosure, the processor 804 can use an extraction module 814 to execute the scheduled processing sequence.

[0096] It is to be understood that all functional units, modules, and functional blocks, such as the receiver 806, the GPS extraction unit 808, the scheduler 810, the determination module 812, and the extraction module 814, can be implemented as hardware units, software modules, or a combination thereof, and they can be communicatively coupled to each other in a selected 1 : 1 manner for signal or message exchange. Alternatively, the functional units, modules, and functional blocks can be linked to the system internal bus system 816 for selective signal or message exchange.

[0097] Embodiments of the disclosure can be implemented with almost any type of computer suitable for storing and / or executing program code, regardless of the platform. For example, Figure 9 An illustrative embodiment of a computing system 900 suitable for executing program code related to the methods disclosed herein is depicted.

[0098] The computing system 900 is only one example of a suitable computer system and is not intended to limit the scope of use or functionality of embodiments described herein. The illustrative computing system 900 is capable of implementing and / or executing any of the functionality set forth above. According to at least some embodiments of the disclosure, the computing system 900 can be a server. Within the computing system 900, there are components, which are capable of operating with numerous other general purpose or special purpose computing system environments or configurations. Examples of well- known computing systems, environments, and / or configurations that can be suitable for use with the computing system 900 include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

[0099] The computer system / server 900 can be described in the general context of computer system executable instructions, such as program modules being executed by the computer system 900. Generally, program modules can include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system / server 900 can be practiced in distributed cloud computing environments with remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in both local and remote computer system storage media including memory storage devices.

[0100] In Figure 9In the illustrated implementation, the computer system / server 900 is a general-purpose computing device. The components of computer system / server 900 can include, but are not limited to, one or more processors or processing units 902, a system memory 904, and a bus 906 that couples various system components including system memory 904 to processor 902. Bus 906 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, and not limitation, such architectures include Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0101] Computer system / server 900 typically includes a variety of computer system readable media. Such media can be any available media that is accessible by computer system / server 900, and it includes both volatile and non-volatile media, removable and non-removable media.

[0102] The system memory 904 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 908 and / or cache memory 910. Computer system / server 900 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 912 can be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown and typically called a "hard disk drive"). Although not shown, a magnetic disk drive can also be provided for reading from and writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive can be provided for reading from or writing to a removable, non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical media). Each can be connected to bus 906 by one or more data media interfaces. As will be further depicted and described below, system memory 904 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0103] By way of example, and not limitation, aspects of the present disclosure can be implemented includes an operating system, one or more applications, other program modules, and program data. Each of the operating system, one or more applications, other program modules, and program data or some combination thereof, can include an implementation of a networking environment. Program modules 916 generally carry out the functions and / or methodologies of embodiments of the present disclosure as described herein.

[0104] Computer system / server 900 can also communicate with one or more external devices 918 such as a keyboard or a pointing device, displays 920, etc. through I / O interface 914. Computer system / server 900 can communicate with one or more networks such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) through network adapter 922. As depicted, network adapter 922 can communicate with the other components of computer system / server 900 through bus 906. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with computer system / server 900. Examples include, but are not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0105] Additionally, the image stitching system 800 for multiple digital images of a surface of an infrastructure for defect detection can be attached to the bus system 906.

[0106] The description of various embodiments of the disclosure have been presented for purposes of illustration but is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

[0107] The present application can be a system, a method, and / or a computer program product at any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0108] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted via a wire cable.

[0109] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions into the computing / processing device for storage in a computer readable storage medium within the respective computing / processing device.

[0110] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction-set-architecture (ISA) instructions, machine instructions, microcode, firmware instructions, state-setting data, configuration data for an integrated circuit, or source code or object code written in any combination of one or more programming languages, including an object-oriented programming language such as Smalltalk, C++, or the like, and a procedural programming language such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0111] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0112] These computer readable program instructions can be provided to a processor of a computer, or other programmable data processing apparatus, to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including

[0113] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0114] The computer program product of the present application can be implemented on one computer or on a plurality of computers in or coupled to the same distributed computing environment. The computer program code can be written in any combination of one or more programming languages, including an object oriented programming language and / or procedural programming language. The computer program code can execute entirely on a user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a computer system, can be configured so that it can provide a portion of or all of the functionality described herein.

[0115] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0116] All means or steps plus function elements in the following claims are intended to cover any structure, material, or acts for performing the function of the means or steps in combination with other claimed The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations will be apparent to those skilled in the art. Embodiments were chosen and described in order to best explain the principles of the disclosure and its practical application, and to thereby enable others skilled in the art to best utilize the disclosure with various modifications as are suited to the particular use contemplated.

[0117] In summary, some aspects of the present disclosure can be illustrated using the following non-limiting list of example embodiments.

[0118] Example embodiment 1 is a computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection. The method includes providing a plurality of partial digital images of the infrastructure surface. The method also includes extracting global positioning system metadata from data of the partial digital images. The method also includes scheduling a processing sequence for the partial digital images based on the extracted global positioning system metadata. The method also includes determining feature descriptions in one or more of the partial digital images. The method also includes performing the scheduled processing sequence to determine an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally locate each of the partial digital images to progressively produce a full view image of the infrastructure surface by digitally stitching together the plurality of partial digital images.

[0119] Example embodiment 2 includes the method of embodiment 1, with or without optional features. In this example embodiment, the method also includes rendering the full view image and mapping the identified defects, annotations, or measurements into the full view image.

[0120] Example embodiment 3 includes the method of either of example embodiments 1 or 2, with or without optional features. In this example embodiment, the method also includes performing at least one activity selected from the group consisting of scene registration, image warping, and pixel-usage comparison for time evolution assessment of the plurality of partial digital images of the infrastructure surface having different time stamps.

[0121] Example embodiment 4 includes the method of any of example embodiments 1 to 3, with or without optional features. In this example embodiment, the method also includes generating the plurality of partial digital images using a camera of an unmanned vehicle.

[0122] Example embodiment 5 includes the method of any of example embodiments 1 to 4, with or without optional features. In this example embodiment, determining the feature descriptions uses a SIFT method.

[0123] Example embodiment 6 includes the method of any of example embodiments 1 to 5, with or without optional features. In this example embodiment, determining the feature descriptions includes detecting interest points.

[0124] Example embodiment 7 includes the method of any of example embodiments 1 to 6, with or without optional features. In this example embodiment, determining the affinity matrix includes using a RANSAC method to incrementally locate each of the partial digital images.

[0125] Example embodiment 8 includes the method of example embodiment 7, with or without optional features. In this example embodiment, incrementally locating each of the partial digital images includes determining a transformation matrix H i to locate the current partial digital image Ii stitching to an already existing intermediate stitched image S i .

[0126] Example embodiment 9 includes the method of any one of example embodiments 1 to 8, with or without optional features. In this example embodiment, the processing sequence of the images includes ordering the partial digital images with increasing distance from a center image of the infrastructure surface, and incrementally positioning each of the partial digital images starting from the center image.

[0127] Example embodiment 10 includes the method of any one of example embodiments 1 to 9, with or without optional features. In this example embodiment, providing the plurality of partial digital images includes capturing the partial digital images parallel to the infrastructure surface.

[0128] Example embodiment 11 includes the method of any one of example embodiments 1 to 10, with or without optional features. In this example embodiment, providing the plurality of partial digital images includes capturing the partial digital images such that the image planes are not parallel to the captured surface, and digitally de-skewing the partial digital images in order to align the partial digital images parallel to the captured surface.

[0129] Example embodiment 12 includes the method of any one of example embodiments 1 to 11, with or without optional features. In this example embodiment, providing the plurality of partial digital images includes capturing the partial digital images on a regular grid of the infrastructure surface.

[0130] Example embodiment 13 is an image stitching system that stitches a plurality of digital images of an infrastructure surface for defect detection. The system includes a memory for storing program code portions. The memory is coupled to a processor such that, when the program code portions are executed, the processor is capable of receiving a plurality of partial digital images of an infrastructure surface. When the program code portions are executed, the processor is also capable of extracting global positioning system metadata from data of the partial digital images. When the program code portions are executed, the processor is also capable of scheduling a processing sequence of the images based on the extracted global positioning system metadata. When the program code portions are executed, the processor is also capable of determining feature descriptions in one or more of the partial digital images. When the program code portions are executed, the processor is also capable of executing the scheduled processing sequence to determine an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally position each of the partial digital images to progressively produce a full view image of the infrastructure surface by digitally stitching the plurality of partial digital images together.

[0131] Example embodiment 14 includes the system of example embodiment 13, including or excluding optional features. In this example embodiment, the processor, when executing the portion of program code, is further capable of rendering a topographic image and mapping the identified defects, annotations, or measurements into the topographic image.

[0132] Example embodiment 15 includes the system of any of example embodiments 13 or 14, including or excluding optional features. In this example embodiment, the processor, when executing the portion of program code, is further capable of performing at least one activity selected from the group consisting of scene registration, image warping, and pixel-usage comparison for a time-evolution assessment of the plurality of partial digital images of the infrastructure surface having different timestamps.

[0133] Example embodiment 16 includes the system of any of example embodiments 13 to 15, including or excluding optional features. In this example embodiment, the plurality of partial digital images are captured by a camera of an unmanned vehicle.

[0134] Example embodiment 17 includes the system of any of example embodiments 13 to 15, including or excluding optional features. In this example embodiment, the processor, when executing the portion of program code, is further caused to determine feature descriptors to detect points of interest.

[0135] Example embodiment 18 includes the system of any of example embodiments 13 to 17, including or excluding optional features. In this example embodiment, the processor, when executing the portion of program code, is further capable of performing a RANSAC method in determining the affinity matrix for incrementally locating each of the partial digital images.

[0136] Example embodiment 19 includes the system of example embodiment 13, including or excluding optional features. In this example embodiment, the processor, when executing the portion of program code, is further caused to capture the partial digital images such that the image plane is not parallel to the captured surface when providing the partial digital images, and to digitally de-skew the partial digital images so as to align the partial digital images parallel to the captured surface.

[0137] Example embodiment 20 includes a computer program product for image stitching of a plurality of digital images of an infrastructure surface for defect detection. The computer program product includes a computer readable storage medium having program instructions. The program instructions are executable by one or more computing systems or controllers to cause the one or more computing systems to receive a plurality of partial digital images of an infrastructure surface. The program instructions are executable to cause the one or more computing systems to extract global positioning system metadata from data of the partial digital images. The program instructions are executable to cause the one or more computing systems to schedule a processing sequence of the images based on the extracted global positioning system metadata. The program instructions are executable to cause the one or more computing systems to determine feature descriptions in the one or more partial digital images. The program instructions are executable to cause the one or more computing systems to perform the scheduled processing sequence to determine an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally locate each of the partial digital images to progressively produce a full view image of the infrastructure surface by digitally stitching together the plurality of partial digital images.

Claims

1. A computer-implemented method of image stitching of a plurality of digital images of an infrastructure surface for defect detection, the method comprising: providing a plurality of partial digital images of an infrastructure surface; extracting global positioning system metadata from data corresponding to the partial digital images; generating a graph based on the extracted global positioning system metadata, wherein each of the plurality of partial digital images is represented by a point on the graph and the points are organized on the graph in a manner that represents the locations of the plurality of partial digital images relative to each other; ordering the plurality of partial digital images with increasing geometric distance to a center image of the infrastructure surface, the center image determined to be at the center of a topographic image of the infrastructure surface; based on the ordering of the plurality of partial digital images, labeling the points on the graph with numbers that indicate the ordering; determining a processing sequence for processing the plurality of partial digital images based on the graph and the extracted global positioning system metadata; determining feature descriptions of features in the plurality of partial digital images; and using the feature descriptions of adjacent partial digital images to determine an affinity matrix to incrementally position each of the partial digital images such that a topographic image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

2. The method of claim 1, further comprising: rendering the topographic image; and mapping at least one of the identified defects, annotations, or measurements onto the topographic image.

3. The method of claim 1, further comprising performing at least one of scene registration, image warping, or pixel-use comparison for time-evolution assessment of the plurality of partial digital images of the infrastructure surface having different timestamps. providing the plurality of partial digital images includes using a camera of an unmanned vehicle.

4. The method of claim 1, wherein, determining feature descriptions includes using a SIFT method.

5. The method of claim 1, wherein, determining feature descriptions includes detecting interest points.

6. The method of claim 1, wherein, determining the affinity matrix further includes using a RANSAC method to incrementally position each of the partial digital images.

7. The method of claim 1, wherein, providing the partial digital images includes capturing the partial digital images at an angle parallel to the infrastructure surface.

8. The method of claim 7, wherein, incrementally positioning each of the partial digital images comprises determining a transformation matrix H i stitching to an already existing intermediate stitched image S i i .​ 9. The method of claim 1, wherein, providing the partial digital images includes:

10. The method of claim 1, wherein, capturing the partial digital images such that image planes are not parallel to the captured surface, and digitally de-skewing the partial digital images to align the partial digital images parallel to the captured surface. providing the partial digital images includes capturing the partial digital images on a regular grid of an infrastructure surface.

11. The method of claim 1, wherein, 12. An image stitching system of a plurality of digital images of an infrastructure surface for defect detection, the system comprising: a memory for storing program code portions, the memory coupled to a processor that, when executing the program code portions, enables the processor to: receive a plurality of partial digital images of an infrastructure surface; extract global positioning system metadata from data corresponding to the partial digital images; ​ generate a graph based on the extracted global positioning system metadata, wherein each of the plurality of partial digital images is represented by a point on the graph and the points are organized on the graph in a manner that represents the positions of the plurality of partial digital images relative to each other; sort the plurality of partial digital images with an increased geometric distance to a center image of the infrastructure surface, the center image being determined as being in the center of a topography image of the infrastructure surface; based on the sorting of the plurality of partial digital images, label the points on the graph with numbers that indicate the sorting; determine a processing sequence for processing the plurality of partial digital images based on the graph and the extracted global positioning system metadata; determine feature descriptions of features in the plurality of partial digital images; and determine an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally position each of the partial digital images such that a topography image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

13. The system of claim 12, wherein, When the program code portions are executed, the processor is further enabled to render the topography image and map the identified defects, annotations, or measurements onto the topography image.

14. The system of claim 12, wherein, When the program code portions are executed, the processor is further enabled to perform at least one of scene registration, image warping, and pixel-usage comparison for a time-evolution assessment of a plurality of partial digital images of the infrastructure surface having different timestamps.

15. The system of claim 12, wherein, The plurality of partial digital images are captured by a camera of an unmanned vehicle.

16. The system of claim 12, wherein, When the program code portions are executed, the processor is further enabled to detect points of interest when determining the feature descriptions.

17. The system of claim 12, wherein, When the program code portions are executed, the processor is further enabled to perform a RANSAC method for incrementally positioning each of the partial digital images when determining the affinity matrix.

18. The system of claim 12, wherein, When the program code portions are executed, the processor is further enabled to capture each partial digital image such that the image plane is not parallel to the captured surface when providing the partial digital images, and to digitally de-skew each partial digital image in order to align the partial digital images to be parallel to the captured surface.

19. A computer program product for image stitching of a plurality of digital images of an infrastructure surface for defect detection, the computer program product comprising program instructions executable by one or more computing systems or controllers to cause: receiving a plurality of partial digital images of an infrastructure surface; extracting global positioning system metadata from data corresponding to the partial digital images; generating a graph based on the extracted global positioning system metadata, wherein each of the plurality of partial digital images is represented by a point on the graph and the points are organized on the graph in a manner that represents the positions of the plurality of partial digital images relative to each other; ordering the plurality of partial digital images with an increased geometric distance to a center image of the infrastructure surface, the center image being determined as a center of a topographic image of the infrastructure surface; based on the ordering of the plurality of partial digital images, marking the points on the graph with numbers indicative of the ordering; determining a processing sequence for processing the plurality of partial digital images based on the graph and extracted global positioning system metadata; determining feature descriptions of features in the plurality of partial digital images; and determining an affinity matrix using the feature descriptions of adjacent partial digital images to incrementally position each of the partial digital images such that a topographic image of the infrastructure surface is produced by iteratively digitally stitching together the plurality of partial digital images.

Images