Maintaining original colors when applying super resolution to individual bands

Abstract

A system and method for generating super-resolution images from geospatial images with any number of bands. A super-resolution model using a deep convolutional neural network (DCNN) is trained using individual image bands, a large crop or tile size of 512x512 pixels, and a denoising algorithm. Applying one or more algorithms to preserve the original colors of the image bands improves the quality metrics of the super-resolution image, as measured by the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the super-resolution image. Furthermore, applying one or more algorithms to remove boundary effects introduced during the disclosed processing reduces and / or removes seam lines between tiles, enhancing the overall accuracy of the super-resolution image.

Description

[Technical Field]

[0001]

[0001] The present disclosure is generally directed to the fields of convolutional neural networks (CNNs) and image super-resolution. More particularly, in one example, the present disclosure relates to deep learning applied to geospatial imagery. In particular, in another example, the present disclosure relates to methods for applying super-resolution (SR) deep learning applications to multispectral and hyperspectral geospatial imagery to infer higher resolution images from lower resolution images while preserving the original colors of the lower resolution pixels. [Background technology]

[0002]

[0002] Geospatial imagery is an image of the Earth's surface acquired using airborne or spaceborne imaging systems, including, for example, aircraft or satellites. These images are useful for several reasons, including studying the natural and man-made environment. Monitoring glacier melt, river valley erosion, and coastline changes are some aspects of the natural environment that can be studied and mapped using geospatial imagery. A mapped city showing roads, buildings, parking lots, traffic flow, etc. is part of the man-made environment that can be monitored using geospatial imagery. These images are also useful for creating maps or mapping software.

[0003]

[0003] Satellite-based geospatial imagery therefore plays an important role in many modern applications, including, but not limited to, navigation, tracking, search and rescue, and scientific research. Other military-based applications may utilize satellite-based imagery for reconnaissance and intelligence gathering, including applications such as tracking the movement of troops and equipment and / or monitoring sensitive sites. The value of these satellite-based images depends heavily on image quality and resolution, which may be limited by the low-resolution instruments found on most geospatial imaging satellites, the large ground sample distance (GSD), or a combination of both.

[0004]

[0004] The resolution of geospatial images is particularly important because some of them are collected from space stations such as satellites that may be hundreds of miles above the ground. For example, the accuracy of object detection from satellite images depends on their resolution, or ground sample distance. Resolution is particularly important for small objects such as vehicles, which may cover fewer than 20 pixels. Developing a high-resolution space imaging system is very expensive and difficult.

[0005]

[0005] Furthermore, the cost of deploying new, more advanced, high-resolution sensors or imaging equipment in Earth orbit is prohibitively expensive, approaching or exceeding $1 billion per satellite launch. Similarly, because launching a manned crew is expensive, time-consuming, and risky, updating the physical hardware of existing geospatial imaging satellites is impractical and even prohibitively expensive, and the cost of launching an alternative, i.e., a robotic system operable to update existing satellite hardware, far exceeds the cost of launching and deploying a new high-resolution sensor.

[0006] An additional challenge for super-resolution geospatial images, particularly satellite images, is that they may have four or more wavelength bands (e.g., colors such as red, green, and blue (RGB)). Current processing generally limits images to RGB coloration. Many applications that use geospatial images can benefit from four or more band images; however, no algorithms are known that can effectively super-resolve images from four or more band spaces.

[0007] Current backend processing algorithms for improving image resolution tend to operate by combining multiple color bands into a single band before applying algorithmic resolution processing. While this may allow the backend processing to somewhat improve the resolution of geospatial images, it does so at the expense of the accuracy of the image's color bands. Because false color bands and / or false color resolution can cause misidentification of objects and / or targets or prevent recognition of targets that are entirely present in the image, inaccurate color bands can result in subsequent errors in image analysis. In other words, the effect of applying a super-resolution algorithm to a combined color band consisting of multiple merged individual color bands can increase subsequent false positive results, reduced recall, and reduced precision. A false positive can occur when something is identified as presenting an image that does not actually exist in real life (true positive). Recall is measured as the ratio of true positives to the total number of detected objects and is therefore reduced by false positives. Similarly, accuracy is measured as the ratio of true positives to total positives (ie, the sum of true and false positives), which is also reduced by an increased number of false positive results.

[0008]

[0008] Thus, the tradeoff of current systems that apply super-resolution algorithms to combined color bands is increased resolution in the geospatial imagery that is less precise and accurate than desired and may produce false positives.

[0009]

[0009] Current super-resolution algorithms do not perform accurately when images are noisy, such as in satellite imagery. Image noise can adversely affect training, so images must be denoised before they can be used for training. Training is the process of learning from data. Summary of the Invention

[0010]

[0010] The present disclosure addresses these and other problems by providing a system and method for preserving original colors when applying super-resolution to individual bands while minimizing changes to super-resolution pixel values. Changes to super-resolution pixel values ​​are independent of each other when applying super-resolution to individual bands. Therefore, these changes may cause changes to its color. To maintain its original color, the super-resolution pixel values ​​are modified. There are an almost infinite number of combinations for modifying the super-resolution pixel values ​​to maintain its original color. The present disclosure minimizes modifications to the super-resolution pixel values ​​while preserving the original color, producing a super-resolution image with greater accuracy.

[0011]

[0011] The present disclosure addresses the noisy image problem by providing a denoising method. For noisy pixel values, the difference between ground truth pixel values ​​and derived pixel values ​​is large. These differences are used to denoise before using them as training pixels. The derived pixel values ​​are generated by applying Gaussian blurring, downsampling, and upsampling. The denoising method produces a super-resolution satellite image with higher accuracy.

[0012]

[0012] The value of satellite imagery is highly dependent on image quality and resolution, and these two components can now be enhanced through advanced artificial intelligence / machine learning (AI / ML) image processing techniques. While the algorithms used in image processing cannot create new information, deep learning, a branch of artificial intelligence (AI), can be used to create higher-quality, higher-resolution, multispectral, and hyperspectral imagery, also known as super-resolution, from lower-resolution satellite imagery. When applying ultra-deep convolutional neural network learning to the geospatial domain, these super-resolution images can support manual and automated image analysis workflows. Therefore, deep learning is particularly useful for analyzing geospatial imagery.

[0013] In one aspect, an exemplary embodiment of the present disclosure may provide a method for training a super-resolution model, the method comprising: supplying a plurality of training images to a deep convolutional neural network (DCNN) processor; identifying at least one pixel having a noisy pixel value; scaling the noisy pixel to minimize any differences between the at least one noisy pixel and an associated ground truth pixel value; inputting a low-resolution test image to the super-resolution model; selecting a plurality of pixels from the low-resolution test image that are neighbors to a plurality of super-resolution pixels inferred from the DCNN; identifying an original pixel from the training image that has a minimum pixel value difference from each super-resolution pixel; and generating a super-resolution test image for each of the plurality of original training images using the plurality of super-resolution pixels. This or another exemplary embodiment may further provide that the plurality of pixels from the low-resolution test image further comprise one of a group of nine pixels neighboring the plurality of super-resolution pixels inferred from the low-resolution test image and a group of four pixels neighboring each individual super-resolution pixel inferred from the low-resolution test image. This or another exemplary embodiment may further provide for adding a super-resolution test image to a new group of training images when a quality metric of the super-resolution test image falls below a predetermined threshold. This or another exemplary embodiment may further provide for retraining the super-resolution model using the new group of training images including each super-resolution test image with a quality metric below a predetermined threshold. This or another exemplary embodiment may further provide for the calculated quality metric to further comprise at least one of a peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM).

[0014]

[0014] In another aspect, an exemplary embodiment of the present disclosure may provide a computer program product including at least one non-transitory computer-readable storage medium in operative communication with a processor, and a framework having an active data storage and an execution protocol, wherein the execution protocol is coupled to the active data storage, and the storage medium having instructions stored thereon, when executed by the processor, implements a method for generating a super-resolution geospatial image from a lower resolution image, the method comprising: generating individual super-resolution image tiles from the lower resolution image utilizing a trained super-resolution model; selecting a plurality of pixels from the low resolution image that are adjacent to a plurality of super-resolution pixels inferred from the super-resolution model; identifying original pixels from the lower resolution image that have a minimum pixel value difference from each super-resolution pixel; maintaining the original color of at least one lower resolution image in the image tile by selecting a reduced output area from the super-resolution image tile; removing all boundary effects introduced from the selection of the reduced output area; and generating a super-resolution geospatial image having a higher resolution than the low resolution image. This or another exemplary embodiment may further provide that the plurality of pixels from the low-resolution image further comprises one of a group of nine pixels adjacent to the plurality of super-resolution pixels inferred from the low-resolution image and a group of four pixels adjacent to each individual super-resolution pixel inferred from the low-resolution image. This or another exemplary embodiment may further provide that each super-resolution image tile has a crop size of 512 x 512 pixels. This or another exemplary embodiment may further provide that the reduced output area of ​​each super-resolution image tile is the central 492 x 492 pixels of the each super-resolution image tile.This or another exemplary embodiment may further provide that the instructions further comprise: for each super-resolution pixel, finding a corresponding original pixel to improve at least one quality metric of the super-resolution geospatial image. This or another exemplary embodiment may further provide that the calculated quality metric further comprises at least one of a Peak Signal-to-Noise Ratio (PSNR) and a Structural Similarity Index (SSIM). This or another exemplary embodiment may further provide that maintaining the original colors of the lower resolution image further comprises modifying each super-resolution pixel value to match a color component of the original pixel corresponding to the super-resolution pixel. This or another exemplary embodiment may further provide that modifying each super-resolution pixel to match a color component of the original pixel further comprises calculating the color component by a ratio between an individual band pixel value and a sum of all band pixel values. This or another exemplary embodiment may further provide that the instructions further comprise successively searching various discrete values ​​centered around each super-resolution pixel to find a minimum difference between the super-resolution pixel value and the modified super-resolution pixel value.

[0015] In yet another aspect, an exemplary embodiment of the present disclosure is a system comprising at least one of a camera and a sensor operable to capture geospatial images of a surface; at least one processor operable to perform logical functions in communication with the vehicle and the at least one of the camera and the sensor; and at least one non-transitory computer-readable storage medium encoded with instructions that, when executed by the processor, implement operations to enhance the resolution of the geospatial images, the instructions including receiving the geospatial images as a low-resolution image; inputting the low-resolution image into a previously trained super-resolution model; and generating a super-resolution image utilizing the trained super-resolution model. generating individual super-resolution image tiles from the lower-resolution image using the super-resolution model, selecting a plurality of pixels from the low-resolution image adjacent to a plurality of super-resolution pixels inferred from the super-resolution model, identifying original pixels from the lower-resolution image having a minimum pixel value difference from each super-resolution pixel, preserving original colors of the at least one lower-resolution image in the image tile by selecting a reduced output area from the super-resolution image tile, removing any boundary effects introduced from the selection of the reduced output area, and generating a super-resolution geospatial image having a higher resolution than the low-resolution image. This or another exemplary embodiment may further provide that the plurality of pixels from the low-resolution test image further comprise one of a group of nine pixels adjacent to the plurality of super-resolution pixels inferred from the low-resolution image and a group of four pixels adjacent to each individual super-resolution pixel inferred from the low-resolution test image. This exemplary embodiment or another exemplary embodiment may further provide that the individual super-resolution image tile has a crop size of 512 x 512 pixels, and the reduced output area of ​​the individual super-resolution image tile is the central 492 x 492 pixels of the individual super-resolution image tile.This or another exemplary embodiment may further provide that the instructions further comprise: finding a corresponding original pixel for each super-resolution pixel to improve at least one of a peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM) of the super-resolution geospatial image. This or another exemplary embodiment may further provide that the instructions further comprise: modifying each super-resolution pixel value to match a color component of the original pixel corresponding to the super-resolution pixel by calculating the color component by a ratio between the individual band pixel value and the sum of all band pixel values ​​to maintain the original color of the lower resolution image. This or another exemplary embodiment may further provide that the instructions further comprise: finding a minimum difference between the super-resolution pixel value and the modified super-resolution pixel value by successively searching various discrete values ​​centered around each super-resolution pixel.

[0016]

[0016] Exemplary embodiments of the present disclosure are set forth in the following description, illustrated in the drawings, and particularly and distinctly pointed out and described in the appended claims. [Brief explanation of the drawings]

[0017] [Figure 1]

[0017] FIG. 1 is a schematic diagram of a system for detecting objects of interest in geospatial images, according to one aspect of the present disclosure. [Figure 2A]

[0018] 1 illustrates a first step of a method for inferring higher resolution pixels from lower resolution pixels, according to one aspect of the present disclosure. [Figure 2B]

[0019] FIG. 10 illustrates a second step of a method for inferring higher resolution pixels from lower resolution pixels, according to one aspect of the present disclosure. [Figure 2C]

[0020] FIG. 10 illustrates a third step of a method for inferring higher resolution pixels from lower resolution pixels, according to one embodiment of the present disclosure. [Figure 3]

[0021] 10 is an exemplary view of a method for removing tile boundary effects by outputting the central 492x492 pixels from among 512x512 pixels, according to one aspect of the present disclosure. [Figure 4]

[0022] 1 is a flowchart illustrating a process for generating super-resolution multispectral and hyperspectral geospatial images while preserving original colors and removing image boundary effects, according to one aspect of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018]

[0023] Like numbers refer to like parts throughout the drawings.

[0019]

[0024] This disclosure relates to systems and methods for generating super-resolution multispectral and hyperspectral satellite imagery. Multispectral satellite imagery tends to have 3 to 10 color wavelength bands, while hyperspectral satellite imagery tends to have hundreds of narrow bands. Both multispectral and hyperspectral imagery are generally low resolution, making it particularly useful to apply super-resolution algorithms to the imagery.

[0020]

[0025] As described herein, super-resolution algorithms (referred to herein as super-resolution) can be applied to individual bands of multispectral and hyperspectral satellite imagery to overcome the limitations of the Y luminance, Cb blue-difference, and Cr red-difference (YCbCr) space, which is typically intended for use with only three bands. Furthermore, although super-resolution is intended to improve image resolution rather than change the color of the image, preserving the original color of multispectral and hyperspectral imagery when applying super-resolution to individual bands can improve quality metrics.

[0021]

[0026] Referring to FIG. 1 , the Earth “E” is shown with a vehicle, such as an airplane 10 or a satellite 12, moving off above the surface of the Earth “E.” The airplane 10 or satellite 12 may be equipped with one or more cameras 14 or sensors 16 for capturing images of an area of ​​the Earth’s surface as the airplane 10 or satellite 12 moves over the Earth “E.” While the cameras 14 and sensors 16 are shown in FIG. 1 as a camera 14 on the airplane 10 and a sensor 16 on the satellite 12, it should be understood that the cameras 14 and / or sensors 16 may be interchangeable and may be used with any suitable vehicle or platform as needed or dictated by the desired implementation. It will be understood that the term “geospatial imagery,” as used herein, includes multispectral and hyperspectral imagery captured by any type of vehicle, camera, sensor, etc., located at any suitable distance above the surface of the Earth “E.”

[0022]

[0027] As one or more geospatial images are captured by camera 14 and / or sensor 16, those geospatial images may be wirelessly uploaded as input data (shown as signal lines at reference numeral 18 and referred to herein as image data 18) to a database stored on one or more central processing units (CPUs) 20. CPU 20 may be any suitable processing unit, including, but not limited to, a processor or series of processors or one or more logic, logic controllers, etc. CPU 20 may include or be in communication with at least one non-transitory storage media device, as further described herein, and may further be operable to execute a set of instructions encoded on the one or more non-transitory storage media devices, including instructions to process and apply super-resolution to one or more images from image data 18.

[0023]

[0028] The CPU 20 may be linked to or in communication with one or more graphics processing units (GPUs) 22, as described herein, which may be any suitable GPU 22 operable to receive, store, and / or display image data 18 as one or more geospatial images. The GPU 22 may further include an ultra-deep super-resolution convolutional neural network (DCNN) 24 (also referred to herein as a deep CNN 24 or simply CNN 24). The CPU 20 and GPU 22 (including one or more non-transitory storage media devices) may be pre-programmed with multiple algorithms (described later herein) designed to execute, operate, or implement one or more super-resolution algorithms on the image data 18 to enhance the resolution of the multispectral and hyperspectral images captured by the camera 14 and / or sensor 16.

[0024]

[0029] The processes described herein may be best understood through a discussion of their applications and advantages. In particular, implementations of the processes described herein may be achieved via any suitable system utilizing suitable camera / sensor equipment in connection with a suitable processor or processing system, including legacy systems. In particular, the processes described herein represent improved processing of geospatial imagery captured by one or more legacy systems, and these processes may be implemented without significantly increasing the size, weight, power, or cost of existing legacy systems. Thus, the innovations in the processing and handling of geospatial imagery of the present disclosure necessarily derive from improvements in existing computer technology and processing of geospatial imagery and arise in the field of computer image processing, and more particularly, in the field of processing geospatial imagery to improve image resolution and accuracy. Furthermore, it will be understood that improvements in existing technology-based functionality do not lie in implementations utilizing specific hardware, but instead in improvements in the functionality and enhanced performance of executing the processes described herein.

[0025]

[0030] Thus, an ultra-deep CNN, such as CNN24, used in artificial intelligence / machine learning (AI / ML) workflows, can learn a transformation between different zoom levels of an image pyramid, also called a resolution set (RSet). The CNN can learn a transformation from a 2:1 RSet at a 60 cm ground sample distance (GSD) to a full-resolution image at a 30 cm GSD by minimizing the difference between the ground truth full resolution and the derived 2x zoom. As used herein, a ground truth pixel is a pixel from the original or raw image. When a super-resolution model is trained, as described herein, the pixel of the original image is used as the ground truth pixel. In other words, the trained super-resolution model modifies the processed pixel values ​​to match the pixel values ​​of the original image. Therefore, the pixel values ​​of the original image are treated as ground truth pixel values. After training, the learned transformation can be applied to the 1:1 RSet of the full-resolution image to transform the pixels to the 2x resolution. The learned transformation, i.e., CNN super-resolution model, has built-in intelligence and can infer higher resolution images.

[0026]

[0031] Training the CNN 24 involves supplying the CPU 20 with a vast number of diverse training images. The images may cover a wide variety of corners, lines, curves, edges, and shading of houses, buildings, streets, bridges, trees, vehicles, airplanes, ships, etc. For example, the conversion between RSet2:1 and RSet1:1 for thin lines (1 or 2 pixels) is different from that for thick lines (more than 2 pixels). Training and generating super-resolution images is an iterative process. When a super-resolution image is of insufficient accuracy, this image is added to the training image set to retrain the model so that the trained model can generate high-accuracy super-resolution images.

[0027]

[0032] A CNN super-resolution model needs to have sufficient model capacity for the vast number of transformations from RSet2:1 to RSet1:1. Training a CNN super-resolution model is resource intensive and can easily take days and weeks depending on the computer hardware. Training image selection is key to reducing training time by eliminating redundant training images. For example, an image with billions of pixels covering a large city should not be used in its entirety for training. Instead, a small subset of this large image should be used for training.

[0028]

[0033] During the training of the CNN super-resolution model, individual image bands are used as training image chips. During the generation stage described below, the super-resolution model can be applied to individual image bands successively. The final super-resolution image is generated by minimizing modifications to the super-resolution pixel values ​​while preserving the original colors.

[0029]

[0034] For three band images, the image may be converted from RGB space to YCbCr space, and the super-resolution algorithm may be applied only to the Y band. As mentioned above, RGB bands are not an efficient representation for storage and transmission because they have a lot of redundancy and there are no known mathematical formulas that can convert from a four or more band space to a space similar to YCbCr space.

[0030]

[0035] The present disclosure relates to a system and method for improving the resolution of multispectral and hyperspectral satellite images by utilizing a super-resolution algorithm for satellite images with three or more bands. The disclosed super-resolution algorithm enables the system to maintain the original color of the image when applying super-resolution to individual bands, resulting in greater accuracy. For example, for a 322,961,408-pixel super-resolution image, the quality metrics PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) can be improved from approximately 33.4067 (PSNR) and 0.948264 (SSIM) to approximately 35.4385 and 0.956724, respectively, after applying a super-resolution algorithm to maintain the original color as described below.

[0031]

[0036] According to one embodiment, a pixel color may be defined by its component ratios. For example, the component ratios for an 8-band pixel (243, 331, 339, 177, 129, 327, 811, 524) are (243 / 2881, 331 / 2881, 339 / 2881, 177 / 2881, 129 / 2881, 327 / 2881, 811 / 2881, 524 / 2881) or (8.43%, 11.49%, 11.77%, 6.14%, 4.48%, 11.35%, 28.15%, 18.19%).

[0032]

[0037] Changes to super-resolution pixel values ​​are independent of each other when applying the super-resolution algorithm to individual bands. Therefore, these changes may result in changes to the color of a particular band. To maintain the original color, the super-resolution pixel values ​​may be modified. There are an almost infinite number of combinations for modifying the super-resolution pixel values ​​to maintain the original color. The super-resolution algorithm of the present disclosure minimizes modifications to the super-resolution pixel values ​​while maintaining the original color and producing a super-resolution image with greater accuracy.

[0033]

[0038] The super-resolution algorithm of the present disclosure may further operate to identify the location of an original pixel for each super-resolution pixel. As described herein, the original pixel may be one of nine neighboring pixels or one of four neighboring pixels in the original image. The original pixel has the smallest pixel value difference with the super-resolution pixel.

[0034]

[0039] According to another aspect, the present system and method may provide iterative training, which may remove redundant training image chips and reduce training time. In this iterative training, which is described further below, a large training image may be divided into tiles, such as tiles containing 512 x 512 pixels. Initially, a small number of tiles may be used for training. After training, test image tiles may then be used to generate super-resolution images. For each test image tile, its quality metrics, such as PSNR and SSIM, may be calculated. If the quality metrics of the test image tile are of insufficient accuracy, the test image tile may be added to the set of training image tiles for future training of the CNN model. If the quality metrics of the test image tile are of high accuracy, the test image tile is not added to the set of training image tiles. When there are enough additional image tiles added back to the set, a new iteration of training begins. During this iterative training process, new images may be divided into new test image tiles and used to generate super-resolution images. As before, if their quality metrics are of insufficient accuracy, they are added to a new set of training image tiles and used in the next iteration of training. In this disclosure, an image may be considered "low accuracy" if the quality metric falls below a predetermined threshold.

[0035]

[0040] The process for generating a super-resolution image described below involves removing boundary effects between tiles. The super-resolution CNN model generates a super-resolution image tile by tile using a tile size of 512x512 pixels. This tends to cause the accuracy at the boundaries of the 512x512 tiles to be less accurate than the accuracy near the center of the tile. To counteract these boundary effects, the super-resolution algorithm uses only the central area of ​​the tile for the final super-resolution image by generating overlapping tiles. According to one embodiment, these images may overlap by approximately 10 pixels. Thus, for a 512x512 pixel tile, only the central 492x492 pixels are used for the final super-resolution image.

[0036]

[0041] According to another aspect, the system of the present disclosure can significantly improve AI / ML object detection accuracy from super-resolution geospatial imagery. The system can significantly improve manual feature extraction accuracy by enabling image analysts to place extraction cursors on precise edges and corners of man-made objects, which can benefit all image analysis workflows and derived products that use satellite imagery. The system can further include a large crop size of 512 pixels for both training and super-resolution generation.

[0037]

[0042] Having now generally described the system and method, specific algorithms and processes will now be described.

[0038]

[0043] Determining higher quality pixels from lower resolution pixels becomes feasible using deep learning. "Image processing cannot create new information" is a long-held truth in remote sensing and photogrammetry. Deep learning CNNs can learn transformations from lower resolution images to higher resolution images for millions of different examples, and these learned transformations can then be used to infer even higher resolution images.

[0039]

[0044] The following description uses a simple example to show how higher resolution pixels can be inferred from lower resolution pixels.

[0040]

[0045] Referring to Figure 2A, there are 12 pixels (individual boxes indicated by reference numeral 26) with a GSD of 30 cm, representing an RSet of 1:1. These 12 pixels 26 represent a central vertical step edge 28 (with pixel intensity values ​​stepping up from 54 to 80 according to this example). The 12 pixels 26 can be grouped into three sets of four, indicated by 30, 32, and 34. When each group of four pixels 30, 32, and 34 is averaged (as indicated by arrow A) into a single pixel 36, 38, and 40, respectively, the result is three single pixels 36, 38, and 40 with a GSD of 60 cm (representing an RSet of 2:1).

[0041]

[0046] Given these three pixels with a GSD of 60 cm, the test is to infer the intensity of the original 12 pixels 26 with a GSD of 30 cm (shown by arrow B). While there is no single definitive solution, it can be seen that the present system and process can reliably interpret the 12 pixels with a GSD of 30 cm from the three pixels with a GSD of 60 cm. In this example, pixel 26 with values ​​54 (p54) and 80 (p80) may be used, with the intensity values ​​of 54 and 80 referring to the unknown values ​​of the original 30 cm pixels. The three pixels 36, 38, and 40 with a GSD of 60 cm may be further denoted as p51 (pixel 36), p67 (pixel 38), and p83 (pixel 40). The names "p54" and "p80" represent the ground truth values ​​of the pixels 26 we are attempting to recover.

[0042]

[0047] Referring to Figure 2B, the three pixels 36, 38, and 40 at the 60 cm GSD can be divided into four pixels (indicated by arrow C) whose intensity values ​​match those of the pixels 36, 38, and 40 at the 60 cm GSD. As can be seen in Figure 2B, the center pixel 38 is highlighted for clarity, but the first and third pixels 36 and 40 are also utilized in this example. Therefore, p54 and p80 are calculated as follows:

[0043]

number

[0044] where w1, w2, and w3 are the weights of p51, p67, and p83, respectively.

[0045]

number

[0046] where f1, f2, and f3 are the weights of p51, p67, and p83, respectively.

[0047]

[0048] Note that the weights p51, p67, and p83 are the same but different from p54 and p80. In the first iteration of CNN training, w1 and f1 are initialized to 0.0, w2 and f2 are initialized to 1.0, and w3 and f3 are initialized to 0.0. Therefore, p54 = p67 = p80. The loss (ground truth minus estimated value) is + / - 13 in both cases.

[0048]

[0049] Initially, w1 and f1 may be assigned to 0.0, w2 and f2 may be assigned to 1.0, and w3 and f3 may be assigned to 0.0. The inferred value p54 is 67, and p80 is then 67. The losses are -13 and +13, respectively. At the next iteration, w1, f1, w2, f2, w3, and f3 may be adjusted so that the loss approaches 0.0. The algorithm that adjusts w1, f1, w2, f2, w3, and f3 based on the loss is called gradient descent, and is a standard CNN algorithm.

[0049]

[0050] Referring to Figure 2C, the values ​​of w1, f1, w2, f2, w3, and f3 may be iteratively updated using gradient descent to minimize the loss (shown by arrow D). The loss never converges to 0, meaning the system can never infer a perfect or zero-loss higher-resolution image, but the CNN will allow the system to achieve a good or low-loss reconstruction of the original image. The more training images used, the closer the system will get to perfectly reconstructing the original higher-resolution image.

[0050]

[0051] Geospatial satellite images are usually noisy, and the inclusion of noisy pixels contributes to a degraded super-resolution model. One of the major factors introducing noise into an image is the distance between the camera and the ground object (the Earth's surface). In the case of satellite imagery, the camera is hundreds of miles above the ground object, and light travels through hundreds of miles of atmosphere, which introduces significant noise. As described herein, significant noise is a major issue in super-resolution because the super-resolution model is intended to minimize the difference between the ground truth or original pixel values ​​and the processed pixel values. Therefore, when training a super-resolution model, the difference between the ground truth pixel values ​​and the derived pixel values ​​needs to be minimized. The derived pixel values ​​are calculated by applying Gaussian blurring, downsampling, and upsampling to the ground truth pixel values. Therefore, the derived pixel values ​​have their noise partially removed. When the ground truth pixels have noisy values, the difference between the ground truth pixel values ​​and the derived pixel values ​​is large. To reduce the effect of noisy pixel values, the denoising algorithms presented below can be used during the training process. These algorithms described below can operate to smooth the original image and reduce noise in the image. Thus, the ground truth pixels have significant noise, but the processed pixels have less noise. When minimizing the difference between them, the model is learning the noise or is adversely affected by the noise rather than the conversion between different resolutions or super-resolutions. Therefore, the processes described herein can operate to reduce noise in the ground truth image.

[0051] Noise Reduction Algorithm Requires: g_p, ground truth pixels. Requires: d_p, derived pixel. Requires: s_g_p, the scaled ground truth pixels used for training. Requires: s_d_p, the scaled derived pixels used for training. We require: c, a scale-down factor which is the maximum pixel value of the ground truth image; Requires: t, a threshold for detecting noise pixels with a default value of 0.5. s_d_p=d_p / c s_g_p=g_p / c if(|s_d_p-s_g_p|>t) ←|| is the absolute value s_g_p=(s_g_p+s_d_p) / 2.0

[0052]

[0052] The CNN model can learn the intelligence to preserve image edges with sharp vertical steps by minimizing a loss function. The CNN model can further learn all transformations that occur between lower-resolution and higher-resolution images, and then apply these learned transformations to the highest-resolution image to infer an even higher-resolution image (i.e., super-resolution). Figures 2A-2C are understood as simple examples, but illustrate similar processing for much more complex CNN models described here and below.

[0053] Original Pixel Algorithm

[0053] Referring to Figure 3, the procedure for finding the corresponding original pixel for any particular super-resolution image pixel will now be described. There are three algorithms utilized in the process of identifying the original pixel: algorithm (1.1) for determining nine neighboring pixel indices in the original image (shown in Figure 3 as pixels 1-9 for any of pixels a-d); algorithm (1.2) for determining four neighboring pixel indices in the original image (shown in Figure 3 as four pixels surrounding any of pixels a-d. For example, for pixel a, the four neighboring indices would be pixels 1, 2, 4, and 5); and algorithm (1.3) for performing the actual search. As can be seen in Figure 2, the exemplary pixel grid shown is organized in a 3x3 grid with three rows (shown as rows 42A, 42B, and 42C) and three columns (shown as columns 44A, 44B, and 44C). For the first two rows 42A and 42B, the last two rows 42B and 42C, the first two columns 44A and 44B, and the last two columns 44B and 44C of pixels in the super-resolution image, they are calculated slightly differently to address boundary conditions. With respect to the grid shown in Figure 3 and described in the algorithm below, the terms sample and column, as used herein, will be understood to be interchangeable, as will the terms row and line.

[0054] Algorithm 1.1 Algorithm for 9 adjacent original pixels FIG. 3 shows an example of nine neighboring pixels (pixels 1-9) in the original image for super-resolution pixels a, b, c, and d. We require: o_i, an index array of neighboring original pixels with size 9. For each super-resolution pixel, this index array stores the pixel indices of the nine neighboring original images. Requires: e_h, an enhancement level that is 2 for 2x super resolution and 4 for 4x super resolution. You need: y and s x , the pixel index in the super-resolution image s. We require: n, the number of samples in the original image; for l=0,...,2 do y=s y / e_h+(l-1) for s=0,...,2 do x=s x / e_h+(s-1) o_i [l*3+s] =y*n+x end for s end for l

[0055] Algorithm 1.2 Algorithm for four adjacent original pixels. FIG. 3 shows an example of four neighboring pixels in the original image for a pixel a in super-resolution. We need: o_i, an index array of neighboring original pixels with size 4. For each super-resolution pixel, this index array stores the pixel indices of its four neighboring original images. Requires: e_h, an enhancement level that is 2 for 2x super resolution and 4 for 4x super resolution. You need: y and s x , the pixel index in the super-resolution image s. We require: n, the number of samples in the original image; for l=0,...,1 do y=s y / e_h+(l-1) for s=0,...,1 do x=s x / e_h+(s-1) o_i [l*2+s] =y*n+x end for s end for l

[0056] Algorithm 1.3 Original pixel algorithm. This procedure determines the original image pixel (m_d_i), which has the smallest pixel value difference from the super-resolution pixel. m_d_i is one of o_i from Algorithm 1.1 or 1.2. Requires: o, m×n original image. We need: m, the number of lines in the original image. We require: n, the number of samples in the original image; Requires: e_h, an enhancement level that is 2 for 2x super resolution and 4 for 4x super resolution. We need: s, a super-resolution image of (e_h*m)×(e_h*n). Requires: o_i, the index array of neighboring original pixels from algorithm 1.1 or 1.2. You need: y and s x , the pixel index in the super-resolution image s. Requires: n_b, number of bands Requires: n_v_a_p, the number of neighboring original pixels, 9 or 4. m_d=999999.9 ← Initialize the minimum pixel value difference for i=0,...,n_v_a_p-1 do d_s=0.0 ← Initialize pixel value difference for b=0,...,n_b-1 do or_i=o_i[i]+b*n*m ←pixel index in the original image sr_i=sy*n*e_h+s x +b*e_h*n*e_h*m ←Pixel index in the SR image d=o [or_i] -s [sr_i] if (d>=0) d_s+= else d_s-=d end for b if (d_s <m_d) m_d = d_s m_d_i=i end for i

[0057] Table 1 below shows an example of a super-resolution pixel "Super", its corresponding original pixel "Original", and its output after performing algorithms 1.1, 1.2, and 1.3 that preserve original color (collectively referred to as "MOC" for preserving original color). The number of bands is 8. The number of bits is 11, and the pixel values ​​range from 0 to 2047. The color components of the original pixel are in the "o com" column. The color components of the super-resolution pixel are in the "s com" column. The color components of the output super-resolution pixel are in the "MOC com" column. After performing a combinatorial search of 21 discrete values, the optimal solution is in the "Optimal" column. The total pixel value difference between "Super" and "MOC" is 141. The total pixel value difference between "Super" and "Optimal" is 140. This indicates that the "MOC" solution is very close to the optimal solution. The optimal solution requires 37,822,859,361 (21 * 21 * 21 * 21 * 21 * 21 * 21 * 21) iterations, while "MOC" requires only 6561 (3 * 3 * 3 * 3 * 3 * 3 * 3 * 3) iterations when using the outputs from Algorithms 2.1 and 2.2 as input. The optimal solution is impractical with current computer hardware. The MOC solution can be performed in real time. Based on a 322,961,408-pixel, 8-band, 11-bit super-resolution image, the quality metric PSNR improves from approximately 33.4067 to 35.4385, and the SSIM quality metric improves from approximately 33.4067 to 0.956724. The higher the PSNR and SSIM, the more accurate the super-resolution image.

[0058] [Table 1]

[0059] Original color preserving algorithm

[0054] We now describe a procedure for performing calculations to preserve the original colors of pixels in a super-resolution image. For each pixel in the super-resolution image, the procedure successively executes algorithms 2.1, 2.2, and 2.3. The output from 2.1 is the input to algorithm 2.2. The output from algorithm 2.2 is the input to algorithm 2.3.

[0060] Algorithm 2.1 Color preservation algorithm. Given an original pixel and a super-resolution pixel, this procedure modifies the super-resolution pixel so that it has a similar or the same color as the original pixel with minimal changes to the super-resolution pixel value. Requires: c_m_s_p, color-preserved super-resolution pixels. Requires: o_p, the original pixel. Requires: s_p, super resolution pixels. Requires: n_i, the number of iterations with a default value of 40; Requires: Δ, delta component with default value of 0.001 or 0.1%. s_o_p=0 ← Sum of the original pixel values ​​of all bands for i=0,...,n_b-1 do s_o_p+=o_p [i] c_m_s_p [i] =s_p [i] end for i c [n_b] ←Color components of the original pixel c2 [n_b] ← The modified color component of the original pixel for i=0,...,n_b-1 do c [i] =o_p [i] / s_o_p end for for i=0,...,n_i-1 do s_c_m_s_p=0 ← Sum of pixel values ​​in super resolution with colors preserved in all bands for b=0,...,n_b-1 do s_c_m_s_p+=c_m_s_p [b] end for b d=0 ← Initialize pixel value difference for b=0,...,n_b-1 do d_m=|c [b] *s_c_m_s_p-s_p [b] | ←|| is absolute value d_l=|(c [b] -Δ)*s_c_m_s_p-s_p [b] | ←|| is absolute value d_r=|(c [b] +Δ)*s_c_m_s_p-s_p [b] | ←|| is absolute value if(d_m<= d_l& d_m<= d_r) d=d_m c2 [b] =c [b] else if(d_l<=d_m&d_l<=d_r) d+=d_l c2 [b] =c [b] -Δ else d+=d_r c2 [b] =c [b] +Δ end for b if(d< m_d) m_d=d for j=0,...,n_b-1 do c_m_s_p [j] =c2 [j] *s_c_m_s_p end for j end for i

[0061] Algorithm 2.2 Discrete value search algorithm. Given an original pixel, a super-resolution pixel, and a color-preserved super-resolution pixel from Algorithm 2.1, this procedure reduces the difference between the color-preserved super-resolution pixel and the super-resolution pixel. Pixel values ​​are discrete integers with a finite number of possible values. For 1-byte or 8-bit images, valid pixel values ​​are, for example, from 0 to 255. For 11-bit images, valid pixel values ​​are from 0 to 2047. To reduce the search range, the output from Algorithm 2.1 is used as the initial value, which is often already very close to its optimal value. Requires: c_m_s_p, color-preserved super-resolution pixels from algorithm 2.1. Requires: o_p, the original pixel. Requires: s_p, super resolution pixels. We need: c, the original color components from Algorithm 2.1 Requires: Δ, delta component with default value of 0.01 or 1%. Requires: s_r, a search range with default value 8 for 1-byte images and 16 for 2-byte images s_v[n_b] ←Starting search value e_v[n_b] ← End search value v[n_b] ← Search value for i=0,...,n_b-1 do s_v [i] =c_m_s_p [i] -s_r ← If < 0, set to 0 e_v [i] =c_m_s_p [i] +s_r ←> If it is the maximum value, set it to the maximum value end for i for i=0,...,n_b-1 do for b=0,...,n_b-1 do for v [b] =s_v [b] ,...,e_v [b] do s_c_m_s_p=v [b] for b2=0,...,n_b-1 do if ( b != b2 ) s_c_m_s_p+= c_m_s_p [b2] end for b2 d=0 ← Initialize pixel value difference c2 [n_b] ← The modified color component of the original pixel for b2=0,...,n_b-1 do d_m=|c [b] *s_c_m_s_p-s_p [b] | ←|| is absolute value d_l=|(c [b] -Δ)*s_c_m_s_p-s_p [b] | ←|| is absolute value d_r=|(c [b] +Δ)*s_c_m_s_p-s_p [b] | ←|| is absolute value if(d_m<= d_l& d_m<= d_r) d+=d_m c2 [b] =c [b] else if(d_l<=d_m&d_l<=d_r) d+=d_l c2 [b] =c [b] -Δ else d+=d_r c2 [b] =c [b] +Δ end for b2 if(d< m_d) m_d=d for j=0,...,n_b-1 do c_m_s_p [j] =c2 [j] *s_c_m_s_p end for j end for v[b] end for b end for i

[0062] Algorithm 2.3: Discrete-valued band unit combination search algorithm. A procedure to perform a combinatorial search of all bands with a very small search range. There are five algorithms for images with three bands, four bands, six and eight bands, and more than eight bands, respectively. The following is for an image with four bands: Requires: c_m_s_p, color-preserved super-resolution pixels from algorithm 2.1 Requires: s_p, super resolution pixels. We require: c, the original color components from Algorithm 2.1; We need: m_d, the minimum difference from Algorithm 2.1. Requires: Δ, delta component with default value of 0.01 or 1%. Requires: s_r, search range. Default value is 9 - number of bands. For 8 bands it is 1. For 4 bands it is 5. For 3 bands it is 6. s_v[4] ←Starting search value e_v[6] ← End search value v[4] ← search value for i=0,...,3 do s_v[i]=c_m_s_p[i]-s_r ←If <0, set to 0 e_v[i]=c_m_s_p[i]+s_r ←> If it is the maximum value, set it to the maximum value end for i c2[4] ← modified color component of the original pixel for v [0] =s_v [0] ,...,e_v [0] do for v [1] =s_v [1] ,...,e_v [1] do for v [2] =s_v [2] ,...,e_v[2] do for v [3] =s_v [3] ,...,e_v [3] do s_c_m_s_p = v [0] + v [1] + v [2] + v [3] d = 0 ← Initialize the pixel value difference for b = 0,...,3 do d_m = |c [b] * s_c_m_s_p - s_p [b] | d_l = |(c [b] - Δ) * s_c_m_s_p - s_p [b] | d_r = |(c [b] + Δ) * s_c_m_s_p - s_p [b] | if (d_m <= d_l & d_m <= d_r) d += d_m c2 [b] = c [b] else if (d_l <= d_m & d_l <= d_r) d += d_l c2 [b] = c [b] - Δ else d += d_r c2 [b] = c [b] + Δ end for b if (d < m_d) m_d = d for j = 0,...,n_b - 1 do c_m_s_p [j] = c2 [j] * s_c_m_s_p end for j<s end for v [3] end for s_v [2] end for v [1] end for v [0]

[0063] When the super-resolution model has a crop size or tile size of 512 x 512 pixels, the effective output tile size is reduced from 512 to 492 to eliminate boundary effects. The number of boundary pixels can be determined experimentally and adjusted based on parameters in a strategy file. The strategy file contains several parameters used to execute all algorithms for super-resolution image generation, including model training. The boundary effects are caused by the super-resolution model. In the boundary areas of tiles, the accuracy of the super-resolution pixels is not as high as that in the central area. Therefore, there are visible seam lines between adjacent tiles. The algorithm set to remove the tile boundary effects (below) outputs only the central area of ​​the tile, and the output SR image does not have visible seam lines between adjacent tiles.

[0064] Algorithm to remove tile border effects Next, we will explain the procedure to remove the tile boundary effect in the super-resolution image. The original image can be partitioned into tiles with a crop size of 512x512 for the super-resolution model. With the number of boundary pixels of 10, the output tile size is 512-10*2=492. Requires: o, m×n original image. We require: m, the number of lines or rows in the original image; Requires: n, the number of samples or columns in the original image. Requires: e_h, an enhancement level that is 2 for 2x super resolution and 4 for 4x super resolution. Requires: t_s, the tile size or crop size, which is 512. Requires: n_b_p, the number of border pixels with a default value of 10 pixels. Requires: n_b, number of bands. o_t_s=t_s / e_h ←Tile size of the original image s_t_s=t_s ←Tile size of super-resolution image o_o_t_s=o_t_s-n_b_p ←Tile size of the output SR image o_s_t_s=s_t_s-e_h*n_b_p ←Tile size of the output SR image n_x_t=n / o_o_t_s ← Number of column tiles if(n% o_o_t_s>0) ←% is integer remainder operation n_x_t+=1 n_y_t=m / o_o_t_s ←Number of row tiles if ( m% o_o_t_s>0) n_y_t+=1 for y_t=0,...,n_y_t-1 do for x_t=0,...,n_x_t-1 do for b=0,...,n_b-1 do Loads tiles of the original image with tile size o_t_s and centers determined by o_o_t_s Upsample by e_h to s_t_s Applying a Super Resolution Model Write the o_s_t_s tile to the SR image, which is 492x492 in the center of a 512x512 image. end for b end for x_t end for y_t

[0065] As previously shown herein, the disclosed method is capable of generating super-resolution images of any number of bands. In multispectral and hyperspectral satellite images, the number of bands is typically between 3 and 10 bands for multispectral and hundreds of narrow bands for hyperspectral. These satellite images have large GSDs and particularly require super-resolution techniques to increase their resolution. The disclosed method, which preserves the original colors, significantly improves the quality metrics of super-resolution images.

[0066]

[0056] Referring to Figure 4, a process for training a CNN and applying a super-resolution algorithm to one or more geospatial images is shown and generally indicated as process 100. In the first component, reference numeral 102, an ultra-deep super-resolution model (such as DCNN 24) can be trained using individual image bands. This can involve multiple geospatial images, any number of bands of which are input to the CPU 20's database to generate multiple training image chips. The multiple training image chips can be of a single band from an image and are 512x512 in size. The multiple training image chips are then utilized to train the ultra-deep super-resolution network DCNN 24. The denoising algorithm presented herein can be utilized by both the CPU 20 and the GPU 22, although the majority of the heavy processing will likely be performed by the GPU 22.

[0067] The output from the first component 102 is a trained super-resolution model (in deep learning, the term "model" is used to indicate that a DCNN 24 network has been instantiated or is ready to be deployed). In the second component 104, which also tends to be performed partially in the CPU 20 and partially in the GPU 22, the system will subsequently use the model from the first component 102 to generate super-resolution image tile (512 x 512 pixel) bands, again by utilizing the original pixel algorithms 1.1-1.3. The output from the second component 104 is input to a third component 106, which outputs only the central 492 x 492 pixels of the image tile to eliminate boundary effects. Once the model is fully trained, the input to the second component 104 can be one or more images of interest having lower resolution. These images can be fed into a DCNN, as described herein, and used to generate super-resolution image tiles to extend the resolution (super-resolution) of the lower-resolution images.

[0068]

[0058] The precision of super-resolution pixels is higher in the center of a tile than at the edges. For pixels close to the tile boundaries, the precision decreases, which can result in seam lines between adjacent tiles. By outputting only pixels in the central area, the algorithm that removes the tile boundary effect from the third component 106 produces a seamless, high-precision super-resolution image. The output from the third component 106 is the input to the fourth component 108.

[0069] As previously described herein, preserving the original color by adjusting the super-resolution pixel values ​​of individual bands to match the color components of the original pixel can significantly improve the super-resolution quality metrics PSNR and SSIM. For each super-resolution pixel, the process described herein finds the most similar original pixel and uses it to preserve the original color. This is because the super-resolution model can shift or translate the original pixels to form the super-resolution image pixels. In other words, one of the main functions of the super-resolution model is this translation / translation between the original pixel and the super-resolution pixel.

[0070] The third component 106 of the process 100 involves selecting the number of boundary pixels. The number of boundary pixels has a default value of 10 pixels. A different number of pixels can be used by changing the default value in a strategy file containing the input parameters used in the process 100. A larger value may improve accuracy, but at the cost of reduced processing speed.

[0071] Continuing with reference to FIG. 4, but with further reference to FIG. 3, in fourth component 108, algorithms 1.1-1.3 and 2.1-2.3 may be utilized to preserve the original color by adjusting the pixel values ​​of individual bands to match the color components of the original pixel. The first two algorithms, 1.1 and 1.2, may determine nine or four neighboring original pixels for a super-resolution pixel. The original pixel must be one of these neighboring pixels. In particular, algorithm 1.1 may determine nine neighboring pixels for super-resolution pixels a, b, c, and d, which have the same nine neighboring original pixels. Algorithm 1.2 may further determine four neighboring pixels. For super-resolution pixel a, the neighboring pixels are pixels 1, 2, 4, and 5. For super-resolution pixel b, the four neighboring pixels are 2, 3, 5, and 6. For super-resolution pixel c, the four neighboring pixels are 4, 5, 7, and 8. For super-resolution pixel d, the four neighboring pixels are 5, 6, 8, and 9.

[0072] Algorithm 1.3 selects an original pixel based on how similar its neighboring pixel is in value to the super-resolved pixel. Applying super-resolution to an individual band can change the color of the band. This is undesirable because super-resolution is intended to improve resolution, not change color. For example, a red car in the original image should appear as a red car in the super-resolved image. To remedy this undesirable side effect, algorithms 2.1, 2.2, and 2.3 can then be applied to preserve the original color while minimizing changes to the super-resolved pixel value.

[0073] Referring to Table 1 above, according to one example, an 11-bit 8-band super resolution pixel is used with the following pixel values ​​(237, 323, 337, 171, 124, 341, 901, 573), listed as "Super" in Table 1. The color components (s com) of the super resolution pixel are (.07882, .10742, .11207, .05687, .04124, .11340, .29963, .19056).

[0074] The original pixel values ​​("original") are (243, 331, 339, 177, 129, 327, 811, 524), and the color components of the original pixel ("o com") are (.08435, .11489, .11767, .06144, .04476, .11350, .28150, .18188), which is significantly different from the color components of the superpixel ("s com").

[0075] Algorithms 2.1, 2.2, and 2.3 are then employed to modify the super-resolution pixel values ​​to (250, 342, 351, 182, 132, 341, 849, 549), shown as "MOC" in Table 1. The modified super-resolution pixel MOC has color components ("MOC com") of (.08344, .11415, .11716, .06075, .04406, .11382, .28338, .18324), which is then very close to or nearly identical to the original color (o com), even though the change to the super-resolution pixel value is only 141. To verify Algorithms 2.1, 2.2, and 2.3, a band-wise combinatorial exhaustive search yields the optimal super-resolution pixel values ​​("optimal") as (253, 345, 354, 183, 133, 341, 856, 554). The optimal solution changes the super-resolution pixel values ​​by 140, which is one less than the 141 from Algorithms 2.1, 2.2, and 2.3. The exhaustive search requires 37,822,859,361 (2^8) iterations, while Algorithm 2.3 requires only 6561 (3^8) iterations when using the outputs from Algorithms 2.1 and 2.2 as input. Although an exhaustive search is impractical with current computer hardware, Algorithms 2.1, 2.2, and 2.3 operate at processing speeds up to 5 million times faster. After applying algorithms 2.1, 2.2, and 2.3, the PSRN and SSIM metrics improve from approximately 33.4067 and 0.948264 to approximately 35.4385 and 0.956724, respectively, for the 322,961,408 pixel super-resolution image.

[0076]

[0066] While the above disclosure relates to geospatial imagery, it should be understood that the same principles applied herein can be utilized for other imagery where it is desirable to improve the resolution of an existing image. Two such examples may include medical imagery and LIDAR point clouds.

[0077]

[0067] The above-described embodiments may be implemented in any of numerous ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code or instructions may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Furthermore, the instructions or software code may be stored on at least one non-transitory computer-readable storage medium.

[0078]

[0068] A computer or smartphone utilized to execute software code or instructions via its processor may also have one or more input / output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output, and a speaker or other sound-generating device for audible presentation of output. Examples of input devices that may be used for a user interface include a keyboard and pointing devices such as a mouse, touchpad, and discretization tablet. As another example, a computer may receive input information through voice recognition or in other audible formats.

[0079]

[0069] Such computers or smartphones may be interconnected by one or more networks of any suitable form, including enterprise networks, and local or wide area networks such as intelligent networks (IN) or the Internet. Such networks may be based on any suitable technology and operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0080] The various methods or processes outlined herein may be coded as software / instructions executable on one or more processors employing any one of a variety of operating systems or platforms. Furthermore, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0081] In this regard, the various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memory, USB flash drives, SD cards, circuitry in a field programmable gate array or other semiconductor device, or other non-transitory or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the present disclosure discussed above. The one or more computer-readable media may be transportable such that the one or more programs stored thereon can be loaded onto one or more different computers or other processors to implement the various aspects of the present disclosure discussed above.

[0082] As used herein, the terms "program" or "software" or "instructions" are used in a generic sense to refer to any type of computer code or set of computer-executable instructions that may be employed to program a computer or other processor to implement various aspects of the embodiments discussed above. Furthermore, it should be appreciated that, according to one aspect, one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.

[0083]

[0073] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0084] Also, data structures may be stored in computer-readable media in any suitable form. For ease of explanation, data structures may be depicted as having fields that are related through their locations in the data structure. Such relationships may similarly be achieved by assigning storage for the fields with locations in the computer-readable media that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information in the fields of a data structure, including through the use of pointers, tags, or other mechanisms that establish relationships between data elements.

[0085]

[0075] All definitions, as defined and used herein, are understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0086] As used herein, "logic" includes, but is not limited to, hardware, firmware, software, and / or combinations of each for performing a function or action and / or causing a function or action from another logic, method, and / or system. For example, based on a desired application or needs, logic may include a software-controlled microprocessor, discrete logic such as a processor (e.g., a microprocessor), an application-specific integrated circuit (ASIC), a programmed logic device, a memory device containing instructions, an electrical device with memory, etc. Logic may include one or more gates, combinations of gates, or other circuit components. Logic may also be implemented entirely as software. When multiple logics are described, it may be possible to integrate the multiple logics into one physical logic. Similarly, when a single logic is described, it may be possible to distribute the single logic among multiple physical logics.

[0087]

[0077] Furthermore, the logic presented herein for achieving the various methods of this system may be directed to improving existing computer-centric or Internet-centric technologies that may not have previous similar versions. The logic may provide specific functionality directly related to structures that address and solve some of the problems identified herein. The logic may also provide significantly more advantages for solving these problems by providing exemplary inventive concepts as specific logical structures and harmonic functions of methods and systems. Furthermore, the logic may also provide specific computer-implemented rules that improve existing technological processes. The logic presented herein extends beyond simply collecting data, analyzing information, and displaying results.

[0088] In the above description, certain terms have been used for brevity, clarity, and understanding. Such terms are used for descriptive purposes and are intended to be broadly construed, so that no unnecessary limitations should be implied therefrom beyond the requirements of the prior art.

[0089] Moreover, the description and illustration of the preferred embodiment of the present disclosure is by way of example, and the present disclosure is not limited to the exact details shown or described.

[0090] As described herein, embodiments of the present disclosure may include one or more electrical, pneumatic, hydraulic, or other similar secondary components and / or systems therein. Accordingly, the present disclosure is contemplated and will be understood to include any necessary operating components thereof. For example, electrical components will be understood to include any suitable and necessary wiring, fuses, etc. for their normal operation. Similarly, any provided air system may include any secondary or peripheral components, such as air hoses, compressors, valves, meters, etc. Furthermore, it will be understood that any connections between various components not explicitly described herein may be made through any suitable means, including more permanent attachment means such as mechanical fasteners or welding. Alternatively, where feasible and / or desirable, the various components of the present disclosure may be integrally formed as a single unit.

[0091] Various inventive concepts may be embodied as one or more methods, examples of which are provided. The acts performed as part of a method may be ordered in any suitable manner. Thus, while an exemplary embodiment may show acts as sequential, embodiments may be constructed in which acts are performed in an order different from that shown, which may include performing some acts simultaneously.

[0092] While various invention embodiments have been described and illustrated herein, those skilled in the art can readily envision a variety of other means and / or structures for performing the functions and / or obtaining one or more of the results and / or advantages described herein, and each such variation and / or modification is deemed to be within the scope of the invention embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the specific application or applications in which the teachings of the present invention are used. Those skilled in the art will recognize or be able to ascertain, using no more than routine experimentation, many equivalents to the specific invention embodiments described herein. Accordingly, it should be understood that the above-described embodiments are presented by way of example only, and that, within the scope of the appended claims and equivalents thereto, invention embodiments may be practiced otherwise than as specifically described and claimed. Inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the inventive scope of the present disclosure, provided that such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0093]

[0083] The above-described embodiments may be implemented in any of numerous ways. For example, embodiments of the technology disclosed herein may be implemented using hardware, software, or a combination thereof. When implemented in software, the software code or instructions may be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Furthermore, the instructions or software code may be stored on at least one non-transitory computer-readable storage medium.

[0094]

[0084] A computer or smartphone utilized to execute software code or instructions via its processor may also have one or more input / output devices. These devices may be used, among other things, to present a user interface. Examples of output devices that may be used to provide a user interface include a printer or display screen for visual presentation of output, and a speaker or other sound-generating device for audible presentation of output. Examples of input devices that may be used for a user interface include a keyboard and pointing devices such as a mouse, touchpad, and discretization tablet. As another example, a computer may receive input information through voice recognition or in other audible formats.

[0095]

[0085] Such computers or smartphones may be interconnected by one or more networks of any suitable form, including enterprise networks, and local or wide area networks such as intelligent networks (IN) or the Internet. Such networks may be based on any suitable technology and operate according to any suitable protocol, and may include wireless networks, wired networks, or fiber optic networks.

[0096] The various methods or processes outlined herein may be coded as software / instructions executable on one or more processors employing any one of a variety of operating systems or platforms. Furthermore, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine code or intermediate code that runs on a framework or virtual machine.

[0097] In this regard, the various inventive concepts may be embodied as a computer-readable storage medium (or multiple computer-readable storage media) (e.g., computer memory, one or more floppy disks, compact disks, optical disks, magnetic tapes, flash memory, USB flash drives, SD cards, circuitry in a field programmable gate array or other semiconductor device, or other non-transitory or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the present disclosure discussed above. The one or more computer-readable media may be transportable such that the one or more programs stored thereon can be loaded onto one or more different computers or other processors to implement the various aspects of the present disclosure discussed above.

[0098]

[0088] The terms "program" or "software" or "instructions" are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that may be employed to program a computer or other processor to implement various aspects of the embodiments discussed above. Furthermore, it should be appreciated that, according to one aspect, one or more computer programs that, when executed, perform the methods of the present disclosure need not reside on a single computer or processor, but may be distributed in a modular manner among several different computers or processors to implement various aspects of the present disclosure.

[0099]

[0089] Computer-executable instructions may be in many forms, such as program modules, executed by one or more computers or other devices. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0100] Also, data structures may be stored in computer-readable media in any suitable form. For ease of explanation, data structures may be depicted as having fields that are related through their locations in the data structure. Such relationships may similarly be achieved by assigning storage for the fields with locations in the computer-readable media that convey the relationship between the fields. However, any suitable mechanism may be used to establish relationships between information in fields of a data structure, including through the use of pointers, tags, or other mechanisms that establish relationships between data elements.

[0101]

[0091] All definitions, as defined and used herein, are understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0102] As used herein, "logic" includes, but is not limited to, hardware, firmware, software, and / or combinations of each for performing a function or action and / or causing a function or action from another logic, method, and / or system. For example, based on a desired application or needs, logic may include a software-controlled microprocessor, discrete logic such as a processor (e.g., a microprocessor), an application-specific integrated circuit (ASIC), a programmed logic device, a memory device containing instructions, an electrical device with memory, etc. Logic may include one or more gates, combinations of gates, or other circuit components. Logic may also be implemented entirely as software. When multiple logics are described, it may be possible to integrate the multiple logics into one physical logic. Similarly, when a single logic is described, it may be possible to distribute the single logic among multiple physical logics.

[0103] Furthermore, the logic presented herein for achieving the various methods of this system may be directed to improvements over existing computer- or Internet-centric technologies that may not have previous similar versions. The logic may provide specific functionality directly related to structures that address and solve some of the problems identified herein. The logic may also provide significantly more advantages for solving these problems by providing exemplary inventive concepts as specific logical structures and harmonic functions of methods and systems. Furthermore, the logic may also provide specific computer-implemented rules that improve existing technological processes. The logic provided herein extends beyond simply collecting data, analyzing information, and displaying results. Furthermore, portions of this disclosure may rely on underlying equalizations derived from specific configurations of equipment or components presented herein. Thus, portions of this disclosure are not directed to abstract concepts as they relate to specific configurations of components. Furthermore, this disclosure and the accompanying claims present teachings that involve more than the performance of well-understood, routine, conventional activities previously known to the industry. In some of the methods or processes of the present disclosure, which may incorporate some aspect of natural phenomena, the process or method step is a new and useful additional feature.

[0104] The articles "a" and "an," as used herein in the specification and claims, should be understood to mean "at least one" unless clearly indicated to the contrary. The phrase "and / or," as used herein in the specification and claims, if at all, should be understood to mean "one or both" of the elements so conjoined, i.e., elements that are jointly present in some cases and disjunctively present in other cases. Multiple elements listed with "and / or" should be construed in the same manner, i.e., "one or more" of the elements so conjoined. Other elements other than the elements specifically identified by the "and / or" clause may optionally be present, whether related or unrelated to those elements specifically identified. Thus, as a non-limiting example, a reference to "A and / or B," when used with open-ended language such as "comprising," may, in one embodiment, refer only to A (possibly including elements other than B); in another embodiment, refer only to B (possibly including elements other than A); in yet another embodiment, refer to both A and B (possibly including other elements); and so forth. As used herein in the specification and claims, "or" should be understood to have the same meaning as "and / or" as defined above. For example, when separating items in a list, "or" or "and / or" should be interpreted as inclusive, i.e., including at least one of the elements of the list, but also including two or more of them, and possibly additional unlisted items. Only terms clearly indicated to the contrary, such as "only one of" or "exactly one of," or, when used in the claims, "consisting of," will refer to the inclusion of exactly one element of the elements of the list.In general, the term "or" as used herein should only be construed as indicating exclusive alternatives (i.e., "one or the other, but not both") when followed by terms of exclusivity such as "any of," "one of," "only one of," or "exactly one of." "Consisting essentially of," when used in the claims, shall have its ordinary meaning as used in the field of patent law.

[0105]

[0095] As used herein in the specification and claims, the phrase "at least one" in reference to a list of one or more elements should be understood to mean at least one element selected from any one or more of the elements in the list of elements, but not necessarily including at least one of every element specifically listed in the list of elements, nor necessarily excluding any combination of elements in the list of elements. This definition also allows for elements other than those specifically identified in the list of elements to which the phrase "at least one" refers, whether related or unrelated to those elements specifically identified, may in some cases be present. Thus, as a non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B," or, equivalently, "at least one of A and / or B") can refer in one embodiment to at least one A, optionally including more than one, but no B (and optionally including elements other than B); in another embodiment to at least one B, optionally including more than one, but no A (and optionally including elements other than A); in yet another embodiment to at least one A, optionally including more than one, and at least one B, optionally including more than one (and optionally including other elements); etc.

[0106] As used herein in the specification and claims, the term "influencing" or a phrase or claim element beginning with the term "influencing" should be understood to mean causing something to happen or bringing about something. For example, influencing an event to occur can result from the actions of a first party even if a second party actually performs or causes the event to occur. In other words, influencing refers to one party providing another party with the tools, objects, or resources to cause the event to occur. Thus, in this example, the claim element "influencing an event to occur" would mean that a first party provides a second party with the tools or resources needed for the second party to perform the event, but the active, single act of providing the tools or resources to cause the event to occur is the responsibility of the first party.

[0107]

[0097] When a feature or element is referred to herein as being "on" another feature or element, it may be directly on the other feature or element, or intervening features and / or elements may also be present. In contrast, when a feature or element is referred to as being "directly on" another feature or element, there are no intervening features or elements present. Also, when a feature or element is referred to as being "connected," "attached," or "coupled" to another feature or element, it will be understood that it may be directly connected, attached, or coupled to the other feature or element, or there may be intervening features or elements present. In contrast, when a feature or element is referred to as being "directly connected," "directly attached," or "directly coupled" to another feature or element, there are no intervening features or elements present. Although described or illustrated with respect to one embodiment, the features and elements so described or illustrated may be applicable to other embodiments. Additionally, it will be appreciated by those skilled in the art that a reference to a structure or feature disposed "adjacent to" another feature may have a portion that overlaps or underlies the adjacent feature.

[0108] Spatially relative terms, such as "below," "below," "lower," "above," "upper," "on," "behind," and "in front of," may be used herein to facilitate describing the relationship of one element or feature shown in a figure to another element or feature. It will be understood that spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation shown in the figures. For example, if a device in a figure were inverted, an element described as "below" or "below" another element or feature would then be oriented "above" that other element or feature. Thus, the exemplary term "below" can encompass both an above and below orientation. A device may be oriented in other ways (rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein will be interpreted accordingly. Similarly, terms such as "upward," "downward," "vertical," "horizontal," "lateral," "transverse," and "longitudinal" are used herein for descriptive purposes only, unless specifically indicated otherwise.

[0109]

[0099] The terms "first" and "second" may be used herein to describe various features / elements, but these features / elements should not be limited by these terms unless the context indicates otherwise. These terms may be used to distinguish one feature / element from another. Thus, a first feature / element discussed herein may be referred to as a second feature / element, and similarly, a second feature / element discussed herein may be referred to as a first feature / element, without departing from the teachings of the present invention.

[0110] An embodiment is an implementation or example of the present disclosure. References herein to "an embodiment," "one embodiment," "some embodiments," "one particular embodiment," "exemplary embodiment," or "other embodiments," etc., mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least some embodiments of the invention, but not necessarily in all embodiments of the invention. Various appearances of "an embodiment," "one embodiment," "some embodiments," "one particular embodiment," "exemplary embodiment," or "other embodiments," etc., are not necessarily all referring to the same embodiment.

[0111]

[0101] When the specification states that a component, feature, structure, or characteristic "may," "might," or "could" be included, that particular component, feature, structure, or characteristic is not required to be included. When the specification or claims refer to "a" or "an" element, it does not mean that there is only one element. When the specification or claims refer to "additional" elements, it does not exclude that there are two or more additional elements.

[0112] As used herein in the specification and claims, including when used in the examples, unless expressly specified otherwise, all numbers may be read as if preceded by the word "about" or "approximately," even if the word does not explicitly appear. The phrase "about" or "approximately" may be used when describing a size and / or location to indicate that the stated value and / or location is within a reasonable expected range of values ​​and / or locations. For example, a numerical value may have a value that is + / - 0.1% of the stated value (or range of values), + / - 1% of the stated value (or range of values), + / - 2% of the stated value (or range of values), + / - 5% of the stated value (or range of values), + / - 10% of the stated value (or range of values), etc. Any numerical range recited herein is intended to include all subranges encompassed therein.

[0113]

[0103] Furthermore, methods embodying the present disclosure may be performed in a sequence different from that described herein. Thus, the sequence of the method should not be read as a limitation unless explicitly stated. It is recognizable that performing some of the method steps in a different order may achieve similar results.

[0114]

[0104] In the claims, as well as in the foregoing specification, all transitional phrases such as "comprise," "include," "carry," "have," "including," "accompany," "hold," "consisting of," and the like, are to be understood to be open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" are closed or semi-closed transitional phrases, respectively, as set forth in the United States Patent Office Manual of Patent Examining Procedures.

[0115] In the above description, certain terms have been used for brevity, clarity, and understanding. Such terms are used for descriptive purposes and are intended to be broadly construed, so that no unnecessary limitations should be implied therefrom beyond the requirements of the prior art.

[0116] Moreover, the description and illustration of various embodiments of the present disclosure are examples, and the present disclosure is not limited to the exact details shown or described.

Claims

1. 1. A method for training a super-resolution model, comprising: providing a plurality of training images to a deep convolutional neural network (DCNN) processor; identifying at least one pixel having a noisy pixel value; scaling the at least one noisy pixel to minimize any difference between the noisy pixel and an associated ground truth pixel value; inputting a low-resolution test image into the super-resolution model; selecting a plurality of pixels from the low-resolution test image that are adjacent to a plurality of super-resolution pixels inferred from the DCNN; identifying an original pixel from the training image having a minimum pixel value difference with each super-resolution pixel; generating a super-resolution test image for each of the plurality of original training images using the plurality of super-resolution pixels; A method comprising:

2. The plurality of pixels from the low resolution test image are groups of nine pixels adjacent to the plurality of super-resolution pixels inferred from the low-resolution test image; and groups of four pixels adjacent to each individual super-resolution pixel inferred from the low-resolution test image. The method of claim 1 , further comprising one of:

3. adding the super-resolution test image to a new group of training images when a quality metric of the super-resolution test image falls below a predetermined threshold; The method of claim 1 further comprising:

4. retraining the super-resolution model using the new group of training images including a test image for each super-resolution having a quality metric below the predetermined threshold. The method of claim 3 further comprising:

5. The calculated quality metric is: At least one of a peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM). The method of claim 3 further comprising:

6. 1. A computer program product comprising: at least one non-transitory computer-readable storage medium in operative communication with a processor; and a framework comprising an active data store and an execution protocol, the execution protocol coupled to the active data store, the storage medium having instructions stored thereon that, when executed by the processor, implement a method for generating super-resolution geospatial images from lower resolution images, the method comprising: generating super-resolution individual image tiles from the lower resolution images utilizing the trained super-resolution model; selecting pixels from the low-resolution image that are adjacent to a plurality of super-resolution pixels inferred from the super-resolution model; identifying an original pixel from the lower resolution image having a minimum pixel value difference from each super-resolution pixel; maintaining original colors of the at least one lower resolution image in the image tile by selecting a reduced output area from the super-resolution image tile; removing any boundary effects introduced from said reduced output area selection; generating a super-resolution geospatial image having a higher resolution than the low-resolution image; A computer program product comprising:

7. The plurality of pixels from the low resolution image are groups of nine pixels adjacent to the plurality of super-resolution pixels inferred from the low-resolution image; and groups of four pixels adjacent to each individual super-resolution pixel inferred from the low-resolution image. The computer program product of claim 6 , further comprising one of:

8. The computer program product of claim 6 , wherein each super-resolution image tile has a crop size of 512×512 pixels.

9. 9. The computer program product of claim 8, wherein the reduced output area of ​​the super-resolution individual image tile is a central 492x492 pixels of the super-resolution individual image tile.

10. The instruction: finding a corresponding original pixel for each super-resolution pixel to improve at least one quality metric of the super-resolution geospatial image; The computer program product of claim 6 further comprising:

11. The calculated quality metric is: At least one of a peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM). The computer program product of claim 6 further comprising:

12. Preserving the original colors of the lower resolution image modifying each super-resolution pixel value to match a color component of the original pixel corresponding to said super-resolution pixel; The computer program product of claim 6 further comprising:

13. Modifying each super-resolution pixel to match the color components of the original pixel includes: Calculating color components by the ratio between the individual band pixel value and the sum of all band pixel values The computer program product of claim 12 further comprising:

14. The instruction: Successively searching various discrete values ​​centered around each super-resolution pixel to find the minimum difference between the super-resolution pixel value and the modified super-resolution pixel value. The computer program product of claim 13 further comprising:

15. 1. A system comprising: at least one of a camera and a sensor operable to capture a geospatial image of the surface; at least one processor capable of performing logical functions in communication with a vehicle and the at least one of the camera and the sensor; at least one non-transitory computer-readable storage medium encoded with instructions that, when executed by the processor, implement operations to enhance the resolution of the geospatial image; and wherein the instructions receiving the geospatial image as a low resolution image; inputting the low-resolution image into a previously trained super-resolution model; generating super-resolution individual image tiles from the lower resolution images utilizing the trained super-resolution model; and selecting pixels from the low-resolution image that are adjacent to a plurality of super-resolution pixels inferred from the super-resolution model; identifying an original pixel from the lower resolution image having a minimum pixel value difference from each super-resolution pixel; maintaining original colors of the at least one lower resolution image in the image tile by selecting a reduced output area from the super-resolution image tile; removing any boundary effects introduced from said reduced output area selection; generating a super-resolution geospatial image having a higher resolution than the low-resolution image; Including, the system.

16. The plurality of pixels from the low resolution image are groups of nine pixels adjacent to the plurality of super-resolution pixels inferred from the low-resolution test image; and groups of four pixels adjacent to each individual super-resolution pixel inferred from the low-resolution test image. The system of claim 15 further comprising one of:

17. 16. The system of claim 15, wherein the super-resolution individual image tile has a crop size of 512x512 pixels, and the reduced output area of ​​the super-resolution individual image tile is a central 492x492 pixels of the super-resolution individual image tile.

18. The instruction: Finding corresponding original pixels for each super-resolution pixel to improve at least one of a peak signal-to-noise ratio (PSNR) and a structural similarity index (SSIM) of the super-resolution geospatial image. The system of claim 15 further comprising:

19. The instruction: modifying each super-resolution pixel value to match a color component of the original pixel corresponding to said super-resolution pixel by calculating the color component according to the ratio between an individual band pixel value and the sum of all band pixel values ​​in order to maintain said original color of said lower resolution image; The system of claim 15 further comprising:

20. The instruction: finding a minimum difference between the super-resolution pixel value and the modified super-resolution pixel value by successively searching various discrete values ​​centered around each super-resolution pixel; 20. The system of claim 19, further comprising:

Images