An optimal stitching line acquisition method for super-large file remote sensing images

Abstract

The application provides an optimal splicing line acquisition method for super large file remote sensing images, comprising the following steps: obtaining two super large file remote sensing images with geographical coordinate reference; performing equal resolution down-sampling processing on the overlapping area of the two super large file remote sensing images, and establishing a directed graph for the down-sampled pixel points; applying a graph cut algorithm to find the minimum cut of the directed graph, and obtaining a rough splicing line; generating a buffer zone based on the rough splicing line; obtaining the strip-shaped overlapping area of the two images in the original super large file remote sensing images by mapping the buffer zone; establishing a directed graph for the pixel points in the strip-shaped overlapping area; and applying the graph cut algorithm to obtain the optimal splicing line of the strip-shaped overlapping area. The application reduces the number of pixel points to be traversed in the process of determining the splicing line by performing down-sampling on the super large file remote sensing images, so that the number of nodes and edges to be calculated in the graph cut algorithm is met, and the global optimal solution is ensured, thereby obtaining the optimal splicing line of the super large file remote sensing images.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image stitching, and more particularly to a method for obtaining the optimal stitching line for ultra-large file remote sensing images. Background Technology

[0002] Image stitching is a technique that combines several images with overlapping areas (possibly acquired at different times and by different sensors) into a seamless panoramic or high-resolution image. In remote sensing image applications, to obtain a wider field of view and better unify the processing, analysis, research, and interpretation of remote sensing image information, it is necessary to stitch two or more remote sensing images with overlapping areas into a single image. Big data remote sensing images mainly refer to satellite remote sensing images; as the resolution of satellite remote sensing images increases, the amount of image data also gradually increases. Graph cut is a very useful and popular energy optimization algorithm, widely used in computer vision for image segmentation, stereo vision, and image matting. Due to the energy optimization characteristics of graph cut algorithms, they can be applied to find the optimal stitching line between images by assigning values ​​to image edges, thus minimizing the spectral difference between pixels on both sides of the stitching line. However, current methods of finding the optimal stitching line between two images by constructing an undirected graph and applying graph cut algorithms are limited by the maximum number of nodes and edges that graph cut algorithms can accommodate, greatly limiting their versatility for remote sensing images. For example, if the overlapping area of ​​two large-scale remote sensing images contains 5000*5000 pixels, while the maximum computational capacity of the graph cut algorithm is 1000*1000, then the graph cut algorithm may not be applicable. Furthermore, simply applying the graph cut algorithm directly can lead to many invalid points being included in the calculation, significantly increasing the computation time.

[0003] Traditional image stitching optimization algorithms are only suitable for small data images. When applied to ultra-large data remote sensing images, they often cannot be calculated due to the excessive amount of data, and cannot guarantee the stitching of remote sensing images with increasingly higher resolution and larger image data volume. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method for obtaining the optimal stitching line for ultra-large file remote sensing images. This method addresses the limitation of computationally intensive or time-consuming methods when searching for stitching lines between two ultra-large file remote sensing images due to the sheer volume of data. It employs a nested graph cut algorithm to find the globally optimal stitching line between the two images. It's important to note that the unique aspect of this invention lies in the nested stitching line search. Unlike typical stitching line searches that are performed in a single step or in blocks, this invention first downsamples the original image to obtain a smaller image. After finding a coarse stitching line, a buffer is created, and a finer stitching line is searched within this buffer. Finally, stitching and color-matching are performed based on these stitching lines to obtain a seamless, large-scale remote sensing image.

[0005] To achieve the above objectives, the specific technical solution of the present invention is as follows:

[0006] A method for obtaining the optimal stitching line in ultra-large remote sensing images, the method comprising the following steps:

[0007] Step 1: Acquire two ultra-large file remote sensing images with geographic coordinate reference, and perform equal-resolution downsampling on the overlapping area of ​​the two ultra-large file remote sensing images to obtain a rough stitching line.

[0008] Step one is specifically implemented through the following sub-steps:

[0009] (1.1) Calculate the geographic overlap area between two ultra-large file remote sensing images;

[0010] (1.2) Calculate the number of pixels in the geographic overlap region of the two ultra-large file remote sensing images and the resampling resolution of the image stitching;

[0011] (1.3) Perform equal-resolution downsampling on the overlapping area of ​​the two ultra-large file remote sensing images;

[0012] (1.4) Construct an undirected graph of the pixel points in the overlapping area of ​​the two ultra-large file remote sensing images after downsampling;

[0013] (1.5) Apply the graph cut algorithm to find the minimum cut of the undirected graph obtained in step (1.4) to obtain a coarse splicing line after resolution reduction;

[0014] Step 2: Generate a buffer zone based on the rough stitching line obtained in Step 1. This buffer zone forms a strip-shaped overlapping area in the original two ultra-large file remote sensing images. Within this strip-shaped overlapping area, obtain the globally optimal stitching line.

[0015] Step two is specifically implemented through the following sub-steps:

[0016] (2.1) Generate a buffer zone based on the rough splicing lines obtained in step one;

[0017] (2.2) The buffer generated in step (2.1) forms a strip-shaped overlapping area between two ultra-large file remote sensing images in the original ultra-large file remote sensing image;

[0018] (2.3) Construct an undirected graph for the pixel points in the strip-shaped overlapping region;

[0019] (2.4) Apply the graph cut algorithm to find the minimum cut of the undirected graph obtained in step (2.3) and obtain the optimal splicing line of the strip overlapping region, which is the global optimal splicing line.

[0020] Further, step (1.3) includes:

[0021] Select a sampling coefficient n, n>1, and apply bilinear interpolation to downsample the two ultra-large file remote sensing images A and B from step one to obtain two downsized images NA and NB that are reduced by the same proportion compared to A and B. These images are used to find the rough stitching line.

[0022] Further, step (1.4) includes:

[0023] The overlapping region is determined on the reduced images NA and NB. Based on the pixels within this overlapping region, an undirected graph G1(V, E) with weighted edges is constructed. The undirected graph G1(V, E) contains nodes V{C(x1, y1), C(x1, y2), ..., C(x1, y3)}. t ), ..., C(x) k y t )} and edges E{R(L), S(L), B(L)}; where C(x1, y1) represents the spectral value at position (x1, y1), C(x1, y2) represents the spectral value at position (x1, y2), and C(x1, y2) represents the spectral value at position (x1, y2). t ) indicates that in (x1, y t The spectral value at point C(x) k y t ) indicates that in (x k y t The spectral value at point (L) represents the edge between nodes in the same row; R(L) represents the edge between nodes in different rows; and B(L) represents the outermost edge of the frame.

[0024] The rules for assigning values ​​to nodes and edges in the undirected graph G1(V,E) are as follows:

[0025] Edges between the same row:

[0026] R(x, y) = |C NA (x1, y1)-C NB (x1, y1)|+|C NA(x2, y1)-C NB (x2, y1) | (1) Edges between different rows:

[0027] S(x, y) = |C NA (x1, y1)-C NB (x1, y1)|+|C NA (x1, y2)-C NB (x1, y2)| (2)

[0028] The outermost edge of the frame:

[0029]

[0030] In the formula, the value of the node is the spectral value of the corresponding pixel in the original ultra-large file remote sensing image: C NA (x1, y1) represents the spectral value of image NA at position (x1, y1), C NA (x2, y1) represents the spectral value of image NA at position (x2, y1), C NA (x1, y2) represents the spectral value of image NA at position (x1, y2), C NA (x p y q ) is the image NA at position (x p y q Spectral value at (); C NB (x1, y1) represents the spectral value of image NB at position (x1, y1), C NB (x2, y1) represents the spectral value of image NB at position (x2, y1), C NB (x1, y2) represents the spectral value of image NB at position (x1, y2), C NB (x p y q ) is the image NB at position (x p y q Spectral value at ().

[0031] Further, step (2.1) includes:

[0032] After obtaining the rough splicing line, a buffer radius of r is selected, and buffers are made on both sides along the normal direction of the rough splicing line. The buffers are then upsampled and enlarged to the original image size to obtain image P.

[0033] Further, step (2.2) includes:

[0034] The positions of the buffers in the image P are mapped one by one to the original images A and B. The strip-shaped overlapping area is obtained in images A and B, and the optimal stitching line is found in the strip-shaped overlapping area.

[0035] Further, step (2.3) includes:

[0036] An undirected graph G2(V, E) is constructed based on the pixels within the overlapping strip region, wherein the assignment rules for each node and edge are the same as those in step (2.4). The beneficial effects of this invention are:

[0037] (1) The present invention uses a nested graph cut algorithm to find the global optimal stitching line between two remote sensing images, avoiding the problem that traditional stitching line finding methods cannot be calculated or take too long due to the large amount of data. Compared with traditional stitching line finding methods, it greatly reduces the number of nodes in image calculation, thereby reducing the amount of calculation and improving the calculation efficiency of stitching line finding.

[0038] (2) This invention is applicable to the splicing line search of various sensors and has universal applicability to different sensors. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method for obtaining the optimal stitching line of ultra-large file remote sensing images according to the present invention;

[0040] Figure 2 This is a schematic diagram of the overlapping region of two images after downsampling in one embodiment of the present invention;

[0041] Figure 3 This is a schematic diagram of establishing multiple buffers based on the obtained coarse splicing lines in one embodiment of the present invention;

[0042] Figure 4(a) shows the position of the coarse splicing line obtained in the overlapping area within the buffer after upsampling in one embodiment of the present invention;

[0043] Figure 4(b) shows the position of the globally optimal splicing line obtained in the buffer zone within the strip overlap area in one embodiment of the present invention;

[0044] Figure 5(a) is a panoramic view of the result after splicing and color matching in one embodiment of the present invention;

[0045] Figure 5(b) is a magnified view of the splicing result in Figure 5(a). Detailed Implementation

[0046] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The objectives and effects of the present invention will become clearer as a result. The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0047] like Figure 1 The diagram shown is a flowchart of the optimal stitching line acquisition method for ultra-large file remote sensing images according to the present invention. The method includes the following steps:

[0048] Step 1: Acquire two ultra-large file remote sensing images (Landsat8) with geographic coordinate reference, perform equal-resolution downsampling on the overlapping area of ​​the two images, and obtain the first rough stitching line.

[0049] Step one is implemented through the following sub-steps:

[0050] 1.1. Calculate the geographic overlap area between two ultra-large file remote sensing images;

[0051] 1.2. Calculate the number of pixels in the geographic overlap region of the two images, and find that the number of pixels in the overlap region is 9,048,576; calculate the resampling resolution of the image stitching;

[0052] 1.3. Perform equal-resolution downsampling on the overlapping areas of the two images. The specific steps are as follows:

[0053] A sampling coefficient of 10, i.e., a 10*10 pixel block in the original image, is selected as one pixel in the downsized image. The original images A and B are downsampled using bilinear interpolation to obtain two downsized images NA and NB, which are reduced by the same proportion compared to the original images A and B. This reduces the number of pixels in the geographic overlap region of the downsized images to 62,697, and the overlap region is used as the range for finding a rough stitching line.

[0054] like Figure 2 The diagram shown illustrates the downsampling of the overlapping region between two images in this embodiment. The black area in the diagram represents the overlapping region, located within the downsampled image NA.

[0055] 1.4. Construct an undirected graph for the pixel points in the overlapping region of the two downsampled images. The specific steps are as follows:

[0056] The overlapping region is determined on the scaled-down image, and an undirected graph G1(V,E) with weighted edges is constructed based on the pixels within this overlapping region. This graph contains nodes V{C(x1,y1), C(x1,y2), ..., C(x1,y3)}. t ), ..., C(x) k y t )} and edges E{R(L), S(L), B(L)}; where C(x1, y1) represents the spectral value at position (x1, y1), C(x1, y2) represents the spectral value at position (x1, y2), and C(x1, y2) represents the spectral value at position (x1, y2).t ) indicates that in (x1, y t The spectral value at point C(x) k y t ) indicates that in (x k y t The spectral value at point () represents the edge between nodes in the same row; R(L) represents the edge between nodes in different rows; and B(L) represents the outermost edge of the graph frame, used for graph cut algorithms.

[0057] The rules for assigning values ​​to nodes and edges in the undirected graph G1(V,E) are as follows:

[0058] Edges between the same row:

[0059] R(x, y) = |C NA (x1, y1)-C NB (x1, y1)|+|C NA (x2, y1)-C NB (x2, y1)| (1)

[0060] Edges between different rows:

[0061] S(x, y) = |C NA (x1, y1)-C NB (x1, y1)|+|C NA (x1, y2)-C NB (x1, y2)| (2)

[0062] The outermost edge of the frame:

[0063]

[0064] In the formula, the value of the node is the spectral value of the corresponding pixel in the original image: C NA (x1, y1) corresponds to the spectral value of image NA at position (x1, y1), C NA (x2, y1) corresponds to the spectral value of image NA at position (x2, y1), C NA (x1, y2) corresponds to the spectral value of image NA at position (x1, y2), C NA (x p y q ) is the image NA at position (x p y q Spectral value at (); C NB (x1, y1) corresponds to the spectral value of image NB at position (x1, y1), C NB (x2, y1) corresponds to the spectral value of image NB at position (x2, y1), C NB(x1, y2) corresponds to the spectral value of image NB at position (x1, y2), C NB (x p y q ) is the image NB at position (x p y q Spectral value at ().

[0065] 1.5. Apply the graph cut algorithm to find the minimum cut in the undirected graph obtained in step 1.4, and obtain a coarse splicing line after resolution reduction. The specific steps are as follows:

[0066] In an undirected graph, all edges are assigned a non-negative weight W. e This is called the cost. Graph cut algorithms can break the edges in an undirected graph, forming two sets of unconnected edges. Such a set of edges is called a "cut". If a "cut" has the minimum sum of the costs of all its edges, then it is called a "minimum cut". In this embodiment, the graph cut algorithm is specifically a graph cut algorithm. The maximum flow / minimum cut algorithm in the graph cut algorithm can obtain the minimum cut in the undirected graph. This cut divides the image into two parts, and this dividing line is the energy-optimal dividing line for reducing the image NA and NB, which is the coarse splicing line.

[0067] Step 2: Generate a buffer based on the rough stitching line obtained in Step 1. This buffer forms a strip-shaped overlapping area in the original two images. Within this strip-shaped overlapping area, obtain the globally optimal stitching line.

[0068] Step two is implemented through the following sub-steps:

[0069] 2.1. Generate a buffer zone based on the rough splicing lines obtained in step two. The specific steps are as follows:

[0070] After obtaining the rough stitching line of the reduced image, select a buffer radius r of 17, and draw buffers on both sides along the normal direction of the stitching line to obtain a strip buffer in the reduced image NA. Assign the pixel value within the buffer area to 1, and upsample to the original image size to obtain image P.

[0071] like Figure 3 The diagram illustrates the buffer created in the scaled-down image based on the obtained coarse stitching lines. The buffer area is represented by the middle gray bar; the area in the upper left of the buffer is where image NA should be selected as the pixel value, and the area in the lower right of the buffer is where image NB should be selected as the pixel value.

[0072] 2.2. Obtain the banded overlap region between the two images formed by the buffer generated in step 2.1 within the original ultra-large file remote sensing image. The specific steps are as follows:

[0073] The positions of the buffers in image P are mapped one by one onto the original images A and B, obtaining the range of the strip buffers on images A and B. Considering that some buffer positions (at the beginning and end of the stitching line) in image p exceed the overlapping area of ​​the original images after upsampling, in this embodiment, after mapping image p to the original images, points at the edges that do not belong to the overlapping area of ​​the original images are searched and deleted, resulting in the strip overlapping area of ​​the two images. The optimal stitching line is then found within this strip overlapping area.

[0074] 2.3. Construct an undirected graph for the pixels within the overlapping strip region;

[0075] An undirected graph G2(V,E) is constructed based on the pixels in the strip-shaped overlapping area obtained in step 2.2, wherein the assignment rules for each node and edge are the same as those for each node and edge in step 2.4.

[0076] 2.4. Apply the graph cut algorithm to find the minimum cut of the undirected graph G2(V,E) obtained in step 2.3. Refer to step 2.5 for the specific steps. Obtain the optimal splicing line of the strip-shaped overlapping region, which is the global optimal splicing line.

[0077] Figure 4(a) shows the corresponding position of the coarse stitching line within the buffer; Figure 4(b) shows the corresponding position of the globally optimal stitching line obtained within the buffer within the strip-shaped overlapping area. It can be seen in the figures that the coarse stitching line obtained in the first search is coarser than the globally optimal stitching line obtained in the second search. In this embodiment, the first coarse stitching line is divided into 10*10 pixels on the original image, while the second globally optimal stitching line is divided into each pixel on the original image.

[0078] Based on the globally optimal stitching line obtained in step two, the two original big data remote sensing images are color-matched and stitched together to obtain a seamlessly stitched big data remote sensing image.

[0079] Figure 5(a) shows a grayscale panoramic image after stitching and color homogenizing two large-scale remote sensing images; Figure 5(b) is a schematic diagram of a magnified portion of the stitched result. As can be seen from the figures, there is no significant color difference at the stitching point between the two images, and the stitching point does not penetrate any prominent ground features.

[0080] In summary, based on the results of the two stitching line searches, the optimal stitching line acquisition method for ultra-large file remote sensing images of this invention can be used to find stitching lines in large-scale remote sensing images. Unlike traditional single-step graph cut algorithm searches, this invention significantly reduces the number of nodes in the graph cut algorithm by nesting graph cut algorithms. It should be noted that this invention is applicable to stitching line searches for various sensors and has universal applicability to different sensors.

[0081] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for obtaining the optimal stitching line in ultra-large file remote sensing images, characterized in that, The method includes the following steps: Step 1: Acquire two ultra-large file remote sensing images with geographic coordinate reference, and perform equal-resolution downsampling on the overlapping area of ​​the two ultra-large file remote sensing images to obtain a rough stitching line. Step one is specifically implemented through the following sub-steps: (1.1) Calculate the geographic overlap area between two ultra-large file remote sensing images; (1.2) Calculate the number of pixels in the geographic overlap region of the two ultra-large file remote sensing images and the resampling resolution of the image stitching; (1.3) Perform equal-resolution downsampling on the overlapping area of ​​the two ultra-large file remote sensing images; (1.4) Construct an undirected graph of the pixel points in the overlapping area of ​​the two ultra-large file remote sensing images after downsampling; (1.5) Apply the graph cut algorithm to find the minimum cut of the undirected graph obtained in step (1.4) to obtain a coarse splicing line after resolution reduction; Step 2: Generate a buffer zone based on the rough stitching line obtained in Step 1. This buffer zone forms a strip-shaped overlapping area in the original two ultra-large file remote sensing images. Within this strip-shaped overlapping area, obtain the globally optimal stitching line. Step two is specifically implemented through the following sub-steps: (2.1) Generate a buffer zone based on the rough splicing lines obtained in step one; (2.2) The buffer generated in step (2.1) forms a strip-shaped overlapping area between two ultra-large file remote sensing images in the original ultra-large file remote sensing image; (2.3) Construct an undirected graph for the pixel points within the overlapping strip region; (2.4) Apply the graph cut algorithm to find the minimum cut of the undirected graph obtained in step (2.3) and obtain the optimal splicing line of the strip overlapping region, which is the global optimal splicing line; Step (1.4) includes: The overlapping region is determined on the scaled-down images NA and NB, and an undirected graph with weighted edges is constructed based on the pixels within this overlapping region. The undirected graph Includes nodes and the edge ;in, Indicates the location Spectral values ​​at that location, Indicates in Spectral values ​​at that location Indicates in Spectral values ​​at that location, Indicates in Spectral values ​​at; This represents the edges between internal nodes in the same row. This represents the edges between different rows of internal nodes. This represents the outermost edge of the drawing frame; Undirected graph The rules for assigning values ​​to nodes and edges are as follows: Edges between the same row: (1) Edges between different rows: (2) The outermost edge of the frame: (3) In the formula, the value of the node is the spectral value of the corresponding pixel in the original ultra-large file remote sensing image: For image NA at location Spectral values ​​at that location, For image NA at location Spectral values ​​at that location, For image NA at location Spectral values ​​at that location, For image NA at location Spectral values ​​at; For image NB at position Spectral values ​​at that location, For image NB at position Spectral values ​​at that location, For image NB at position Spectral values ​​at that location, For image NB at position The spectral value at that location.

2. The method for obtaining the optimal stitching line for ultra-large file remote sensing images according to claim 1, characterized in that, Step (1.3) includes: Select a sampling coefficient n, n>1, and apply bilinear interpolation to downsample the two ultra-large file remote sensing images A and B from step one to obtain two downsized images NA and NB that are reduced by the same proportion compared to A and B. These images are used to find the rough stitching line.

3. The method for obtaining the optimal stitching line of ultra-large file remote sensing images according to claim 1, characterized in that, Step (2.1) includes: After obtaining the rough splicing line, a buffer radius of r is selected, and buffers are made on both sides along the normal direction of the rough splicing line. The buffers are then upsampled and enlarged to the original image size to obtain image P.

4. The method for obtaining the optimal stitching line for ultra-large file remote sensing images according to claim 3, characterized in that, Step (2.2) includes: The positions of the buffers in the image P are mapped one by one to the original images A and B, and the strip-shaped overlapping areas are obtained in images A and B. The optimal stitching line is then found within the strip-shaped overlapping areas.

5. The method for obtaining the optimal stitching line of ultra-large file remote sensing images according to claim 4, characterized in that, Step (2.3) includes: An undirected graph is constructed based on the pixels within the aforementioned overlapping band region. The assignment rules for each node and edge are the same as those for each node and edge in step (2.4).

Images