Remote sensing image splicing method based on Delaunay triangular feature constraint
Through the remote sensing image splicing method of Delaunay triangle feature constraints, feature points are extracted using SIFT and RANSAC algorithms, combined with the Delaunay triangulation and triangle similarity principles, the remote sensing image matching is optimized, and the problem of topographic influence in remote sensing image splicing is solved, and high-precision image splicing and chromatic difference reduction is achieved.
Patent Information
- Application Number
- CN202510501134.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-21
- Publication Date
- 2025-07-18
AI Technical Summary
During the splicing of remote sensing images, due to factors such as photography platforms, sensors, and terrain, the images are prone to geometric distortion and radiation distortion, resulting in layering of image splicing and low accuracy. The existing methods fail to effectively deal with the differences in image brightness and texture characteristics of the terrain and topography, resulting in poor splicing accuracy and chromatic aberration problems.
The remote sensing image splicing method based on Delaunay triangle feature constraints is adopted, feature points are extracted through the SIFT algorithm, feature points are screened using RANSAC, combined with the Delaunay triangulation and triangle similarity principles, image matching is optimized, and weighted fusion algorithm is used for splicing to reduce the impact of terrain features.
The accuracy of remote sensing image feature matching is improved, the image splicing results are optimized, the visual effect of terrain features on the image after splicing is reduced, and the splicing accuracy and consistency are improved.
Smart Images
Figure CN120339560A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of remote sensing image processing, and specifically to a remote sensing image stitching method based on Delaunay triangle feature constraints. Background Art
[0002] Remote sensing image stitching is a technology that combines two or more overlapping images into a large-scale, seamless panoramic image, which has wide application value in many fields such as virtual reality, aerospace, geographic information systems, art, and cultural heritage protection. It can make the image more conform to the actual geographical distribution, improve the data readability and application value, and provide complete basic data for environmental monitoring, urban planning, etc.
[0003] However, during the image stitching process, affected by factors such as the photography platform, sensor, and terrain, the image is prone to problems such as geometric distortion and radiation distortion, which may lead to image stitching stratification, low accuracy, or even stitching failure, directly affecting the subsequent adjustment solution accuracy and causing error accumulation and propagation.
[0004] The influence of terrain on image stitching and fusion is multi-faceted, and there are still some problems to be solved urgently in the existing remote sensing image stitching and fusion methods:
[0005] (1) Affected by terrain factors, the brightness difference and local shadows of remote sensing images are obvious. There are differences in texture features, shape features, and direction features of remote sensing images obtained at different times and angles in the same area, resulting in poor accuracy of image registration.
[0006] (2) In the processing of remote sensing image stitching, less consideration is given to the texture differences of images with different terrain features, and there are obvious color difference problems in the stitched and fused remote sensing images. Summary of the Invention
[0007] The purpose of the present invention is to provide a remote sensing image stitching method based on Delaunay triangle feature constraints to solve the disadvantages of the existing remote sensing image stitching methods in the background art.
[0008] To achieve the above purpose, the present invention provides the following technical solutions:
[0009] A remote sensing image stitching method based on Delaunay triangle feature constraints, comprising the following steps:
[0010] S1: Use the SIFT algorithm to construct an image pyramid, perform Gaussian difference and extreme value detection and positioning, extract feature points and descriptors of the remote sensing image. For each extracted feature point, use a texture feature descriptor to describe the feature, and use the RANSAC algorithm to screen the image feature points;
[0011] S2: Divide the area around each feature point selected in step S1 into rectangular regions of a fixed size according to the pixel density around the feature point, calculate the RGB values within each rectangular region, and count the frequencies of the RGB values. Then, perform L1 norm normalization on the RGB values;
[0012] Take the normalized frequency distributions of the color histograms of each pair of feature points as two vectors, calculate their similarity using the Euclidean distance similarity method, and select the pairs of feature points with similarity higher than the set threshold as the initial matching point pairs;
[0013] S3: Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance of the matching point pairs. If it is less than the set threshold, retain the feature point matching pair; otherwise, discard the feature point matching pair;
[0014] S4: Perform matching optimization based on the feature points in step S3, and then use the Delaunay triangulation algorithm to triangulate the feature points selected from the stitching image and the image to be stitched to obtain the triangular network of the stitching image;
[0015] S5: First calculate the triangular network similarity, and then calculate the distances between the vertices of the Delaunay triangles of the stitching image and the image to be stitched, and remove the vectors of the pseudo-matched triangles;
[0016] S6: Reconstruct the triangles based on step S5, divide the remote sensing image to be stitched into small blocks so that each small block contains a part of the terrain features;
[0017] S7: For each small block, calculate the texture coordinates of each triangle in it using the barycentric coordinate method, and then perform stitching using the weighted fusion algorithm. Perform smooth transition processing on the overlapping and non-overlapping regions of the stitching image and the image to be stitched to obtain the final result image.
[0018] Based on the above technical solutions, the present invention also provides the following alternative technical solutions:
[0019] In an alternative solution: in step S2, assume that the image feature point coordinates are (x, y), the scale is s, and the direction is θ. Represent the feature point using a feature vector containing this information (i.e., Eigenvector = [x, y, s, θ]). The feature vectors of all feature points in the stitching image are arranged in rows to form feature matrices V1 and V2, and calculate the Euclidean distance between the texture feature descriptors of V1 and V2 to measure the similarity of the feature points
[0020] where V1 = [x i1 , y i1 , s i1 , θ i1 ; x i2 , y i2 , si2 , θ i2 ; …; x i128 , y i128 , s i128 , θ i128 ;
[0021] V2 = [x j1 , y j1 , s j1 , θ j1 ; x j2 , y j2 , s j2 , θ j2 ; …; x j128 , y j128 , s j128 , θ j128 。
[0022] In an alternative solution: in step S3, calculate the Euclidean distance between the initial matching point pairs and the feature points with similar textures in the image to be stitched. The shortest Euclidean distance is the nearest neighbor distance, the second shortest is the second nearest neighbor distance, the feature point with the shortest Euclidean distance is the nearest neighbor point, and the feature point with the second shortest Euclidean distance is the second nearest neighbor point.
[0023] In an alternative solution: in step S5, the calculation method of the triangular mesh similarity is as follows: in triangular meshes A k and B l , each row vector represents a triangle, and its element value is the serial number of the SIFT feature point. Execute the formula Row intesct = intersection(A k , B l , 'row'), to obtain the common part of matrices A k and B l , that is, the feature point matching pairs in the overlapping area of the images, and calculate the triangular mesh similarity.
[0024] In an alternative solution: write the calculated triangular mesh similarity value into an M×N matrix, where M and N are the numbers of triangles in the two Delaunay triangular meshes respectively. Eliminate the triangle pairs with a similarity less than 0.75 to reduce the redundancy of the triangles and further simplify the triangular mesh.
[0025] In an alternative solution: in step S7, first use the barycentric coordinate method to calculate the texture coordinates of each triangle, then perform an affine transformation on the vertex coordinates of each triangle to obtain the target vertex coordinates. Then, according to the transformed coordinates, map the overlapping area of the image to be stitched onto the stitched image, and map the texture information of each small block to the corresponding position on the stitched image, and use the weighted fusion algorithm for stitching.
[0026] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0027] 1. The present invention uses the ratio of the nearest neighbor and the next-nearest neighbor distances and the Delaunay triangulation to constrain the SIFT feature matching, which fits the terrain and deletes highly similar and duplicate feature points, improving the accuracy of remote sensing image feature matching.
[0028] 2. Based on the Delaunay triangulation, the present invention combines the shape features of triangles and the image texture features, and uses the triangle similarity principle, the cosine theorem and the similarity features to obtain the optimal image projection transformation matrix, further optimizing the remote sensing image mosaicking result.
[0029] 3. The present invention abstracts the terrain into texture information and color histograms for local mosaicking, reducing the influence of terrain features on the visual effect of the mosaicked remote sensing image. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] Figure 1 is the principle block diagram of the present invention.
[0031] Figure 2 is the overall process working diagram of the present invention.
[0032] Figure 3 is the Delaunay triangulation diagram of the mosaicked remote sensing image of the present invention.
[0033] Figure 4 is the Delaunay triangulation diagram of the remote sensing image to be mosaicked of the present invention.
[0034] Figure 5 is the schematic diagram of the Delaunay triangulation network of the overlapping area of the remote sensing image of the present invention.
[0035] Figure 6 is the simplified principle diagram of the triangulation network of the present invention.
[0036] Figure 7 is the principle diagram of the triangulation network matching of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0037] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0038] In one embodiment, as Figures 1-7 shown, a method for mosaicking remote sensing images based on Delaunay triangle feature constraints includes the following steps:
[0039] S1. Feature point extraction and screening:
[0040] Construct an image pyramid using the classical SIFT algorithm, perform difference of Gaussians and extreme value detection and localization, and extract feature points and descriptors of remote sensing images.
[0041] L(x,y,σ) = G(x,y,σ) * I(x,y)
[0042]
[0043] D(x,y,σ) = L(x,y,kσ) - L(x,y,σ)
[0044] Among them, L(x,y,σ) is the Gaussian function of the change of the image scale space, G(x,y,σ) is the Gaussian kernel function, D(x,y,σ) = L(x,y,kσ) - L(x,y,σ) is the difference of Gaussians function, I(x,y) is the pixel value matrix of the original image, * represents convolution operation in the x and y directions, m and n are the dimensions of the Gaussian template, σ is the scale space factor, and k is the scale ratio coefficient.
[0045] For each extracted feature point, use the texture feature descriptor to describe the feature, and use the RANSAC (Random Sample Consensus) algorithm to screen the image feature points.
[0046] S2. Texture similarity constraint:
[0047] According to the pixel density around each screened feature point, divide a certain area around it into rectangular areas of a fixed size, calculate the RGB values within each rectangular area, and count the frequencies of the RGB values, and then perform L1 norm normalization on the RGB values; use the normalized frequency distributions of the color histograms of each pair of feature points as two vectors, and use the Euclidean distance similarity method to calculate their similarity, and screen out the feature point pairs with similarity higher than the set threshold as the initial matching point pairs.
[0048] Let the coordinates of the image feature point be (x,y), the scale be s, and the direction be θ. Use the eigenvector containing this information to represent the feature point, as shown in the following formula:
[0049] Eigenvector = [x,y,s,θ]
[0050] Through the above formula, form the feature matrix V1 by rows of the feature vectors of all feature points of the stitched image, and the feature matrix formed by the feature points of the image to be stitched is represented as V2. Calculate the Euclidean distance d(V1,V2) between the texture feature descriptors of V1 and V2 to measure the similarity of feature points, as shown in the following formula:
[0051] V1 = [x i1 ,y i1 ,s i1, θ i1 ; x i2 , y i2 , s i2 , θ i2 ; …; x i128 , y i128 , s i128 , θ i128
[0052] V2 = [x j1 , y j1 , s j1 , θ j1 ; x j2 , y j2 , s j2 , θ j2 ; …; x j128 , y j128 , s j128 , θ j128
[0053]
[0054] Meanwhile, the correct feature matching needs to meet the following conditions:
[0055]
[0056] Among them, max(S) represents S elements, and the sorted Euclidean distance sequence D is:
[0057]
[0058] Nearest neighbor distance constraint: For each initial matching point in the stitched image, calculate the Euclidean distance between it and the feature points with similar textures in the image to be stitched. Define the feature point with the smallest Euclidean distance as the nearest neighbor point, and the second nearest feature point as the second nearest neighbor point. Calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance. If it is less than the set threshold, retain the feature point matching pair; otherwise, discard the feature point matching pair.
[0059] S3. Delaunay triangulation:
[0060] After the feature point matching optimization is completed, use the Delaunay triangulation algorithm to triangulate the feature points of the stitched image and the image to be stitched to obtain the triangular mesh of the stitched image, as Figure 3 shown, and the triangular mesh of the image to be stitched is as Figure 4 shown.
[0061] The Delaunay triangulation meshes of the stitched image and the image to be stitched contain n and m triangles respectively, and their matrix representations are:
[0062]
[0063] In triangular network A k and B l each row vector represents a triangle, and its element value is the serial number of the SIFT feature point. Execute the following formula to obtain the common part Row k of A l and B intesct , that is, the feature point matching pairs in the image overlapping area.
[0064] Row intesct = intersection(A k , B l , 'row')
[0065] S4. Feature-constrained triangle matching:
[0066] Construct a Delaunay triangular network based on the feature point matching pairs in the overlapping area of the stitched image and the image to be stitched. As Figure 5 shown, since there are still a large number of collinear feature points, it is necessary to further optimize it.
[0067] Calculate the fuzzy similarity of the corresponding triangles in the two images. Let the three angles of triangle ΔABC in the stitched image be ∠A, ∠B, ∠C, and the angle corresponding to ∠A is marked as α. The ∠A' of triangle ΔA'B'C' in the image to be stitched is marked as α'. Then the similarity S α of triangle ∠A and ∠A' is calculated as follows:
[0068]
[0069] Where:
[0070]
[0071] Similarly, calculate the similarity values S b and S c of the other two angles of the triangle pair. Then the similarity ΔS of the triangle is:
[0072] ΔS = (S α + S b + S c ) / 3
[0073] Calculate the similarity of all pairs of triangles between the spliced image and the image to be spliced, and write the corresponding similarity values into an M×N matrix, where M and N are the numbers of triangles in the two Delaunay triangulations respectively. Remove the triangle pairs with similarity less than 0.75 to reduce the redundancy of triangles. On the basis of reducing triangle redundancy, further simplify the triangulation. Solve the interior angles of the triangles according to the cosine theorem. If it is greater than 160°, it is considered that the three vertices of the triangle are basically on the same horizontal line, as Figure 6 shown, then this triangle can be removed and the matching optimized, as Figure 7 shown.
[0074] S5. Removal of spurious triangle matches:
[0075] After the characteristic triangulations of the spliced image and the image to be matched are simplified, the problem of spurious triangle matches needs to be further processed. A spurious match refers to a situation in the matching process where a certain node is matched to two other nodes, forming a triangle structure, but in fact this match is not the optimal result. In the present invention, by calculating the distances between the vertices of the Delaunay triangles of the spliced image and the image to be spliced, the spurious triangle vectors are removed. First, traverse the already matched feature points of the spliced image and the image to be spliced. If the distance between the feature points exceeds the set threshold, then this point is marked as a false match; otherwise, it is added to the re-constructed point set, and the feature matching point vector is re-constructed.
[0076] S6. Image splicing process:
[0077] After removing the spurious triangles, re-construct the Delaunay triangulation. After re-construction, divide the remote sensing image to be spliced into small pieces, each small piece containing a part of the terrain features; for each small piece, use the barycentric coordinate method to calculate the texture coordinates of each triangle therein. Then, represent the vertices A, B, and C of the triangle ΔABC in the spliced image as column vectors of homogeneous coordinates [A x , A y , 1], [B x , B y , 1], [C x , C y , 1], and the target vertices of the triangle ΔA'B'C' to be spliced are A', B', and C', represented as [A′ x , A′ y , 1], [B′ x , B′ y , 1], [C′ x , C′ y , 1]. On this basis, construct two matrices P and P', where each row of P is the coordinate vector of the vertices A, B, and C of ΔABC, and each row of P' is the coordinate vector of the vertices A', B', and C' of ΔA'B'C', as shown in the following formula:
[0078]
[0079] Then, calculate the inverse matrix P_inv of matrix P:
[0080] det(P) = A x (B y -C y ) - B x (A y -C y ) + C x (A y -B y )
[0081]
[0082] P_inv = (1 / det(P)) * adj(P)
[0083] M = P' * P_inv
[0084] Among them, det(P) is the determinant of matrix P, adj(P) is the adjoint matrix of matrix P, and * represents matrix multiplication. Substitute the values of det(P) and adj(P) into the formula: P_inv = (1 / det(P)) * adj(P) to calculate P_inv, and solve the affine transformation matrix M through the formula: M = P ′ * P_inv; then perform an affine transformation on the vertex coordinates of each triangle to obtain the target vertex coordinates; then, according to the transformed coordinates, map the overlapping area of the image to be stitched onto the stitched image, and map the texture information of each small block to the corresponding position on the stitched image, and use the weighted fusion algorithm for stitching; finally, perform a smooth transition process on the overlapping area and non-overlapping area between the stitched image and the image to be stitched to obtain the final result image.
[0085] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims
1. A remote sensing image stitching method based on Delaunay triangle feature constraints, characterized in that, The following steps are involved: S1: Use SIFT algorithm to build image pyramid, perform Gaussian difference and extreme value detection and positioning, extract remote sensing image feature points and descriptors, use texture feature descriptors to describe each extracted feature point, and use RANSAC algorithm to screen image feature points; S2: According to the pixel density around each feature point screened out in step S1, the area around the feature point is divided into rectangular areas of fixed size, the RGB value in each rectangular area is calculated, the frequency of the RGB value is counted, and then the RGB value is normalized by L1 norm; The normalized frequency distribution of the color histogram of each pair of feature points is taken as two vectors, and the Euclidean distance similarity method is used to calculate their similarity, and the feature point pairs with similarity higher than the set threshold are selected as the initial matching point pairs; S3: Calculate the ratio of the nearest neighbor distance to the next nearest neighbor distance of the matching point pair. If it is less than the set threshold, the feature point matching pair is retained. Otherwise, the feature point matching pair is discarded. S4: performing matching optimization based on the feature points in step S3, and then using the Delaunay triangulation algorithm to triangulate the feature points selected from the stitched image and the image to be stitched, to obtain a triangulated network of the stitched image; S5: First calculate the triangulated network similarity, then calculate the distance between the Delaunay triangle vertices of the stitched image and the image to be stitched, and remove the false matching triangle vectors; S6: Reconstructing the triangles based on step S5, dividing the remote sensing image to be stitched into small blocks, so that each small block contains a part of the terrain features; S7: For each small block, the barycentric coordinate method is used to calculate the texture coordinates of each triangle therein, and then the weighted fusion algorithm is used for stitching. The overlapping area and non-overlapping area of the stitched image and the image to be stitched are smoothly transitioned to obtain the final result image.
2. The method for remotely sensed image mosaicking based on Delaunay triangle feature constraints according to claim 1, wherein, In the step S2, assuming the coordinates of the image feature point are (x, y), the scale is s, and the direction is θ, the feature vector containing this information (i.e., Eigenvector = [x, y, s, θ]) is used to represent the feature point. The feature vectors of all feature points of the spliced image are composed into feature matrices V1 and V2 by rows, and the Euclidean distance between the texture feature descriptors of V1 and V2 is calculated to measure the similarity of the feature points. where V1 = [x i1 , y i1 , s i1 , θ i1 ; x i2 , y i2 , s i2 , θ i2 ; …; x i128 , y i128 , s i128 , θ i128 ; V2 = [x j1 , y j1 , s j1 , θ j1 ; x j2 , y j2 , s j2 , θ j2 ; …; x j128 , y j128 , s j128 , θ j128 ; 3. A remote sensing image mosaicing method based on Delaunay triangle feature constraints according to claim 1, wherein In step S3, the Euclidean distance between the initial matching point pair and the feature point with similar texture in the image to be stitched is calculated, the shortest Euclidean distance is the nearest neighbor distance, the second shortest is the second nearest neighbor distance, the feature point with the shortest Euclidean distance is the nearest neighbor point, and the second shortest feature point is the second nearest neighbor point.
4. A method for remote sensing image mosaicing based on Delaunay triangle feature constraints according to claim 1, characterized in that In the step S5, the calculation method of the triangular mesh similarity is as follows: in the triangular meshes A k and B l , each row vector represents a triangle, and its element value is the serial number of the SIFT feature point. Execute the formula Row intesct =intersection(A k , B l , 'row') to obtain the common part of the matrices A k and B l , that is, the feature point matching pairs in the image overlapping area, and calculate the triangular mesh similarity.
5. A method for remotely sensed image mosaicking based on Delaunay triangle feature constraints according to claim 4, wherein The calculated triangulated network similarity values are written into an M×N matrix, where M and N are the number of triangles in the two Delaunay triangulated networks, respectively. Triangle pairs with similarity less than 0.75 are eliminated to reduce the redundancy of triangles and further simplify the triangulated network.
6. The method for remotely sensed image mosaicing based on Delaunay triangle feature constraint according to claim 1, wherein In step S7, the texture coordinates of each triangle are first calculated using the barycentric coordinate method, and then the vertex coordinates of each triangle are affine transformed to obtain the target vertex coordinates. Then, based on the transformed coordinates, the overlapping area of the images to be stitched is mapped to the stitched image, and the texture information of each small block is mapped to the corresponding position on the stitched image, and stitching is performed using a weighted fusion algorithm.