A hyperspectral image splicing method based on a new spectral SIFT feature

Abstract

The application discloses a hyperspectral image splicing method based on a new spectral SIFT feature, which comprises the following steps: extracting a single-band SIFT feature point of each hyperspectral image, and then extracting spectral dimension information of the feature point; adding the spectral dimension information into a SIFT feature point descriptor of the corresponding hyperspectral image respectively to obtain a new spectral SIFT feature descriptor; matching the new spectral SIFT feature points in two hyperspectral images according to the new spectral SIFT feature descriptor; purifying the obtained new spectral SIFT feature matching pairs to remove the mis-matched feature points, and then judging the image matching relationship; and further obtaining a final splicing image. The application can obtain a better splicing effect under the premise of ensuring the image accuracy, effectively improves the registration accuracy, and meets the actual industrial application requirements.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and specifically to a hyperspectral image stitching method based on novel spectral SIFT features. Background Technology

[0002] Hyperspectral images can simultaneously acquire image and spectral information of ground features, often containing dozens or even hundreds of consecutive spectral bands, thus containing a wealth of information and attracting significant attention in the field of remote sensing image technology. Image stitching is a technique that can fuse multiple narrow-view images with certain overlapping areas into a single wide-view image. When applying image stitching technology to hyperspectral images, the large data volume and redundant information of hyperspectral images can easily lead to mismatches of feature points, causing traditional image stitching algorithms to fall short of accuracy requirements; however, the high quality requirements for the resulting panoramic image make it essential to improve hyperspectral image stitching algorithms and optimize stitching effects. Image stitching technology mainly consists of four steps: image acquisition, image preprocessing, image registration, and image fusion. The most crucial step is image registration, where the transformation matrix parameters between matching images are calculated based on the positional information of the extracted SIFT (Scale-invariant feature transform) feature points, and then the geometric alignment of the images is achieved based on the transformation matrix. SIFT features exhibit good invariance to affine distortion, image noise, and variations in viewpoint and illumination, making them suitable for most image processing scenarios. However, due to the large number of feature points extracted from typical images, mismatches are prone to occur during matching. Therefore, the RANSAC (Random Sample Consensus) algorithm, which removes mismatched point pairs, is often used after feature matching. Nevertheless, due to the greater differences in height and angle during capture and the presence of more redundant information in hyperspectral images compared to typical images, a significant number of mismatched point pairs remain, resulting in poor registration performance in the stitched hyperspectral images.

[0003] Therefore, it is essential to design a hyperspectral image stitching method that improves feature point matching accuracy and optimizes stitching results. This invention proposes a hyperspectral image stitching method based on novel spectral SIFT features. Summary of the Invention

[0004] In order to overcome the above-mentioned shortcomings and deficiencies of the prior art, the purpose of this invention is to provide a hyperspectral image stitching method based on novel spectral SIFT features.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A hyperspectral image stitching method based on novel spectral SIFT features includes the following:

[0007] Read two hyperspectral images to be stitched together, and divide each image into multiple single-band images;

[0008] Extract SIFT feature points for each single band of each hyperspectral image, including SIFT feature point location information and SIFT feature point descriptors;

[0009] Extract the spectral dimension information of SIFT feature points;

[0010] The extracted spectral dimension information is added to the corresponding SIFT feature point descriptor to obtain a new spectral SIFT feature point descriptor, which forms a new spectral SIFT feature point.

[0011] Based on the novel spectral SIFT feature point descriptor, novel spectral SIFT feature points in two hyperspectral images are matched to obtain novel spectral SIFT feature matching pairs.

[0012] The novel spectral SIFT feature matching pairs are purified to obtain the feature in-place points between image pairs and the matching relationship between the image pairs is calculated.

[0013] Based on the matching relationship of the matching images and the purified feature in-points, calculate the transformation matrix between the two matching images;

[0014] Iteratively optimize the transformation matrix parameters of the image and calculate the final transformation matrix;

[0015] Based on the calculated transformation matrix of the image, the relative position of the image is obtained, and the final stitched image is obtained through the image fusion step.

[0016] Furthermore, the extraction of the spectral dimension information of SIFT feature points specifically involves:

[0017] For each image, based on the extracted single-band SIFT feature point locations, the gray values ​​at the corresponding SIFT feature point locations are extracted from all corresponding single-band images and combined sequentially into a column vector, i.e., the spectral dimension. Let the same feature point appear in each image... All corresponding single-band images The positions in the middle are all The spectral dimension of this feature point is calculated as follows:

[0018]

[0019] in, This represents the total number of bands in the original hyperspectral image.

[0020] Furthermore, the step of adding the extracted spectral dimension information to the corresponding SIFT feature point descriptor specifically involves:

[0021] For each SIFT feature point, let its original 128-dimensional descriptor be as follows:

[0022]

[0023] Embedding the spectral dimension into this 128-dimensional descriptor, a novel spectral SIFT feature descriptor that fuses spectral information is formed as follows:

[0024] .

[0025] in, This represents a novel spectral SIFT feature descriptor obtained after fusing spectral information; For each of the aforementioned SIFT feature points, the original 128-dimensional descriptor is... This is the spectral dimension descriptor for the SIFT feature point extracted above.

[0026] Furthermore, the step of matching novel spectral SIFT feature points in two hyperspectral images based on the novel spectral SIFT feature descriptor to obtain novel spectral SIFT feature matching pairs is as follows:

[0027] A coarse matching of feature descriptor vectors is achieved by using the ratio of nearest Euclidean distance to second nearest Euclidean distance.

[0028] Furthermore, the coarse matching specifically refers to:

[0029] Select a feature point in the reference image, then find the two feature points in the target image that are closest and second closest to that feature point in terms of Euclidean distance. Divide these two Euclidean distance values. If the result is less than a set threshold, then the pair of feature points with the closest Euclidean distance are considered to be matched.

[0030] Furthermore, the step of obtaining the feature in-place points between image pairs and calculating the matching relationship between the image pairs specifically involves:

[0031] Let the total number of SIFT feature matching pairs extracted be... The number of feature in-point pairs obtained after purification using the RANSAC algorithm is ,like If so, then the two images are matched.

[0032] Furthermore, the transformation matrix parameters of the images are iteratively optimized, and the final transformation matrix is ​​calculated. Specifically, the steps involve: [The text abruptly shifts to a different topic]... The error between them is defined as the sum of the distances between the new coordinate points of all feature points after being transformed by the homography transformation matrix and the original coordinate points. The calculation method is as follows:

[0033]

[0034] in, and Images and images The first in Each feature interior point Representing an image and images All feature interior points, Representing an image and images The homography transformation matrix between the images is used to calculate the cumulative error of all images, which is the sum of the errors between each image and its matching image. The calculation method is as follows:

[0035]

[0036] in, Indicates the number of images to be stitched together. Represents all images The matched images are then used for iterative optimization to calculate the parameters of the homography transformation matrix. The final transformation matrix is ​​represented as follows:

[0037] .

[0038] Furthermore, the final stitched image is obtained using the average value fusion method.

[0039] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hyperspectral image stitching method.

[0040] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the hyperspectral image stitching method.

[0041] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0042] This invention proposes a method utilizing spectral data information to augment feature point descriptors. By combining the location of feature points with spectral information, the SIFT feature point descriptors integrate spectral information with the original image information. When further applied to the feature point matching step, this makes it easier to match feature point pairs with similar spectral information, while eliminating mismatches between feature point pairs with dissimilar spectral information. This results in precisely matched feature point pairs, allowing for the calculation of optimal transformation matrix parameters and achieving better registration results. Compared to traditional SIFT feature points, the stitching effect using novel spectral SIFT features is significantly improved. Attached Figure Description

[0043] Figure 1 This is a flowchart of the process of this invention;

[0044] Figure 2(a) shows the original image to be stitched, Figure 2(b) shows the stitching effect using traditional SIFT features, and Figure 2(c) shows the stitching effect using the method described in this embodiment. Detailed Implementation

[0045] The present invention will be further described in detail below with reference to the embodiments, but the implementation of the present invention is not limited thereto.

[0046] This embodiment provides a hyperspectral image stitching method based on novel spectral SIFT features. It extracts a single-band SIFT feature point from each hyperspectral image, then extracts the spectral dimension information of the feature point, and adds this spectral dimension information to the SIFT feature point descriptor of the corresponding hyperspectral image. Based on the novel spectral SIFT feature descriptor, the novel spectral SIFT feature points in two hyperspectral images are matched. Feature point pairs with similar spectral information are more likely to match successfully, while mismatches of feature point pairs with dissimilar spectral information are eliminated, resulting in finely matched feature point pairs. Therefore, better transformation matrix parameters are calculated based on these finely matched point pairs, achieving better registration results. Compared to traditional SIFT features, the novel spectral SIFT features significantly improve the stitching effect.

[0047] like Figure 1 As shown, it includes the following steps:

[0048] S1 reads two original hyperspectral images to be stitched together and preprocesses them, dividing them into hundreds of single-band images;

[0049] S2 directly extracts a single-band SIFT feature point from each hyperspectral image and saves it;

[0050] S3 reads all bands of the two hyperspectral images to be stitched together and extracts the spectral dimension information of the feature points. The specific calculation steps are as follows: For each image, based on the extracted SIFT feature point positions of the single bands, extract the gray values ​​at the corresponding SIFT feature point positions in all corresponding single band images, and combine them sequentially into a column vector, i.e., the spectral dimension. Assume the same feature point appears in each image... All corresponding single-band images The positions in the middle are all The spectral dimension of this feature point is calculated as follows:

[0051]

[0052] in, This represents the total number of bands in the original hyperspectral image.

[0053] S4 adds the spectral dimension information to the SIFT feature point descriptors of the corresponding hyperspectral images. The specific steps are as follows: For each SIFT feature point, let its original 128-dimensional descriptor be as follows:

[0054]

[0055] Embedding the spectral dimension into this 128-dimensional descriptor, a new higher-dimensional spectral SIFT feature descriptor that incorporates spectral information is formed as follows:

[0056] .

[0057] in, This represents a novel spectral SIFT feature descriptor obtained after fusing spectral information; For each of the aforementioned SIFT feature points, the original 128-dimensional descriptor is... This is the spectral dimension descriptor for the SIFT feature point extracted above.

[0058] Furthermore, the feature points include feature point descriptors and feature point locations. The feature point matching process is implemented through the feature point descriptors, and the feature point locations are mainly used for subsequent purification and calculation of the transformation matrix between the two images.

[0059] S5 matches novel spectral SIFT feature points in two hyperspectral images based on the novel spectral SIFT feature descriptor. The specific steps are as follows: A coarse matching of feature descriptor vectors is achieved using the ratio of the nearest Euclidean distance to the second nearest Euclidean distance. A feature point in the reference image is selected, and then the two feature points in the target image that are closest and second nearest to that feature point in terms of Euclidean distance are found. These two Euclidean distance values ​​are then divided. If the result is less than a set threshold, the pair of feature points with the closest Euclidean distance is considered a match.

[0060] Let the feature point descriptor of the reference image be... Feature point descriptors of the target image It is expressed as follows:

[0061]

[0062]

[0063] The formula for calculating the Euclidean distance between the two is as follows:

[0064]

[0065] After obtaining the nearest and second nearest Euclidean distances using the above formula, the ratio of the two distances can be used to determine whether the pair of feature points match.

[0066]

[0067] in, This represents the threshold, the size of which determines the number of SIFT feature matches obtained.

[0068] S6 uses the RANSAC algorithm to refine the obtained novel spectral SIFT feature matching pairs, removing mismatched feature points and retaining correctly matched in-feature points for later use, thereby determining the image matching relationship. Assume the total number of directly extracted SIFT feature matching pairs is... The number of interior points obtained after purification using the RANSAC algorithm is: .like This allows us to determine if two images match;

[0069] S7 calculates the transformation matrix between the two matched images based on the matching relationship of the matched images and the purified feature in-points;

[0070] The S8 iterative optimization optimizes the transformation matrix parameters of the image and calculates the final transformation matrix. The specific steps are: [The text abruptly shifts to a different topic] ...two matching images... The error between them is defined as the sum of the distances between the new coordinate points of all feature points after being transformed by the homography transformation matrix and the original coordinate points. The calculation method is as follows:

[0071]

[0072] in, and Images and images The first in Each feature interior point Representing an image and images All feature interior points, Representing an image and images The homography transformation matrix between them. The cumulative error of all images is the sum of the errors between each image and its matching image, calculated as follows:

[0073]

[0074] in, Indicates the number of images to be stitched together. Represents all images The matched images are then used for iterative optimization to calculate the parameters of the homography transformation matrix. The final transformation matrix is ​​represented as follows:

[0075]

[0076] S9 obtains the relative position of the image based on the calculated image transformation matrix, and then uses the average value fusion method to obtain the final stitched image through the image fusion step.

[0077] Figure 2(a) shows the original image to be stitched, Figure 2(b) shows the stitching effect using traditional SIFT feature points, and Figure 2(c) shows the stitching effect using the method described in this embodiment. The traditional method of matching and calculating the transformation matrix using SIFT feature points has a large registration error, obvious ghosting and blurring phenomena, and poor stitching quality, which cannot meet the needs of actual industrial applications. In contrast, the method based on novel spectral SIFT features proposed in this embodiment achieves better stitching results and smaller image registration errors, indicating that this embodiment is more in line with the needs of actual applications compared to existing algorithms.

[0078] This embodiment also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the hyperspectral image stitching method.

[0079] A storage medium having a computer program stored thereon, which, when executed by a processor, implements the hyperspectral image stitching method.

[0080] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the embodiments described above. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A hyperspectral image stitching method based on novel spectral SIFT features, characterized in that, Including the following: Read two hyperspectral images to be stitched together, and divide each image into multiple single-band images; Extract SIFT feature points for each single band of each hyperspectral image, including SIFT feature point location information and SIFT feature point descriptors; Extracting the spectral dimension information of SIFT feature points, specifically: For each image, based on the extracted single-band SIFT feature point locations, the gray values ​​at the corresponding SIFT feature point locations are extracted from all corresponding single-band images and combined sequentially into a column vector, i.e., the spectral dimension. Let the same feature point appear in each image... All corresponding single-band images The positions in the middle are all The spectral dimension of this feature point is calculated as follows: in, This represents the total number of bands in the original hyperspectral image. The extracted spectral dimension information is added to the corresponding SIFT feature point descriptor, specifically: For each SIFT feature point, let its original 128-dimensional descriptor be as follows: Embedding the spectral dimension into this 128-dimensional descriptor, a novel spectral SIFT feature descriptor that fuses spectral information is formed as follows: ； in, This represents a novel spectral SIFT feature descriptor obtained after fusing spectral information; For each of the aforementioned SIFT feature points, the original 128-dimensional descriptor is... This is the spectral dimension descriptor for the SIFT feature point extracted above; Based on the novel spectral SIFT feature point descriptor, novel spectral SIFT feature points in two hyperspectral images are matched to obtain novel spectral SIFT feature matching pairs. The novel spectral SIFT feature matching pairs are purified to obtain the feature in-place points between image pairs and the matching relationship between the image pairs is calculated. Based on the matching relationship of the matching images and the purified feature in-points, calculate the transformation matrix between the two matching images; Iteratively optimize the transformation matrix parameters of the image and calculate the final transformation matrix; Based on the calculated transformation matrix of the image, the relative position of the image is obtained, and the final stitched image is obtained through the image fusion step.

2. The hyperspectral image stitching method according to claim 1, characterized in that, The process involves matching novel spectral SIFT feature points in two hyperspectral images based on the novel spectral SIFT feature descriptor to obtain novel spectral SIFT feature matching pairs. Specifically: A coarse matching of feature descriptor vectors is achieved by using the ratio of nearest Euclidean distance to second nearest Euclidean distance.

3. The hyperspectral image stitching method according to claim 2, characterized in that, The coarse matching specifically refers to: Select a feature point in the reference image, then find the two feature points in the target image that are closest and second closest to that feature point in terms of Euclidean distance. Divide these two Euclidean distance values. If the result is less than a set threshold, then the pair of feature points with the closest Euclidean distance are considered to be matched.

4. The hyperspectral image stitching method according to claim 1, characterized in that, The process of acquiring the feature in-place points between image pairs and calculating the matching relationship between the image pairs specifically involves: Let the total number of SIFT feature matching pairs extracted be... The number of feature in-point pairs obtained after purification using the RANSAC algorithm is ,like If so, then the two images are matched.

5. The hyperspectral image stitching method according to claim 1, characterized in that, The transformation matrix parameters of the image are iteratively optimized, and the final transformation matrix is ​​calculated. The specific steps are as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The error between them is defined as the sum of the distances between the new coordinate points of all feature points after being transformed by the homography transformation matrix and the original coordinate points. The calculation method is as follows: in, and Images and images The first in Each feature interior point Representing an image and images All feature interior points, Representing an image and images The homography transformation matrix between the images is used to calculate the cumulative error of all images, which is the sum of the errors between each image and its matching image. The calculation method is as follows: in, Indicates the number of images to be stitched together. Represents all images The matched images are then used for iterative optimization to calculate the parameters of the homography transformation matrix. The final transformation matrix is ​​represented as follows: 。 6. The hyperspectral image stitching method according to claim 1, characterized in that, The final stitched image is obtained using the average value fusion method.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the hyperspectral image stitching method according to any one of claims 1 to 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the hyperspectral image stitching method according to any one of claims 1 to 6.

Images