Unmanned aerial vehicle push-broom hyperspectral image splicing method and system

Abstract

This invention provides a method and system for stitching hyperspectral images using a pushbroom method from an unmanned aerial vehicle (UAV). The method includes: acquiring at least two adjacent hyperspectral images; performing multi-dimensional gradient extraction on the hyperspectral images to construct a second-order spatial spectral interaction matrix; generating a high-discrimination response map by weighted fusion of bi-branch feature responses based on the second-order spatial spectral interaction matrix; performing feature point matching on the hyperspectral images based on the high-discrimination response map to obtain initial matched feature point pairs; performing image segmentation and secondary matching based on the initial matched feature point pairs to obtain globally matched feature point pairs; performing outlier detection and correction on the globally matched feature point pairs to obtain corrected matched point pairs; and stitching and fusing the corrected matched point pairs to obtain a stitched hyperspectral image. This method can improve the geometric accuracy and visual effect of the stitched hyperspectral image, providing high-quality data support for subsequent applications of hyperspectral data.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method and system for stitching hyperspectral images using a drone pushbroom technique. Background Technology

[0002] Hyperspectral image registration and stitching are key technologies in remote sensing data processing, precision agriculture, and other fields. Pushbroom imaging acquires images by moving an unmanned aerial vehicle (UAV) platform along its flight path, requiring stitching techniques to generate complete scene images. However, during UAV flight, factors such as airflow disturbances and motor vibrations can cause non-rigid camera jitter, resulting in complex local geometric shifts between adjacent images. Existing technologies mostly employ spatial feature-based extraction algorithms, constructing descriptors through single spatial features such as grayscale gradients and local textures, and relying on global matching strategies to obtain corresponding point pairs.

[0003] Existing hyperspectral image stitching methods mostly rely on global feature matching. This results in large data volumes and high computational complexity for each processing step, making it difficult to adapt to the high-dimensional characteristics of hyperspectral images. Furthermore, feature point extraction fails to incorporate the rich spectral dimensional information of the hyperspectral image, leading to insufficient feature point extraction or low matching accuracy in regions with significant spectral features but weak spatial texture. Moreover, feature point matching is susceptible to interference from local offsets, resulting in poor quality stitched hyperspectral images. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for stitching hyperspectral images using a pushbroom method from an unmanned aerial vehicle (UAV). The specific technical solution adopted is as follows:

[0005] In a first aspect, embodiments of the present invention provide a method for stitching up hyperspectral images using a UAV pushbroom, the method comprising:

[0006] Acquire at least two adjacent hyperspectral images;

[0007] The hyperspectral image is subjected to dimensional gradient extraction to construct a spatial spectral interaction second-order matrix;

[0008] Based on the aforementioned second-order spatial spectral interaction matrix, a high-discrimination response map is generated by weighted fusion of bi-branch feature responses.

[0009] Based on the high-discrimination response map, feature point matching is performed on the hyperspectral image to obtain initial matching feature point pairs;

[0010] Based on the initial matching feature point pairs, image segmentation and secondary matching are performed to obtain global matching feature point pairs;

[0011] Anomaly detection and correction are performed on the global matching feature point pairs to obtain corrected matching point pairs;

[0012] The corrected matching point pairs are then stitched and fused to obtain a stitched hyperspectral image.

[0013] Secondly, a drone pushbroom hyperspectral image stitching system is provided, the system comprising:

[0014] The acquisition module is used to acquire at least two adjacent hyperspectral images;

[0015] The extraction module is used to perform multi-dimensional gradient extraction on the hyperspectral image to construct a spatial spectral interaction second-order matrix;

[0016] The fusion module is used to perform weighted fusion of bi-branch feature responses based on the second-order spatial spectral interaction matrix to generate a high-discrimination response map;

[0017] The first matching module is used to perform feature point matching on the hyperspectral image based on the high-discrimination response map to obtain an initial matching feature point pair;

[0018] The second matching module is used to perform image segmentation and secondary matching based on the initial matching feature point pairs to obtain global matching feature point pairs.

[0019] The correction module is used to detect and correct outliers in the global matching feature point pairs to obtain corrected matching point pairs.

[0020] The stitching module is used to stitch and fuse the corrected matching point pairs to obtain a stitched hyperspectral image.

[0021] Thirdly, a computer program product is provided, comprising: computer program code, which, when run on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0022] Fourthly, a computer-readable storage medium is provided that stores computer program code, which, when executed on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0023] This invention offers the following advantages: By acquiring at least two adjacent hyperspectral images and performing dimensional gradient extraction on these images, a spatial spectral interaction second-order matrix is ​​constructed. This effectively reduces mismatches caused by illumination variations and spatial noise during feature point extraction and matching, thereby improving the accuracy of feature point pairs. Subsequently, based on the spatial spectral interaction second-order matrix, a weighted fusion of bi-branch feature responses is performed to generate a high-discrimination response map. Based on this high-discrimination response map, feature point matching is performed on the hyperspectral image to obtain initial matched feature point pairs. Then, image segmentation and secondary matching are performed on the initial matched feature point pairs to obtain global matched feature point pairs. This secondary matching of the initial matched feature point pairs improves the accuracy of the resulting global matched feature point pairs. Finally, outlier detection and correction are performed on the global matched feature point pairs to obtain corrected matched point pairs. These corrected matched point pairs are then image-stitched and fused to obtain a stitched hyperspectral image. In this way, by detecting and correcting outliers in globally matched feature point pairs, abnormal matching points can be accurately removed, outputting corrected feature point pairs that possess uniqueness, orderliness, and spatial uniformity. This significantly improves the purity of the feature point set and reduces the interference of mismatches on the stitching process. By stitching and fusing the corrected matching point pairs into images, the geometric accuracy and visual effect of the stitched hyperspectral image can be improved, providing high-quality data support for subsequent applications of hyperspectral data. Attached Figure Description

[0024] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram illustrating the implementation process of a drone pushbroom hyperspectral image stitching method provided in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of another implementation process of a drone pushbroom hyperspectral image stitching method provided in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram illustrating another implementation process of a drone pushbroom hyperspectral image stitching method provided in this embodiment of the invention;

[0028] Figure 4 This is a schematic diagram illustrating the implementation framework of a drone pushbroom hyperspectral image stitching method provided in an embodiment of the present invention;

[0029] Figure 5 This is a schematic diagram of the composition structure of a drone pushbroom hyperspectral image stitching system provided in an embodiment of the present invention;

[0030] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0031] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a UAV pushbroom hyperspectral image stitching method proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments may be combined by any suitable form.

[0032] In the description of the embodiments of the present invention, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" means two or more.

[0033] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] This invention addresses image stitching quality issues caused by camera shake in UAV pushbroom hyperspectral imaging (such as low feature matching accuracy, anomaly interference, and discontinuous stitching boundary transitions). It constructs a comprehensive technical framework from feature extraction to image fusion, based on the principle of "high-precision feature matching - clean feature constraints - seamless stitching fusion." First, spectral feature enhancement improves feature discriminative power, combined with a block-based recursive refinement mechanism to achieve high-precision matching. Second, feature point sets are cleaned based on local spatial continuity to eliminate anomaly interference and improve the consistency of the offset sequence. Finally, guided by corrected feature points, the fusion region is dynamically divided and smooth transition weights are constructed to achieve seamless stitching of hyperspectral images. Thus, through a hierarchical progression of "feature enhancement - matching optimization - anomaly cleanup - segmentation constraints - adaptive fusion," key problems in traditional methods such as feature mismatch, poor global consistency, and discontinuous stitching boundaries are solved, providing a high-quality image foundation for subsequent applications of hyperspectral data.

[0036] The specific solution of the UAV pushbroom hyperspectral image stitching system provided by the present invention is described below with reference to the accompanying drawings. Please refer to the accompanying drawings. Figure 1 The diagram illustrates a flowchart of an embodiment of the UAV pushbroom hyperspectral image stitching method provided by the present invention, which includes:

[0037] 101. Acquire at least two adjacent hyperspectral images.

[0038] Here, at least two adjacent hyperspectral images are acquired, or at least two adjacent hyperspectral images are received from a third-party device.

[0039] 102. Perform dimensional gradient extraction on the hyperspectral image to construct a spatial spectral interaction second-order matrix.

[0040] Here, feature point detection employs a multi-dimensional gradient extraction strategy to construct a second-order matrix of spatial-spectral interaction. Addressing the issue of insufficient feature representation capabilities in hyperspectral image feature point detection methods that rely solely on two-dimensional spatial gradients or process spatial / spectral information alone, this application proposes a feature point detection method that integrates spatial and spectral gradient information. By extracting spatial and spectral gradient features multi-dimensionally, a second-order matrix containing spatial and spectral interaction information is constructed. Based on a bi-branch feature response calculation and weighted fusion strategy, a collaborative representation of spatial structure and spectral characteristics is achieved, thereby improving the expressive power of the feature response and detection robustness.

[0041] In some possible implementations, step 102 above can be achieved through the following steps 121 to 123 (not shown in the figures):

[0042] 121. The spatial dimension gradient and spectral gradient of the hyperspectral image are extracted to obtain the spatial gradient and spectral gradient.

[0043] Here, for the three-dimensional data cube structure of hyperspectral images, a dimensionality-splitting strategy is adopted to extract gradient features in the spatial and spectral domains. Spatial gradient calculation: The Gaussian-Sobel joint algorithm is executed independently for each band: 1. A Gaussian filter (kernel parameter range is [0.5, 2.0]) is used to smooth and denoise the original image. 2. The spatial gradient components are obtained by convolution along the horizontal (X-axis) and vertical (Y-axis) directions using the Sobel operator, and the spatial gradient magnitude is obtained through vector synthesis. Spectral gradient calculation: A difference-smoothing joint algorithm is used: 1. The difference matrix between adjacent bands is calculated along the band dimension to construct the initial spectral gradient field; 2. Mirror continuation processing is applied to the spectral boundaries (the gradient value of the last band is set to the gradient value of the second-to-last band). 3. A one-dimensional Gaussian smoothing filter is applied to the spectral gradient sequence of each pixel (smoothing parameter range is [0.5, 1.5]). Finally, a three-dimensional feature matrix (i.e., spatial gradient and spectral gradient) containing the horizontal spatial gradient, vertical spatial gradient, and spectral gradient is obtained.

[0044] 122. Local gradient field extraction is performed on the spatial gradient and spectral gradient to obtain a local gradient field that includes spatial and spectral local structure information.

[0045] Here, for the obtained three-dimensional gradient field (including the spatial gradient in the X / Y directions and the three channels of the spectral gradient), a local neighborhood is extracted for each pixel. The range of the local neighborhood is determined by the window radius, and the window size is usually set to 3×3 to include the spatial and spectral local structure information around the pixel.

[0046] 123. Based on the local gradient field, construct the second-order spatial spectral interaction matrix.

[0047] Here, a weighted operation is performed on the local gradient field to calculate six second-order moment elements: the sum of squared spatial gradients (G x ²+G ²), spectral gradient square (G s ²), gradient product cross term (G x G G x G s G G s Construct a 3×3 spatial spectral interaction second-order matrix M, which has the following form:

[0048] ;

[0049] 103. Based on the aforementioned spatial spectral interaction second-order matrix, a weighted fusion of bi-branch feature responses is performed to generate a high-discrimination response map.

[0050] Here, after constructing a second-order spatial spectral interaction matrix, a high-discrimination response map is generated by weighted fusion of the bi-branch feature responses.

[0051] In some possible implementations, firstly, spatial eigenvalues ​​are calculated and noise is suppressed on the second-order spatial spectral interaction matrix to obtain a spatial feature response map; then, spectral gradient energy is extracted from the second-order spatial spectral interaction matrix; finally, the spatial feature response map and the spectral gradient energy are weighted and fused to generate the high-discrimination response map.

[0052] Here, spatial and spectral branch feature responses are extracted based on a second-order matrix. Spatial feature response calculation: Based on the Harris corner detection principle, spatial feature values ​​are calculated using the determinant (det) and trace (tr) of the second-order matrix submatrices, employing empirical formulas (such as...) (where k is an empirical constant) to suppress noise interference, generate a spatial characteristic response map, and perform non-negative processing. The spectral characteristic response calculation process includes: extracting the spectral gradient energy as the spectral response value, and performing a weighted summation of the squares of the local spectral gradients (where k is an empirical constant). , Spectral feature values ​​are obtained by using weighting coefficients, and then non-negative processing is performed to generate spectral feature response maps.

[0053] Spatial response map ( ) and spectral response diagram ( The normalization process includes: performing maximum value normalization on the bi-branch response graphs respectively. , This eliminates dimensional differences. Weighted fusion: A linear weighting strategy is used to fuse normalized response values. The contribution ratios of spatial and spectral features are controlled by weighting coefficients α and β (α+β=1) to generate the final feature response map. .

[0054] 104. Based on the high-discrimination response map, feature point matching is performed on the hyperspectral image to obtain initial matching feature point pairs.

[0055] Here, by extracting spatial and spectral features from the high-discrimination response map, block-level spatial features and spectral segmentation features are obtained. Then, feature descriptors are generated based on these block-level spatial features and spectral segmentation features. Feature point matching is performed using the feature descriptors to obtain initial matching feature pairs.

[0056] In some possible implementations, a dual-branch feature extraction framework is constructed using key technologies such as joint 3D gradient modeling, HOG3D extension, dynamic spectral segmentation, and two-stage normalization: First, block histogram statistics based on the 3D gradient field are used to capture local spatial structure characteristics; second, dynamic spectral segmentation encoding is used to enhance the continuity of spectral dimensions; finally, robust feature point descriptors are generated through weighted fusion and global normalization. That is, step 104 above can be achieved through... Figure 2 The steps shown are to be implemented as follows:

[0057] 201. The high-discrimination response map is divided into three-dimensional cells and blocks to obtain feature cells.

[0058] Here, based on the obtained three-dimensional gradient field, the local spatial spectral gradient distribution features are extracted through block histogram statistics. First, three-dimensional cells and blocks are divided to obtain feature cells, including: constructing a three-dimensional local region centered on the feature point, setting the cell size to 4×4 pixels (spatial dimension), and the block size to 2×2 cells (i.e., 8×8 pixel spatial range). The spectral dimension and spatial dimension use the same cell division method.

[0059] 202. Based on the feature cells, generate a three-dimensional gradient histogram.

[0060] Here, each cell is traversed, and the spatial x-gradient, spatial y-gradient, and spectral gradient are quantized into 8 directions respectively. The distribution of gradient magnitudes in the three-dimensional directions is statistically analyzed to form an 8×8×8=512-dimensional three-dimensional gradient histogram.

[0061] 203. Fill the cells in the three-dimensional gradient histogram that exceed the image boundary with zero vectors and concatenate the histograms to obtain block-level spatial features.

[0062] Here, boundary overflow handling and block-level feature generation are performed to obtain block-level spatial features, including: filling cells that exceed the image boundary with zero vectors to ensure consistent histogram dimensions; performing local normalization on the histograms of all cells within each block (to eliminate scale differences in sub-features), and then stitching the normalized histograms together to form block-level HOG3D features (i.e., block-level spatial features, with a single block feature length of 512 × the number of cells within the block).

[0063] 204. Perform dynamic spectral segmentation encoding on the three-dimensional gradient histogram to obtain spectral segmentation features.

[0064] Here, to enhance the continuity of the spectral dimension, when segmenting and encoding the spectral information of the feature point's neighborhood, all pixels within a 5×5 spatial neighborhood around the feature point are first extracted, and the spectral mean vector of this neighborhood (dimension equal to the number of bands B) is calculated. Then, the spectral mean vector is adaptively divided into 10 segments according to the number of bands B. If B is not divisible by 10, the segment length is dynamically adjusted to achieve uniform segmentation. Finally, the mean of each spectral mean vector is calculated to generate a 10-dimensional spectral segmentation feature, and the 10-dimensional vector is locally L2 normalized to eliminate spectral intensity differences.

[0065] 205. Based on the spectral segmentation features and the block-level spatial features, generate feature point descriptors with fixed dimensions.

[0066] In some possible implementations, a joint feature vector is generated using the spectral segmentation features and the block-level spatial features; and the joint feature vector is then globally normalized to generate a feature point descriptor with fixed dimensions.

[0067] Here, a weighted fusion strategy is adopted to integrate HOG3D features and spectral features. The weight of HOG3D features is set to 0.5 and the weight of spectral segmentation features is set to 0.5. The two types of features are weighted and summed and then concatenated by channel to form a joint feature vector. Subsequently, the joint feature vector is subjected to global L2 normalization. To avoid the division by zero problem, a minimum value (such as 1e(-10)) is added before normalization. Finally, a feature point descriptor with fixed dimension is generated.

[0068] 206. Perform feature point matching on the feature point descriptor to obtain the initial matched feature point pair.

[0069] Here, by performing dimensionality reduction on the feature point descriptors and calculating the matching distance matrix, the resulting matching distance matrix is ​​filtered and bidirectional consistency is verified to obtain initial matching feature point pairs. In some possible implementations, step 206 above can be achieved through... Figure 3 The steps shown are to be implemented as follows:

[0070] 301. Perform dimensionality reduction processing on the feature point descriptor to obtain a dimensionality-reduced descriptor.

[0071] 302. The matching distance matrix of the reduced-dimensional descriptor is calculated using a broadcast mechanism.

[0072] 303. Perform positive matching filtering on the feature points corresponding to the rows in the matching distance matrix to obtain positive matching results.

[0073] 304. A reverse verification mechanism is used to perform bidirectional consistency verification on the matching of feature points corresponding to columns in the matching distance matrix to feature points corresponding to rows, so as to obtain initial matching feature point pairs with bidirectional consistency.

[0074] In some possible implementations, the corresponding column vectors are extracted by traversing the feature points corresponding to the columns; then the column vectors are sorted to obtain the sorting result; and the sorting result of the column vectors is combined with the nearest neighbor index of the feature points corresponding to the rows to obtain the combination result; finally, the combination result is compared with the positive matching result to obtain the initial matching feature point pair that is consistent in both directions.

[0075] In steps 301 to 304 above, joint principal component analysis (PCA) is performed on the two input feature descriptor sets for dimensionality reduction. Specifically, the two descriptor sets are merged as training data and normalized. A PCA model is constructed to reduce the dimensionality to the target dimension of 128. The trained PCA model is then applied to the two descriptor sets to achieve feature compression. Subsequently, the matching distance matrix of the dimensionality-reduced features is calculated through a broadcast mechanism. That is, the Euclidean distance between all feature pairs of the two descriptor sets is calculated and stored as a two-dimensional array, where rows correspond to img1 feature points (i.e., the feature points corresponding to the rows), and columns correspond to img2 feature points (i.e., the feature points corresponding to the columns). Then, a forward matching filter is performed on the img1 feature points, iterating through... For each img1 feature point, the corresponding row vector is extracted from the distance matrix and sorted in ascending order of distance to obtain the top two matching candidates. The Lowe ratio test is used to filter reliable matching pairs. Finally, the bidirectional consistency verification of the matching from img2 to img1 is performed through a reverse verification mechanism. For each img2 feature point, the corresponding column vector is extracted and sorted. The img2 feature point index is combined with the img1 nearest neighbor index and compared with the forward matching results. Only the results with consistent bidirectional matching indices are retained as the final matching pairs. The feature redundancy is reduced by joint PCA dimensionality reduction, and the combination of Lowe ratio test and bidirectional verification strategy effectively suppresses mismatches.

[0076] 105. Based on the initial matching feature point pairs, perform image segmentation and secondary matching to obtain global matching feature point pairs.

[0077] Here, feature point correction aims to improve the matching accuracy and spatial consistency of hyperspectral image feature point pairs through multi-level matching optimization, outlier removal, and sequence segmentation correction, providing high-quality feature constraints for subsequent registration or stitching. First, a block-based recursive refinement strategy is adopted, reducing computational complexity through vertical block division and achieving accurate local alignment through a multi-level "coarse matching-fine matching" mechanism. A global matching point set is then constructed after global coordinate adjustment and deduplication sorting. Second, outliers are iteratively removed based on the offset differences of neighboring feature points, purifying the matching point set. Finally, adaptive segmentation processing based on accumulated deviation smooths out abrupt sequence changes and optimizes segment boundaries, improving the local consistency of the feature point sequence. The three stages work synergistically to effectively balance local accuracy and global consistency, eliminate redundancy and outlier interference, and output high-quality feature point pairs that are unique, ordered, and spatially uniform.

[0078] In some possible implementations, firstly, based on the initial matching feature point pairs, the hyperspectral image is divided into image blocks to obtain multiple image blocks; secondly, coarse matching and vertical offset estimation are performed on any two image blocks to obtain the relative misalignment of the two image blocks in the vertical direction, and based on the relative misalignment of the two image blocks in the vertical direction, the two image blocks are vertically aligned until a coarsely aligned image block that meets a preset granularity is obtained; thirdly, the coarsely aligned image block is recursively divided to obtain the recursive division result, and the matching point coordinates of the recursive division result are globally calibrated to construct a candidate matching point set for the entire image range; finally, the candidate matching point set is deduplicated and sorted to obtain the global matching feature point pairs.

[0079] Here, the amount of data processed in a single step is reduced by dividing the image into blocks vertically. A multi-level matching mechanism of "coarse matching-fine matching" is adopted: First, coarse matching is performed on the image blocks to estimate the vertical offset and achieve preliminary alignment. Then, fine-grained matching is performed through recursive block division to improve local accuracy. Finally, after global coordinate adjustment, matching point merging, and deduplication sorting, high-quality globally matched feature point pairs with both local accuracy and global consistency are output. The specific steps are as follows:

[0080] The first step is to perform image segmentation processing;

[0081] Here, two hyperspectral images (data1, data2) and their corresponding grayscale images (img1, img2) are input for matching. The images are then divided into several sub-blocks vertically according to a preset block height (block_height). By segmenting the image into blocks, a large image is transformed into multiple local sub-blocks, effectively reducing the computational complexity and memory usage of a single matching operation, and laying the foundation for subsequent block-by-block fine-grained processing.

[0082] The second step is to perform coarse matching and vertical offset estimation;

[0083] For each segmented image block, the feature point extraction method proposed in 4.2.1 above is used to obtain an initial matching point pair (key1, key2), where key1 and key2 are the feature point sets of the two image blocks, respectively. The vertical offset of the matching point pair is then calculated. , is defined as the mean of the differences in the y-coordinates of all matching points (i.e. (where N is the number of matching point pairs), thereby estimating the relative misalignment of the two image patches in the vertical direction.

[0084] The third step is to vertically align the image blocks;

[0085] Based on the vertical offset estimated in step 2 A vertical translation affine transformation is performed on the second image patch and its corresponding hyperspectral data to achieve coarse alignment between the two image patches. To avoid image content truncation after translation, the size of the transformed image is dynamically adjusted to fully accommodate all pixel information after translation, ensuring the effectiveness of subsequent fine matching.

[0086] The fourth step is to perform fine-grained matching recursively.

[0087] For the coarsely aligned image blocks, a recursive block division strategy is adopted to further improve the matching accuracy: the block division process in step 1 is performed, with the block height being 1 / n of the initial block height. For fine-grained block division, the value of n is generally set to 2. The coarse matching and vertical alignment process in steps 2-3 is repeated to form a multi-level matching mechanism of "coarse matching-fine matching". Through recursive refinement, local region features are gradually focused, reducing the impact of the initial offset estimation error on the matching accuracy.

[0088] The fifth step is to perform global coordinate adjustment and merge matching points;

[0089] During the recursive process, the coordinates of matching points at each level are based on the local block coordinate system, and need to be adjusted to the original image global coordinate system by accumulating the vertical translation offset. Specifically, the vertical translations of each recursive layer are integrated to perform global calibration on the coordinates of all matching points; then, the matching point pairs of all image blocks are merged to construct an initial matching point set covering the entire image.

[0090] The sixth step is to remove duplicates and sort the matching points.

[0091] The merged global matching point set is optimized as follows: First, the matching point pairs are sorted in ascending order of the y-coordinates of the feature points in img1; second, the sorted point pairs are traversed, and a y-coordinate threshold is set. Only retain points whose y-coordinate difference from the previous retained point is greater than 1. The point pairs are used to eliminate redundant matching of dense feature points in the vertical direction; finally, the matching point pairs that maintain the original correspondence after deduplication are output to ensure the uniqueness, order and spatial uniformity of the matching points, and provide high-quality feature constraints for subsequent hyperspectral image registration or stitching.

[0092] 106. Perform outlier detection and correction on the global matching feature point pairs to obtain corrected matching point pairs.

[0093] In some possible implementations, firstly, outlier detection is performed on the globally matched feature point pairs to obtain detected feature point pairs; secondly, the cumulative deviation detection mutation features of the detected feature point pairs are determined; thirdly, based on the cumulative deviation detection mutation features, initial segments are generated for the detected feature point pairs; fourthly, numerical points in the detected feature point pairs that are not included in the segments are assigned to the current segments; fifthly, based on the mean of the current segments, a segment boundary list and a segment mean list are generated, and the segment boundary list is merged to obtain a merged segment boundary list; finally, the order of the corrected value vectors corresponding to the merged segment boundary list and the segment mean list is mapped back to the order of the original data to obtain the corrected matching point pairs.

[0094] Here, the image matching outlier removal algorithm based on the offset difference of neighboring feature points aims to purify feature point pairs and improve matching quality by focusing on the local spatial continuity of the matching point sequence. Its core innovation lies in abandoning the traditional method that relies on global statistics or single-point distance thresholds, and instead using the offset difference of adjacent matching points as the basis for anomaly judgment, and combining iterative removal mechanism to avoid over-removal of valid points. The specific steps are as follows: First, input the coordinate sets of the two sets of matching feature points to be processed (key1s and key2s, both of which are N×2 dimensional NumPy arrays), and initialize the horizontal / vertical offset anomaly thresholds (dx_threshold, dy_threshold) and the maximum number of iterations (max_iter); then, in the maximum number of max_iter iterations, first calculate the horizontal offset dx and vertical offset dy of the current matching point set; next, for each matching point, calculate the difference between its dx and dy values ​​with its neighboring points before and after it. If any difference exceeds the corresponding threshold, the point is marked as an anomaly; finally, if there are no anomaly points in this round, the iteration is terminated early; otherwise, all anomaly points are removed and the matching point set is updated before entering the next round of iteration, and finally the purified matching feature point pairs are output.

[0095] Here, an adaptive segmentation method based on cumulative bias is used to achieve adaptive segmentation processing. After removing outliers, feature point pairs are segmented and corrected. Abrupt features in the sequence are detected by cumulative bias, and initial segments are generated by combining minimum segment length constraints. A mean substitution strategy is used to smooth abrupt change regions, and overly dense segments are eliminated through boundary merging. Finally, a corrected sequence that maintains the original data order is output to improve the local consistency of the sequence. This method is suitable for scenarios such as image displacement correction where dense abrupt changes need to be eliminated. The adaptive segmentation process includes:

[0096] First, data preprocessing is performed;

[0097] The input consists of a sequence of numerical values ​​to be processed (such as the horizontal offset dx and vertical offset dy of the current matching point set) and their corresponding position coordinates. First, the data is rearranged in ascending order of position coordinates to ensure the continuity of the sequence in the spatial dimension. The following data structures are initialized: a list of segment boundaries (to record the start and end positions of each segment), a list of segment means (to store the mean of each segment), and a correction value vector (of the same length as the input sequence, used to store the corrected values). The baseline value is set to the first value of the sorted sequence. The starting position of the current segment is recorded as the index of the first position in the sorted sequence, and this first value is included in the current segment's cumulative dataset.

[0098] Secondly, dynamic segmentation detection is performed;

[0099] The sorted numerical sequence is traversed point by point, and abrupt changes are detected based on cumulative bias to generate initial segments. The specific process is as follows:

[0100] (1) Calculate the absolute deviation between the current traversal value and the current reference value;

[0101] (2) If the absolute deviation exceeds the preset threshold T or the sequence has been traversed to the end, the current segment is determined to be over;

[0102] (3) Check if the current segment length (current position index - starting position + 1) meets the preset minimum segment length L_min; if it does, record the current position index as the new boundary and add it to the segment boundary list; calculate the mean μ of the current segment cumulative dataset, fill the elements of the corresponding segment position in the correction value vector with μ; update the baseline value to the current value, reset the starting position to the current position index, and initialize a new current segment cumulative dataset (containing the current value); if it does not meet the requirements, only include the current value in the current segment cumulative dataset, update the baseline value to the current value, and do not generate a new segment.

[0103] (4) If the absolute deviation does not exceed the threshold T, the current value will continue to be included in the current segment cumulative dataset and the next value will continue to be traversed.

[0104] Next, perform final processing;

[0105] After the traversal is complete, the last segment that was not fully divided is processed: all remaining numerical points not yet included in the segment are added to the current segment, and its mean is calculated. If the segment mean list is not empty, the mean is taken from the last valid mean in the list; if the segment mean list is empty (i.e. no valid segment is generated), the mean is replaced by the overall median of the input numerical sequence. The mean is filled into the elements corresponding to the remaining positions in the correction value vector, and the start and end positions of the segment and the mean are added to the segment boundary list and the segment mean list, respectively.

[0106] Next, perform boundary optimization;

[0107] To eliminate overly dense segments, the segment boundary list is merged and optimized: the optimized boundary list is initialized, retaining the first boundary; subsequent boundaries are traversed, and the distance d between the current boundary and the last boundary in the optimized boundary list is calculated. If d > L_min, the current boundary is added to the optimized boundary list; after the traversal is completed, the last boundary is retained in the optimized boundary list, and finally the segment boundary list is updated to the optimized boundary list.

[0108] Finally, restore the original order.

[0109] Using the sorting index recorded during the data preprocessing stage, the corrected value vector is mapped back from the sorted order to the original data order to preserve the original spatial distribution of the data. The final output includes an optimized list of segment boundaries, a list of means for each segment, and the corrected complete numerical sequence.

[0110] 107. The corrected matching point pairs are stitched and fused to obtain a stitched hyperspectral image.

[0111] Here, after feature point set correction, accurate coordinates of corresponding feature points are obtained, providing a reliable geometric registration basis for image stitching. Based on the corrected feature point set, this section proposes a feature point-guided multi-segment adaptive image stitching algorithm. Through a feature point-guided segmented fusion strategy and dynamic weight calculation, it solves the problem of discontinuous boundary transitions caused by UAV flight jitter in hyperspectral image stitching, achieving seamless image stitching. This can be achieved through the following process:

[0112] First, feature point matching and global displacement estimation are performed.

[0113] Based on the corrected feature point set, the feature point coordinates are first quantized into integers. The horizontal offset between the two images is obtained by calculating the coordinate difference between pairs of feature points with the same name. The average of all horizontal offsets is taken as the global horizontal displacement parameter to construct the spatial geometric mapping relationship between the images, laying the foundation for the spatial registration of subsequent stitching.

[0114] Secondly, dynamic boundary expansion and spatial registration are performed;

[0115] Based on the vertical coordinate distribution of the corrected feature points, the dynamic expansion height of the reference image (first image) is determined: the boundary of the reference image is expanded using the extreme difference of the vertical coordinates of the feature points as a reference, and the expanded area is filled with zero values ​​to maintain the boundary space; the global horizontal displacement parameter is used to achieve accurate registration of the image to be stitched (second image) in the expanded space of the reference image, ensuring complete coverage of the area to be stitched.

[0116] Finally, the fusion region is segmented and smoothed.

[0117] Based on the horizontal coordinate distribution of feature points, the image is divided into multiple fusion sub-regions along the horizontal direction, and the start and end coordinates of each sub-region are calculated. A moving average filter is used to smooth the horizontal coordinates of the feature points, eliminating coordinate jitter caused by local noise, ensuring the continuity and stability of the fusion region division, and providing accurate region boundaries for segmented fusion. This can be achieved through the following steps:

[0118] The first step is to implement a multi-segment adaptive fusion strategy;

[0119] Based on the vertical coordinate distribution of feature points, the image is adaptively divided into three fusion regions along the vertical direction: top, middle, and bottom.

[0120] Top fusion segment: For the region above the first feature point, a gradient weight function is constructed based on the spatial distribution of non-zero pixels to achieve a weight transition from the image to be stitched to the reference image;

[0121] Intermediate fusion segment: The region between feature points is segmented, and a linear interpolation algorithm is used to calculate the weight coefficient of each horizontal position to ensure linear transition and natural connection of pixel values ​​between regions;

[0122] Bottom fusion segment: Continuing the weight calculation logic of the middle segment, it processes the area below the last feature point to complete the fusion coverage of the entire image domain.

[0123] The second step is to perform dynamic weight calculation and pixel-level fusion.

[0124] For each row of pixels, a dynamic weighting function based on the non-zero pixel distribution of the reference image is constructed: for valid pixels, the weight value increases linearly with the number of consecutive non-zero pixels, achieving a smooth transition from 0 to 1; this is achieved through a weighted fusion formula. (in , (Weighing the reference image and the image to be stitched, respectively) to achieve pixel-level fusion, effectively eliminating grayscale jumps and stitching marks at the stitching boundary.

[0125] The third step is to extract and crop the effective image region.

[0126] By detecting the smallest bounding rectangle of non-zero pixels in the fused image, the boundary coordinates of the effective image region are determined: using the minimum / maximum row and column coordinates of non-zero pixels as the cropping boundary, the extended region filled with zero values ​​is removed, the real image information is preserved, and the final seamless stitching result is output.

[0127] In this embodiment of the invention, at least two adjacent hyperspectral images are acquired, and dimensional gradient extraction is performed on the hyperspectral images to construct a spatial spectral interaction second-order matrix. Thus, during feature point extraction and matching, combining the spectral features of the hyperspectral images to construct the spatial spectral interaction second-order matrix effectively reduces mismatches caused by illumination variations and spatial noise, significantly improving the accuracy of feature point pairs. Subsequently, based on the spatial spectral interaction second-order matrix, a weighted fusion of bi-branch feature responses is performed to generate a high-discrimination response map. Based on the high-discrimination response map, feature point matching is performed on the hyperspectral images to obtain initial matched feature point pairs. Then, image segmentation and secondary matching are performed based on the initial matched feature point pairs to obtain global matched feature point pairs. This secondary matching of the initial matched feature point pairs improves the accuracy of the obtained global matched feature point pairs. Finally, outlier detection and correction are performed on the global matched feature point pairs to obtain corrected matched point pairs. The corrected matched point pairs are then image-stitched and fused to obtain a stitched hyperspectral image. In this way, by detecting and correcting outliers in globally matched feature point pairs, abnormal matching points can be accurately removed, outputting corrected feature point pairs that possess uniqueness, orderliness, and spatial uniformity. This significantly improves the purity of the feature point set and reduces the interference of mismatches on the stitching process. By stitching and fusing the corrected matching point pairs into images, the geometric accuracy and visual effect of the stitched hyperspectral image can be improved, providing high-quality data support for subsequent applications of hyperspectral data.

[0128] This invention provides a method for stitching up hyperspectral images using a drone pushbroom, which can... Figure 4 The process shown is implemented as follows:

[0129] First, two adjacent spectral images are acquired. Then, image segmentation and secondary matching are performed, such as feature point detection, feature descriptor generation, and feature point matching, to obtain preliminary matching point pairs (i.e., globally matched feature point pairs). The preliminary matching point pairs are then optimized for matching accuracy and consistency, such as outlier detection and adaptive segmentation, to obtain corrected high-quality matching point pairs (i.e., corrected matching point pairs). Finally, the corrected matching point pairs are stitched and fused, such as feature point-guided segmentation, dynamic weight fusion, and effective region extraction, to obtain the stitched hyperspectral image.

[0130] In this embodiment of the invention, during the feature point extraction and matching process, a multimodal feature descriptor (spatial features + spectral features) is constructed by combining the spectral features of the hyperspectral image, enhancing the feature discrimination in scenarios where similar land features have significant spectral differences. Compared to traditional methods that rely solely on spatial features, the introduction of spectral information expands the feature point descriptor dimension from a single spatial dimension to a multidimensional dimension of "space and spectrum," effectively reducing mismatches caused by illumination variations and spatial noise, and significantly improving the accuracy of feature point pairs. Through a "block-based recursive refinement" mechanism, a large image is decomposed into multi-level sub-blocks, gradually focusing on local features and reducing the cumulative impact of initial offset estimation errors. Combined with global coordinate adjustment and deduplication sorting, feature point pairs with uniqueness, orderliness, and spatial uniformity are output, effectively solving the problem of poor global consistency in related technologies and improving matching accuracy and global consistency. Moreover, this embodiment of the invention proposes an anomaly detection algorithm based on "neighboring feature point offset differences," which determines anomalies by the offset difference between adjacent points. Combined with an iterative elimination mechanism, it avoids over-elimination, significantly improving the purity of the feature point set, reducing the interference of mismatches on stitching, and accurately eliminating abnormal matching points. Adaptive segmentation processing, based on the cumulative bias detection sequence abrupt change features, smooths out abrupt change regions by replacing them with the mean and optimizes the boundaries, improving the local consistency of the feature point offset sequence and providing more reliable geometric constraints for subsequent registration, thus enhancing the local continuity of the sequence. This effectively solves the stitching quality problem caused by camera shake in UAV pushbroom hyperspectral imaging, improving the geometric accuracy and visual effect of the stitched image, and providing high-quality data support for subsequent applications of hyperspectral data.

[0131] This invention provides a method for stitching hyperspectral images using a drone pushbroom technique. Please refer to [link to relevant documentation]. Figure 5 The diagram illustrates the structural composition of a drone pushbroom hyperspectral image stitching system according to an embodiment of the present invention. The system 500 includes:

[0132] The acquisition module 501 is used to acquire at least two adjacent hyperspectral images;

[0133] Extraction module 502 is used to perform dimensional gradient extraction on the hyperspectral image to construct a spatial spectral interaction second-order matrix;

[0134] The fusion module 503 is used to generate a high-discrimination response map by performing weighted fusion of bi-branch feature responses based on the second-order spatial spectral interaction matrix.

[0135] The first matching module 504 is used to perform feature point matching on the hyperspectral image based on the high-discrimination response map to obtain an initial matching feature point pair;

[0136] The second matching module 505 is used to perform image segmentation and secondary matching based on the initial matching feature point pairs to obtain global matching feature point pairs.

[0137] Correction module 506 is used to detect and correct outliers in the global matching feature point pairs to obtain corrected matching point pairs;

[0138] The stitching module 507 is used to stitch and fuse the corrected matching point pairs to obtain a stitched hyperspectral image.

[0139] In some possible implementations, the extraction module 502 is further configured to extract the spatial dimension gradient and spectral gradient of the hyperspectral image to obtain the spatial gradient and spectral gradient; extract the local gradient field of the spatial gradient and spectral gradient to obtain a local gradient field including spatial and spectral local structural information; and construct the spatial-spectral interaction second-order matrix based on the local gradient field.

[0140] In some possible implementations, the fusion module 503 is further configured to perform spatial eigenvalue calculation and noise suppression on the second-order spatial spectral interaction matrix to obtain a spatial feature response map; extract spectral gradient energy from the second-order spatial spectral interaction matrix; and perform weighted fusion of the spatial feature response map and the spectral gradient energy to generate the high-discrimination response map.

[0141] In some possible implementations, the first matching module 504 is further configured to perform three-dimensional cell and block division on the high-discrimination response map to obtain feature cells; generate a three-dimensional gradient histogram based on the feature cells; fill cells in the three-dimensional gradient histogram that exceed the image boundary with zero vectors and concatenate the histograms to obtain block-level spatial features; perform dynamic spectral segmentation encoding on the three-dimensional gradient histogram to obtain spectral segmentation features; generate feature point descriptors with fixed dimensions based on the spectral segmentation features and the block-level spatial features; and perform feature point matching on the feature point descriptors to obtain the initial matched feature point pairs.

[0142] In some possible implementations, the first matching module 504 is further configured to generate a joint feature vector based on the spectral segmentation features and the block-level spatial features; and to perform global normalization processing on the joint feature vector to generate a feature point descriptor with fixed dimensions.

[0143] In some possible implementations, the first matching module 504 is further configured to perform dimensionality reduction processing on the feature point descriptor to obtain a dimensionality-reduced descriptor; calculate the matching distance matrix of the dimensionality-reduced descriptor using a broadcast mechanism; perform forward matching filtering on the feature points corresponding to the rows in the matching distance matrix to obtain forward matching results; and perform bidirectional consistency verification on the matching of the feature points corresponding to the columns in the matching distance matrix to the feature points corresponding to the rows using a reverse verification mechanism to obtain bidirectionally consistent initial matching feature point pairs.

[0144] In some possible implementations, the first matching module 504 is further configured to traverse the feature points corresponding to the column to extract the corresponding column vector; sort the column vector to obtain the sorting result of the column vector; combine the sorting result of the column vector with the feature points corresponding to the row using the nearest neighbor index to obtain the combination result; and compare the combination result with the positive matching result to obtain an initial matching feature point pair that is consistent in both directions.

[0145] In some possible implementations, the second matching module 505 is further configured to: divide the hyperspectral image into multiple image blocks based on the initial matching feature point pairs; perform coarse matching and vertical offset estimation on any two image blocks to obtain the relative misalignment of the two image blocks in the vertical direction; and perform vertical alignment on the two image blocks based on the relative misalignment in the vertical direction until a coarsely aligned image block with a preset granularity is obtained; perform recursive block division processing on the coarsely aligned image block to obtain the recursive block division processing result; perform global calibration of the matching point coordinates on the recursive block division processing result to construct a candidate matching point set for the entire image range; and perform deduplication and sorting of the matching points on the candidate matching point set to obtain the global matching feature point pairs.

[0146] In some possible implementations, the correction module 506 is further configured to: perform outlier detection on the global matching feature point pairs to obtain detected feature point pairs; determine the cumulative deviation detection mutation feature of the detected feature point pairs; generate initial segments for the detected feature point pairs based on the cumulative deviation detection mutation feature; include numerical points in the detected feature point pairs that are not included in the segments into the current segments; generate a segment boundary list and a segment mean list based on the mean of the current segments; merge the segment boundary lists to obtain a merged segment boundary list; and map the order of the correction value vectors corresponding to the merged segment boundary list and the segment mean list back to the order of the original data to obtain the corrected matching point pairs.

[0147] Optionally, the transmission medium can be a wired link (e.g., but not limited to, coaxial cable, optical fiber, and Digital Subscriber Line (DSL)) or a wireless link (e.g., but not limited to, Wireless Fidelity (WIFI), Bluetooth, and mobile device networks). It should be noted that the control device provided in the above embodiments is only an example illustrating the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the method embodiments provided in the above embodiments belong to the same concept, and their specific implementation processes are detailed in the method embodiments, and will not be repeated here.

[0148] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. For example, as shown... Figure 6 As shown, the computer device 600 includes: a memory 601, a processor 602, and a computer program 603 stored in the memory 601 and running on the processor 602, wherein when the processor 602 executes the computer program 603, the computer device can execute any of the aforementioned UAV pushbroom hyperspectral image stitching methods.

[0149] Furthermore, this embodiment of the invention also protects a control device, which may include a memory and a processor. The memory stores executable program code, and the processor is used to call and execute the executable program code to perform a UAV pushbroom hyperspectral image stitching method provided by this embodiment of the invention. This embodiment can divide the control device into functional modules based on the above method example. For example, each module can correspond to a specific function, or two or more functions can be integrated into a processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division; other division methods may exist in actual implementation. It should also be noted that all relevant content of each step involved in the above method embodiment can be referenced to the functional description of the corresponding functional module, and will not be repeated here. It should be understood that the control device provided in this embodiment is used to execute the above-mentioned UAV pushbroom hyperspectral image stitching method, and therefore can achieve the same effect as the above-mentioned implementation method. When using integrated units, the control device may include a processing module and a storage module. When the control device is applied to an equipment, the processing module can be used to control and manage the actions of the equipment. The storage module can be used to support the device in executing relevant program code, etc. The processing module can be a processor or controller, which can implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, etc., and the storage module can be a memory.

[0150] Furthermore, the control device provided in the embodiments of the present invention may specifically be a chip, component, or module. The chip may include a connected processor and a memory. The memory stores instructions, and when the processor calls and executes the instructions, the chip can execute the UAV pushbroom hyperspectral image stitching method provided in the above embodiments. This embodiment also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned method steps to implement the UAV pushbroom hyperspectral image stitching method provided in the above embodiments.

[0151] This embodiment also provides a computer program product. When the computer program product is run on a computer, it causes the computer to execute the aforementioned related steps to implement the UAV pushbroom hyperspectral image stitching method provided in the above embodiment. The control device, computer-readable storage medium, computer program product, or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects they achieve can be referred to in the beneficial effects of the corresponding methods provided above, and will not be repeated here. Through the description of the above embodiments, those skilled in the art can understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the control device can be divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by this invention, it should be understood that the disclosed control device and method can be implemented in other ways. For example, the control device embodiments described above are merely illustrative. For example, the division of modules or units is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another control device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, control device or unit, and can be electrical, mechanical or other forms.

[0152] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multiple task processing and parallel processing are possible or may be advantageous. The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. The above content is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. An unmanned aerial vehicle pushbroom hyperspectral image stitching method, characterized in that, The method includes: Acquire at least two adjacent hyperspectral images; The hyperspectral image is subjected to dimensional gradient extraction to construct a spatial-spectral interaction second-order matrix, including: extracting the spatial dimensional gradient and spectral gradient of the hyperspectral image to obtain the spatial gradient and spectral gradient; Local gradient field extraction is performed on the spatial gradient and spectral gradient to obtain a local gradient field that includes spatial and spectral local structure information; Based on the local gradient field, construct the second-order spatial spectral interaction matrix; Based on the second-order spatial spectral interaction matrix, a high-discrimination response map is generated by weighted fusion of the two-branch feature responses, including: calculating spatial eigenvalues ​​and suppressing noise on the second-order spatial spectral interaction matrix to obtain the spatial feature response map; Extract the spectral gradient energy from the second-order spatial spectral interaction matrix; The spatial feature response map and the spectral gradient energy are weighted and fused to generate the high-discrimination response map; Based on the high-discrimination response map, feature point matching is performed on the hyperspectral image to obtain initial matching feature point pairs; Based on the initial matching feature point pairs, image segmentation and secondary matching are performed to obtain global matching feature point pairs; Anomaly detection and correction are performed on the global matching feature point pairs to obtain corrected matching point pairs; The corrected matching point pairs are then stitched and fused to obtain a stitched hyperspectral image.

2. The method according to claim 1, characterized in that, The step of performing feature point matching on the hyperspectral image based on the high-discrimination response map to obtain initial matched feature point pairs includes: The high-discrimination response map is divided into three-dimensional cells and blocks to obtain feature cells; Based on the feature cells, a three-dimensional gradient histogram is generated; The cells in the three-dimensional gradient histogram that exceed the image boundary are filled with zero vectors and the histograms are concatenated to obtain block-level spatial features. Dynamic spectral segmentation encoding is performed on the three-dimensional gradient histogram to obtain spectral segmentation features; Based on the spectral segmentation features and the block-level spatial features, a feature point descriptor with fixed dimensions is generated. Feature point matching is performed on the feature point descriptor to obtain the initial matched feature point pair.

3. The method according to claim 2, characterized in that, The step of generating a fixed-dimensional feature point descriptor based on the spectral segmentation features and the block-level spatial features includes: Based on the spectral segmentation features and the block-level spatial features, a joint feature vector is generated; The joint feature vector is subjected to global normalization to generate feature point descriptors with fixed dimensions.

4. The method according to claim 2, characterized in that, The step of performing feature point matching on the feature point descriptor to obtain the initial matched feature point pair includes: The feature point descriptor is reduced in dimensionality to obtain a reduced-dimensional descriptor; The matching distance matrix of the reduced-dimensional descriptor is calculated using a broadcast mechanism; The feature points corresponding to the rows in the matching distance matrix are subjected to positive matching filtering to obtain positive matching results; A reverse verification mechanism is used to perform bidirectional consistency verification on the matching of feature points corresponding to columns in the matching distance matrix to feature points corresponding to rows, thereby obtaining initial matching feature point pairs with bidirectional consistency.

5. The method according to claim 4, characterized in that, The reverse verification mechanism is used to perform bidirectional consistency verification on the matching of feature points corresponding to columns in the matching distance matrix to feature points corresponding to rows, obtaining initial matching feature point pairs with bidirectional consistency, including: Traverse the feature points corresponding to the columns to extract the corresponding column vectors; Sort the column vectors to obtain the sorted column vectors; The sorting result of the column vector is combined with the nearest neighbor index of the feature points corresponding to the row to obtain the combined result; The combined result is compared with the positive matching result to obtain an initial matching feature point pair that is consistent in both directions.

6. The method according to claim 1, characterized in that, The step of performing image segmentation and secondary matching based on the initial matching feature point pairs to obtain global matching feature point pairs includes: Based on the initial matching feature point pairs, the hyperspectral image is divided into multiple image blocks. Coarse matching and vertical offset estimation are performed on any two image blocks to obtain the relative misalignment of the two image blocks in the vertical direction. Based on the relative misalignment of the two image blocks in the vertical direction, the two image blocks are vertically aligned until a coarsely aligned image block that meets the preset granularity is obtained. The coarsely aligned image block is recursively divided into blocks to obtain the recursive block division result; The recursive block processing results are globally calibrated to match point coordinates, and a candidate matching point set for the entire map is constructed. The candidate matching point set is deduplicated and sorted to obtain the global matching feature point pairs.

7. The method according to claim 1, characterized in that, The step of performing outlier detection and correction on the global matching feature point pairs to obtain corrected matching point pairs includes: Anomaly detection is performed on the globally matched feature point pairs to obtain the detected feature point pairs; Determine the abrupt change characteristics of the cumulative deviation of the detected feature point pairs; Based on the cumulative deviation detection mutation characteristics, the detected feature point pairs are used to generate initial segments; The numerical points in the detected feature point pairs that were not included in the segment are assigned to the current segment; Based on the mean of the current segment, generate a segment boundary list and a segment mean list; The segment boundary list is merged to obtain a merged segment boundary list; The order of the corrected value vectors corresponding to the merged segment boundary list and the segment mean list is mapped back to the order of the original data to obtain the corrected matching point pair.

8. A drone pushbroom hyperspectral image stitching system, characterized in that, The system includes: The acquisition module is used to acquire at least two adjacent hyperspectral images; The extraction module is used to extract the spatial dimension gradient and spectral gradient of the hyperspectral image to obtain the spatial gradient and spectral gradient; to extract the local gradient field of the spatial gradient and spectral gradient to obtain a local gradient field including spatial and spectral local structure information; and to construct a spatial-spectral interaction second-order matrix based on the local gradient field. The fusion module is used to calculate spatial eigenvalues ​​and suppress noise on the second-order spatial spectral interaction matrix to obtain a spatial feature response map; extract spectral gradient energy from the second-order spatial spectral interaction matrix; and perform weighted fusion of the spatial feature response map and the spectral gradient energy to generate a high-discrimination response map. The first matching module is used to perform feature point matching on the hyperspectral image based on the high-discrimination response map to obtain an initial matching feature point pair; The second matching module is used to perform image segmentation and secondary matching based on the initial matching feature point pairs to obtain global matching feature point pairs. The correction module is used to detect and correct outliers in the global matching feature point pairs to obtain corrected matching point pairs. The stitching module is used to stitch and fuse the corrected matching point pairs to obtain a stitched hyperspectral image.

Images