A high-precision band registration method and system for spaceborne hyperspectral imager

Through the blocking and precise registration method based on CMOS windowing index and SuperPoint and LightGlue, the problems of band registration accuracy and efficiency in hyperspectral remote sensing images are solved, and high-precision multi-spectral and large-scale wide image registration is achieved. Suitable for satellite-borne hyperspectral imager.

CN118411392BActive Publication Date: 2024-11-15WUHAN UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410585611.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-13
Publication Date
2024-11-15
Estimated Expiration
2044-05-13

AI Technical Summary

Technical Problem

The prior art has heading offset and imaging distortion when registering bands in hyperspectral remote sensing images, especially when the spectral characteristics between different mode bands are large and the image width is large, it is difficult to achieve high-precision registration.

Method used

Using coarse registration based on CMOS window row index and chunked and precise registration methods of SuperPoint and LightGlue, the CMOS window row sequence number difference, feature point extraction and matching, sliding window blocking processing, MTF high-pass filter and percentage Techniques such as truncated stretch lookup tables to achieve high-precision band registration.

Benefits of technology

It improves the accuracy and efficiency of band registration, reduces information redundancy, and is suitable for multi-spectral, large and wide-ranging satellite-borne hyperspectral images, significantly improving the registration accuracy of visible light and near-infrared images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118411392B_ABST
    Figure CN118411392B_ABST
Patent Text Reader

Abstract

The present invention discloses a high-precision band registration method and system for a satellite-borne hyperspectral imager, including selecting a reference band and a transformation band according to the variance of the band image DN value; using band translation to perform coarse band registration; extracting feature points from the coarse band registration results and the reference band in windows, and performing feature matching; merging the feature matching results of each window, and performing outlier elimination; based on the matching points after outlier elimination, using a polynomial transformation model to perform geometric transformation on the coarse band registration results to obtain a band fine registration result; combining the images of each band after fine registration, and cutting to obtain the final band matching result. The present invention uses a high-precision and high-speed feature extraction and matching method, combined with moving window blocking, which can significantly improve the speed and accuracy of satellite-borne hyperspectral image band registration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of remote sensing image processing, and in particular relates to a high-precision band registration method and system for a satellite-borne hyperspectral imager. Background Art

[0002] Satellite-borne hyperspectral remote sensing technology achieves large-scale, rapid and precise identification of the earth's surface by acquiring the spectral characteristics of ground objects. Linear gradient filtering technology is widely used in micro-nano satellite-borne hyperspectral imagers due to its advantages of low cost and light weight. However, linear gradient filtering technology has different imaging geometries for different bands of the same target. At the same time, under the influence of terrain, there are large heading offsets and imaging distortions between bands, and the satellite is accompanied by random jitter during the imaging process, resulting in the problem of precise band registration between hyperspectral images that cannot be solved by band translation alone.

[0003] In response to the above problems, the existing band registration technologies are divided into two categories: region-based registration methods and feature-based registration methods. The former is based on the pre-established similarity measurement and conversion model, calculates the similarity of the image according to the shallow information such as image grayscale or phase, and then adjusts the conversion model parameters through a certain optimization strategy to make the similarity measurement reach the ideal threshold, and finally achieves high-precision image alignment. The latter extracts and matches the geometric features of the image points, lines, and surfaces, and uses the spatial correspondence of the geometric features to solve the image conversion model parameters. The registration accuracy of the region-based registration method is heavily dependent on the similarity measurement, and the registration efficiency is low. It cannot be applied to the band registration of wide bandwidth and many spectral bands. The feature-based registration method uses the sparse and significant spatial geometric features in the image to abstractly express the original grayscale image, effectively suppressing the problem of information redundancy and achieving efficient registration, such as traditional feature matching methods such as FAST, ORB, SIFT, BRIEF and feature matching algorithms based on deep learning such as LoFTR, SuperGlue, and LightGlue.

[0004] When currently applied to hyperspectral remote sensing images, feature-based registration methods face problems such as large differences in spectral features between different modal bands, geometric differences in the same object in different band images, and the large width of remote sensing images that cannot be directly applied to matching algorithms based on currently popular deep learning. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a high-precision band registration method for a satellite-borne hyperspectral imager, which completes the high-precision band registration of hyperspectral remote sensing images through coarse registration based on CMOS windowing row index and block-by-block fine registration based on SuperPoint and LightGlue, and includes the following steps:

[0006] Step 1: Obtain the original data of N bands of satellite-borne hyperspectral images, calculate the variance of DN values ​​of all pixels on each band image, and select the reference band and the transformation band;

[0007] Step 2: Calculate the CMOS window row number corresponding to each band according to the wavelength of the band, the CMOS imaging components and the spectral characteristics of the linear gradient filter, and then calculate the difference Diff between the CMOS window row number of each conversion band and the reference band. i,ref ;

[0008] Step 3: Shift all pixels of the transformed band in the column direction by Diff i,ref In the next row, the missing pixels after translation are assigned a value of 0 to obtain the rough band registration result;

[0009] Step 4, extracting feature points from the band rough registration result obtained in step 3 and the reference band window, and performing feature matching;

[0010] Step 4.1, respectively calculating the percentage truncation stretching lookup table for the band rough registration result and the reference band;

[0011] Step 4.2, set the window size and moving step size, and divide the window into the band rough registration result and the reference band;

[0012] Step 4.3, filter the band rough registration result and each window of the reference band using a high-pass filter based on MTF, and then use the percentage truncation stretching lookup table calculated in step 4.1 to perform grayscale stretching;

[0013] Step 4.4, use SuperPoint feature point extractor to extract the feature point position and feature description of each window after the band rough registration result and the reference band grayscale stretching, and use LightGlue feature matcher to match the feature points of each window after the band rough registration result and the reference band grayscale stretching;

[0014] Step 5, summarizing the band coarse registration results and the matching results of each window feature of the reference band, and removing the outliers, and based on the matching points after the outliers are removed, using the polynomial transformation model to perform geometric transformation on the band coarse registration results to obtain the band fine registration results;

[0015] Step 5.1, merging the band rough registration results with the reference band each window feature point matching results;

[0016] Step 5.2, based on the perspective transformation constraint, use the USAC_MAGSAC algorithm to remove external points and obtain the matching result that meets the perspective transformation constraint;

[0017] Step 5.3, using the projection parameters of the satellite-borne hyperspectral image, transform the matching result obtained in step 5.2 that meets the perspective transformation constraint into the corresponding projection coordinate system;

[0018] Step 5.4, substitute the coordinates of the matching points in the projection coordinate system calculated in step 5.3 into the polynomial transformation model, solve the parameters of the polynomial transformation model, use the polynomial transformation model to perform geometric transformation on the rough registration result of the band, and obtain the fine registration result;

[0019] Step 6: Combine the images of each band after precise registration to obtain a multi-band image, calculate the maximum adjacent rectangle of the multi-band image, and cut out the image according to the maximum adjacent rectangle to obtain the final band matching result.

[0020] Furthermore, in step 1, the original data of N bands of satellite-borne hyperspectral images are obtained, the variance of the DN values ​​of all pixels on each band image is calculated, and the band corresponding to the minimum variance value is selected as the reference band B. ref , other bands are transformed band B i ,i∈[1,2,…,ref)∪(ref,…,N].

[0021] Moreover, the calculation formula of the CMOS window row number corresponding to each band in step 2 is as follows:

[0022] Line k =a×Wavelength k +b (1)

[0023] In the formula, Line k is the CMOS window row number of the kth band, Wavelength k is the wavelength of the kth band, k∈(1,2,…,N), a and b are linear transformation parameters, which are related to the parameters of CMOS imaging components and linear gradient filters.

[0024] The difference in CMOS window row number between each conversion band and the reference band Diff i,ref The calculation is as follows:

[0025] Diff i,ref =Line ref -Line i (2)

[0026] In the formula, Line ref 、Line i They are the window row numbers of the reference band and the transformed band respectively.

[0027] Moreover, in step 4.1, the grayscale histogram of the band image is calculated, and the grayscale histogram is truncated at a ratio of δ1% (δ1% is set according to actual needs) to obtain the maximum and minimum values ​​after truncation, and the maximum and minimum values ​​are used as the linear stretching range to generate a percentage truncation stretching lookup table.

[0028] The present invention also provides a high-precision band registration system for a space-borne hyperspectral imager, which is used to implement the high-precision band registration method for a space-borne hyperspectral imager as described above.

[0029] Furthermore, the invention comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the high-precision band registration method for a space-borne hyperspectral imager as described above.

[0030] Alternatively, it comprises a readable storage medium having a computer program stored thereon, and when the computer program is executed, the high-precision band registration method for a space-borne hyperspectral imager as described above is implemented.

[0031] Compared with the prior art, the present invention has the following advantages:

[0032] 1) The present invention starts from the imaging principle of linear gradient filter and translates the transformed band image based on CMOS window opening line, so that there is no need to perform window search during sub-window matching, thereby accelerating the band registration speed.

[0033] 2) The present invention uses sliding window block parallel processing technology, which can not only apply the feature matching technology based on deep learning in the form of a small window, but also improve the registration speed of multi-spectral and wide-band satellite-borne hyperspectral image bands.

[0034] 3) After using the sliding window to divide the image into blocks, the present invention uses a high-pass filter based on MTF to filter each window, which can effectively improve the clarity and details of the image and reduce the image blur and distortion caused by the optical system.

[0035] 4) The present invention adopts the most advanced SuperPoint feature point extractor and LightGlue feature matcher, among which LightGlue can adapt the calculation amount according to the difficulty of each pair of images, is more efficient and accurate in memory and calculation, has faster reasoning speed, and has great potential in the field of satellite-borne hyperspectral image band registration.

[0036] 5) The present invention uses a percentage truncation stretch lookup table for grayscale stretching, which can homogenize the grayscale distribution of each window and reduce the impact of window grayscale distribution differences caused by central wavelength differences. Combined with the SuperPoint feature point extractor and the LightGlue feature matcher, it can significantly improve the registration accuracy of visible light images and near-infrared images. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 This is a flow chart of a high-precision band registration method for a space-borne hyperspectral imager according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following takes the 9-band hyperspectral image of "Venus-1" as an example, combined with the drawings in the present invention, to clearly and completely describe the technical solutions in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Example 1

[0041] like Figure 1 As shown, the embodiment of the present invention provides a high-precision band registration method for a space-borne hyperspectral imager, comprising the following steps:

[0042] Step 1: Get the original data of the 9-band hyperspectral image of Venus-1. The original image size is 12560×5056. Calculate the variance of the DN values ​​of all pixels in each band image, and select the band corresponding to the minimum variance value as the reference band B. ref , use other bands as transformation band B i ,i∈[1,2,…,ref)∪(ref,…,9].

[0043] Step 2: Calculate the CMOS window row number corresponding to each band according to the band wavelength, CMOS imaging components and the spectral characteristics of the linear gradient filter.

[0044] The calculation formula for the CMOS window row number corresponding to each band is as follows:

[0045] Line k =a×Wavelength k +b (1)

[0046] In the formula, Line k is the CMOS window row number of the kth band, Wavelength k is the wavelength of the kth band, k∈(1,2,…,9), a and b are linear transformation parameters, which are related to the parameters of the CMOS imaging components and the linear gradient filter, and the values ​​of this embodiment are 3.457 and -1036.3 respectively.

[0047] Step 3, calculate the difference in CMOS window row number between each transformed band and the reference band.

[0048] The difference in CMOS window row number between each conversion band and the reference band Diff i,ref The calculation is as follows:

[0049] Diff i,ref =Line ref -Line i (2)

[0050] In the formula, Line ref 、Line i They are the window row numbers of the reference band and the transformed band respectively.

[0051] Step 4: Diff calculated in step 3 i,ref , will transform band B i All pixels are shifted in the column direction by Diff i,ref The missing pixels after translation are assigned a value of 0 to obtain the band rough registration result B i ′.

[0052] Step 5: Coarse registration of the bands obtained in step 4 i ′ and reference band B ref Extract feature points by window and perform feature matching.

[0053] Step 5.1, respectively for reference band B ref With the band rough registration result B i ' Calculate the percentage cutoff stretch lookup table.

[0054] Calculate the grayscale histogram of the band image, truncate the grayscale histogram at a ratio of 1%, obtain the maximum and minimum values ​​after truncation, use the maximum and minimum values ​​as the linear stretching range, and generate a percentage truncation stretching lookup table for grayscale stretching.

[0055] Step 5.2: Use a window size of 1024×1024 and a moving step size of 768×768 to perform a ref With the band rough registration result B i 'Divide the window and get the window Block ref,j and Block i,j , where j is the window number.

[0056] Step 5.3, for each Block ref,j and Block i,j The window is filtered using a high-pass filter based on the Modulation Transfer Function (MTF), and then the grayscale is stretched using the percentage truncation stretch lookup table calculated in step 5.1 to obtain Block r ' ef,j and Block i ' ,j .

[0057] Step 5.4, use the SuperPoint feature point extractor to extract each Block r ' ef,j and Block i ' ,j The feature point position and feature description of the Block are matched using the LightGlue feature matcher. r ' ef,j and Block i ' ,j Perform feature point matching.

[0058] Step 6: Summarize the band rough registration results B i ′ and reference band B ref The feature matching results of each window are obtained, and outliers are removed. Based on the matching points after outliers are removed, the polynomial transformation model is used to transform B i ′ Perform geometric transformation to obtain the band precise registration result B i ″.

[0059] Step 6.1: Combine the band rough registration results with the reference band window feature point matching results to obtain P i and P ref .

[0060] Step 6.2: Based on the perspective transformation constraint, use the USAC_MAGSAC algorithm to remove external points and obtain the matching result P that meets the perspective transformation constraint. i ′ and P r ' ef .

[0061] Step 6.3, use the projection parameters of the Venus-1 hyperspectral image to convert P i ′ and P r ' ef Convert to the corresponding projection coordinate system coordinate P i ″ and P r ' e ' f .

[0062] Step 6.4, based on the coordinates P of the matching point in the projection coordinate system calculated in step 6.3 i ″ and P r ' e ' f , using the polynomial transformation model for B i ′ Perform geometric transformation to obtain the precise registration result B i ″.

[0063] The coordinates of the matching point P in the projection coordinate system calculated in step 6.3 i ″ and P r ' e ' f Substitute it into the polynomial transformation model, solve the polynomial transformation model parameters, and use the polynomial transformation model to transform B i ′ Perform geometric transformation to obtain the precise registration result B i ″.

[0064] Step 7, combining the precisely registered band images to obtain a multi-band image, and calculating the maximum adjacent rectangle of the multi-band image effective value, and cropping the image according to the maximum adjacent rectangle to obtain the final band matching result.

[0065] Example 2

[0066] Based on the same inventive concept, the present invention also provides a high-precision band registration system for a space-borne hyperspectral imager, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the program instructions in the memory to execute the high-precision band registration method for a space-borne hyperspectral imager as described above.

[0067] Example 3

[0068] Based on the same inventive concept, the present invention also provides a high-precision band registration system for a satellite-borne hyperspectral imager, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a high-precision band registration method for a satellite-borne hyperspectral imager as described above is implemented.

[0069] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0070] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific implementation cases described, or replace them with similar methods, without departing from the spirit of the present invention or exceeding the scope defined by the appended claims.

Claims

1. A high-precision band registration method for a spaceborne hyperspectral imager, characterized in that: The following steps are involved: Step 1: Obtain the original data of N bands of satellite-borne hyperspectral images, calculate the variance of DN values ​​of all pixels on each band image, and select the reference band and the transformation band; Step 2: Calculate the CMOS window row number corresponding to each band according to the wavelength of the band, the CMOS imaging components and the spectral characteristics of the linear gradient filter, and then calculate the difference Diff between the CMOS window row number of each conversion band and the reference band. i,ref ; The calculation formula for the CMOS window row number corresponding to each band is as follows: Line k =a×Wavelength k +b (1) In the formula, Line k is the CMOS window row number of the kth band, Wavelength k is the wavelength of the kth band, k∈(1,2,…,N), a and b are linear transformation parameters, which are related to the parameters of CMOS imaging components and linear gradient filters; The difference in CMOS window row number between each conversion band and the reference band Diff i,ref The calculation is as follows: Diff i,ref =Line ref -Line i (2) In the formula, Line ref 、Line i are the window row numbers of the reference band and the transformed band respectively; Step 3: Shift all pixels of the transformed band in the column direction by Diff i,ref In the next row, the missing pixels after translation are assigned a value of 0 to obtain the rough band registration result; Step 4, extracting feature points from the band rough registration result obtained in step 3 and the reference band window, and performing feature matching; Step 4.1, respectively calculating the percentage truncation stretching lookup table for the band rough registration result and the reference band; Step 4.2, set the window size and moving step size, and divide the window into the band rough registration result and the reference band; Step 4.3, filter the band rough registration result and each window of the reference band using a high-pass filter based on MTF, and then use the percentage truncation stretching lookup table calculated in step 4.1 to perform grayscale stretching; Step 4.4, use SuperPoint feature point extractor to extract the feature point position and feature description of each window after the band rough registration result and the reference band grayscale stretching, and use LightGlue feature matcher to match the feature points of each window after the band rough registration result and the reference band grayscale stretching; Step 5, summarizing the band coarse registration results and the matching results of each window feature of the reference band, and removing the outliers, and based on the matching points after the outliers are removed, using the polynomial transformation model to perform geometric transformation on the band coarse registration results to obtain the band fine registration results; Step 6: Combine the images of each band after precise registration to obtain a multi-band image, calculate the maximum adjacent rectangle of the multi-band image, and cut out the image according to the maximum adjacent rectangle to obtain the final band matching result.

2. The high-precision band registration method for a space-borne hyperspectral imager according to claim 1, characterized in that: In step 1, the original data of N bands of satellite-borne hyperspectral images are obtained, the variance of the DN values ​​of all pixels in each band image is calculated, and the band corresponding to the minimum variance value is selected as the reference band B. ref , other bands are transformed band B i ,i∈[1,2,…,ref)∪(ref,…,N].

3. The high-precision band registration method for a space-borne hyperspectral imager according to claim 1, characterized in that: In step 4.1, the grayscale histogram of the band image is calculated, and the grayscale histogram is truncated at a ratio of δ1%. δ1% is set according to actual needs to obtain the maximum and minimum values ​​after truncation. The maximum and minimum values ​​are used as the linear stretching range to generate a percentage truncation stretching lookup table.

4. The high-precision band registration method for a space-borne hyperspectral imager according to claim 1, characterized in that: Step 5 includes the following steps: Step 5.1, merging the band rough registration results with the reference band each window feature point matching results; Step 5.2, based on the perspective transformation constraint, use the USAC_MAGSAC algorithm to remove external points and obtain the matching result that meets the perspective transformation constraint; Step 5.3, using the projection parameters of the satellite-borne hyperspectral image, transform the matching result obtained in step 5.2 that meets the perspective transformation constraint into the corresponding projection coordinate system; Step 5.4, substitute the coordinates of the matching points in the projection coordinate system calculated in step 5.3 into the polynomial transformation model, solve the parameters of the polynomial transformation model, use the polynomial transformation model to perform geometric transformation on the rough band registration result, and obtain the fine registration result.

5. A high-precision band registration system for a spaceborne hyperspectral imager, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store program instructions, and the processor is used to call the program instructions in the memory to execute a high-precision band registration method for a space-borne hyperspectral imager as described in any one of claims 1 to 4.

6. A high-precision band registration system for a spaceborne hyperspectral imager, characterized in that: It comprises a readable storage medium, on which a computer program is stored. When the computer program is executed, a high-precision band registration method for a space-borne hyperspectral imager as described in any one of claims 1 to 4 is implemented.

Citation Information

Patent Citations

  • Reference signal detection method and system, readable storage medium and electronic equipment

    CN111835495A

  • Multi-source remote sensing monitoring method for crop phenotype information

    CN112147078A

  • Push-broom remote sensing satellite image high-precision wave band alignment method based on integrogram

    CN115761528A

  • Spectrum calibration method and system for satellite-borne hyperspectral imager

    CN117168619A

  • Method for processing hyperspectral images

    US20220270362A1