An imaging spectrometer image band registration method and system

Abstract

This invention discloses an image band registration method and system for imaging spectrometers. The method extracts initial values ​​of registration errors between other multispectral imaging bands and a reference band using phase correlation, then uses the registration error at the nadir point position as the principal point error. Based on the initial values ​​of a rigorous imaging model, a simulated image of a oscillating mirror scanning at a constant speed is generated using image simulation to establish a scanning instability error model. Finally, the initial values ​​of inter-band registration errors are corrected by fusing principal point offset and scanning error. Furthermore, a dynamic frame edge interval detection mechanism is employed, combined with geolocation data for object-space coordinate interpolation and inverse calculation of image-space coordinates for corresponding points. Innovatively, a weighted fusion function of corresponding points and resampling weights is introduced to achieve grayscale value weighted fusion, and iterative processing completes the correction of image point errors across the entire image domain. This invention effectively improves inter-band registration accuracy and is suitable for processing spaceborne remote sensing images using oscillating scanning imaging.

Description

Technical Field

[0001] This invention belongs to the field of satellite image processing technology, and relates to image preprocessing of meteorological, marine and other satellite imaging systems. Specifically, it relates to an image band registration method and system based on a spaceborne sweeping camera for an imaging spectrometer. Background Technology

[0002] Oscillating cameras, through their unique oscillating scanning method, can acquire large-area ground images covering multiple spectral bands, providing abundant multispectral image data resources for numerous applications such as topographic mapping, environmental monitoring, and agricultural remote sensing. Although the multispectral images acquired by spaceborne oscillating cameras contain rich and valuable information, due to various factors during their formation, the images in different spectral bands exhibit geometric and radiometric differences, resulting in an misaligned state that cannot be directly used for comprehensive analysis and quantitative applications.

[0003] Due to their unique oscillating scanning method, oscillating cameras exhibit a phenomenon known as the "bowlet effect" during image acquisition. This effect stems from the change in viewing angle caused by the oscillation of the scanning platform during the scanning process, leading to geometric distortion between adjacent scan strips. Specifically, the ground sampling areas in adjacent frames overlap at the scanning ends due to factors such as the scanning angle, camera oscillation trajectory, and sensor dynamic response. This means that adjacent frames may include the same ground area, resulting in a significant discontinuity in grayscale values ​​between them. Since image registration requires consistent grayscale texture, this discontinuity not only affects the radiometric consistency of the image but also significantly reduces the registration accuracy between multispectral bands, posing a challenge for oscillating cameras in multispectral data fusion and quantitative analysis. Therefore, effectively addressing this phenomenon and reducing the impact of the bowlet effect on inter-band registration is crucial for achieving high-precision multispectral image processing with oscillating cameras. Summary of the Invention

[0004] The purpose of this invention is to solve / reduce the impact of the bowtie effect on inter-band registration in the prior art, avoid significant discontinuities in grayscale values ​​between adjacent image frames, lay the foundation for improving the image registration accuracy of spaceborne sweeping cameras, achieve high-precision registration between multispectral images of spaceborne sweeping cameras, and eliminate spatial inconsistencies between multispectral image bands. The technical solution of this invention provides an image band registration method and system for imaging spectrometers, which can effectively improve the accuracy of multispectral registration of sweeping cameras and minimize spatial inconsistencies and differences between multispectral images.

[0005] Therefore, the present invention provides the following technical solution:

[0006] This invention provides an image band registration method for an imaging spectrometer, applied to multispectral images acquired by a sweeping camera. The method includes the following steps:

[0007] S1: Obtain the registration model parameters of the multispectral images of the sweep camera between bands, and perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters.

[0008] S2: Based on the image-side coordinates of the offset-corrected image points in the image to be registered, determine the frame number of the current image point and the interval from the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0009] S3: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points of the current image point in adjacent frames;

[0010] S4: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive-size intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging.

[0011] In this process, the grayscale value of each pixel in the image to be registered is resampled according to steps S2-S4 above.

[0012] Preferably, the adaptive-size intra-sampling kernel is determined based on the frame number of the pixel and the interval from the edges of the preceding and following frames. Specifically, when using a bicubic polynomial sampling kernel or a bilinear sampling kernel, the correspondence between the interval type and the intra-sampling kernel type is as follows:

[0013] Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 4*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 3*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 2*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel.

[0014] Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single pixel sampling kernel, that is, 2*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual pixel sampling kernel, that is, 1*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel.

[0015] The resampling formula is:

[0016]

[0017] In the formula, g is the gray value of the resampled pixel, N1 and N2 represent the number of sampling kernel pixels in the current frame and the adjacent frame, respectively, and g i and g j ω represents the grayscale value of each sampling point within the two sampling kernels of the current frame and the adjacent frame. i and ω j All of these are the weights corresponding to the sampling points.

[0018] Preferably, when using a bicubic polynomial sampling kernel, in step S4, the weights are calculated for the sampling points selected within their respective sampling kernels for the image point and the corresponding point. The resampling weight kernel function is as follows:

[0019]

[0020] In the formula, x is the distance between the image point or the corresponding point and the sampling point in its respective sampling kernel, and ω(x) is the weight.

[0021] Preferably, if a bicubic polynomial sampling kernel is used, and the interval between the current pixel points is greater than 2 pixels; or if a bilinear sampling kernel is used, and the interval between the current pixel points is greater than 1 pixel, then no corresponding points are introduced to participate in resampling. That is, for the current pixel point, step S3 is not executed, and in step S4, resampling is performed at the offset-corrected pixel coordinates within the corresponding sampling kernel of this frame.

[0022] In addition, step S3 is performed in all cases, and the calculation process of the image coordinates of the image point in step S3 for the corresponding points in adjacent frames is as follows:

[0023] Using initial geolocation data and based on the image coordinates (i) of the current image point tran ,j tran Within the current frame, perform linear interpolation on the geolocation data in both the flight and scan directions to obtain the latitude and longitude difference Δlat between the current pixel and its neighboring pixels in both directions. x Δlat y Δlon x Δlon y lat represents latitude, lon represents longitude, and xy corresponds to the x and y axes respectively;

[0024] Next, the object space coordinates of the current image point in the current frame are calculated using bilinear interpolation.

[0025] Using the object coordinates of the current image point The image-side coordinate correction value (Δx, Δy) is calculated with the image-side coordinates (x0, y0) of the initial image point in the adjacent frame. The image-side coordinates (x0, y0) of the initial image point in the adjacent frame are arbitrarily selected within the range of the adjacent frames (generally the upper left corner of the adjacent frame is selected).

[0026] Based on the initial image point image-space coordinates (x0, y0), the corresponding object-space coordinates lon(x0, y0) and lat(x0, y0) are calculated using bilinear interpolation. Simultaneously, the two-dimensional gradient change lon' of the object-space coordinates is calculated based on the object-space coordinates of adjacent image points. x (x0,y0),lat' x (x0,y0),lon' y (x0,y0),lat' y (x0, y0), abbreviated as lon' x ,lat' x ,lon' y ,lat' y ;

[0027] Through formula Calculate the differences in object coordinates Δlon and Δlat; then substitute the above parameters into the formula. Calculate the image-side coordinate corrections Δx and Δy;

[0028] Then, the image-side coordinates of the new iterative image point are obtained by replacing (x0, y0) with (x0+Δx, y0+Δy);

[0029] The process involves iteratively calculating each parameter according to the steps outlined above until the image-side coordinate correction value is less than 1, indicating that two image points are within one pixel. This process yields the final image-side coordinate correction value (Δx) at the end of the iteration. n ,Δy n Then, the image coordinates of the corresponding point (i) are linearly interpolated from the object-space two-dimensional gradient of the iterative image point and the given object-space coordinates. adjoin ,j adjoin Then, the image coordinates of the current image point and the image coordinates of the corresponding points are transmitted and registered.

[0030] Furthermore, this invention provides an image band registration error estimation method based on the above-mentioned imaging spectrometer image band registration method, used in step S1 to obtain the registration model parameters of the multispectral images of the sweeping camera between bands and to perform coordinate offset correction. The method includes the following steps:

[0031] S11: Sources of registration error between bands in multispectral images based on a sweeping camera, wherein the sources of registration error include at least: principal point error of the focal plane and scanning instability error, with the following relationship:

[0032]

[0033] In the formula, Δp x and Δp y Δx represents the image square error in the flight and scanning directions, respectively. 01 and Δy 01 These represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; μ represents the load CCD magnitude; f and Δf are the design value and change value of the principal distance, respectively; and x... 01 The principal point position indicates the direction of flight, and Δθ represents the change in scanning angle caused by the non-uniform scanning speed. btb This indicates the angle of the oscillating mirror scan within the sampling interval;

[0034] S12: Select the reference band and use the phase correlation method to extract the overall registration error between multispectral image bands, and then take the overall registration error of the nadir position. Calculate the principal point error Δx 01 and Δy 01 ;

[0035] S13: A simulated image of the reference band during uniform scanning of the swing mirror is generated by an image simulation method. Then, a high-precision matching point between the original image of the reference band and the simulated image of the corresponding wavelength range is obtained by using image correlation matching combined with the least squares matching algorithm, which serves as the control point.

[0036] S14: Based on the image point offset of the control points between the original image and the simulated image as the error, the median error of every eight columns of control points is taken in the scanning direction to make the offset curve of the flight direction and the scanning direction, wherein the scanning direction is the image column direction;

[0037] S15: Based on the characteristics of control point error changing with scanning angle, the scanning angle variation model is fitted using the sum of sine functions on the offset curve of the scanning direction, i.e., the scanning non-stationary error model of the reference band:

[0038]

[0039] In the formula, θ(t) is the scanning angle, n is the number of sine functions, and ω i φ is the sample frequency. i Let a be the phase of the sine function. i Let be the amplitude of the sine function, and t represent time.

[0040] S16: Introduce the principal point error Δx for principal point offset estimation 01 and Δy 01As the initial phase offset, the scanning non-stationary error model θ(t2) of the band to be registered is then fitted. The difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered is then used to obtain the inter-band scanning non-stationary registration model Δθ.

[0041] Δθ(s)=θ(t1)-θ(t2)

[0042] In the formula, s represents the coordinates of the image in the scanning direction, and t1 and t2 represent the design time for starting scanning and imaging in the two bands, respectively.

[0043] S17: Using the aforementioned inter-band scanning non-stationary registration model, pixel-by-pixel (x,y) offset correction is performed between the remaining multispectral imaging bands and the reference band: Obtain the offset image-side coordinates (x', y');

[0044] In the formula, Δp x and Δp y It is based on The parameters of the registration model are determined.

[0045] The present invention also provides a registration system based on the above registration method, comprising:

[0046] The registration module is used to obtain the registration model parameters of the multispectral images of the sweep camera between bands, and to perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters.

[0047] The sampling kernel determination module is used to determine the frame number of the current image point and the interval from the edge of the previous and next frames based on the image-side coordinates of the offset-corrected image points in the image to be registered, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0048] The corresponding point processing module is used to obtain the object-space coordinates of the current image point and, based on the object-space coordinates, obtain the image-space coordinates of the corresponding points of the current image point in adjacent frames.

[0049] The resampling module is used to select a corresponding number of neighboring image points around the current image point and the corresponding image point as sampling points based on the image coordinates of the current image point and the intra-frame sampling kernel of adaptive size. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the gray value of the current image point after resampling is obtained by weighted average calculation.

[0050] In this process, the grayscale value of each pixel in the image to be registered is resampled according to the above procedure.

[0051] Preferably, the registration module of the system includes:

[0052] The registration error analysis unit is used to analyze the sources of registration errors between bands in multispectral images from a sweeping camera. These sources include at least: principal point error of the focal plane and scanning instability error, with the following relationship:

[0053]

[0054] In the formula, Δp x and Δp y Δx represents the image square error in the flight and scanning directions, respectively. 01 and Δy 01 These represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; μ represents the load CCD magnitude; f and Δf are the design value and change value of the principal distance, respectively; and x... 01 The principal point position indicates the direction of flight, and Δθ represents the change in scanning angle caused by the non-uniform scanning speed. btb This indicates the angle of the oscillating mirror scan within the sampling interval;

[0055] The principal point error analysis unit is used to select a reference band and extract the overall registration error between multispectral image bands using the phase correlation method, and then obtain the overall registration error of the nadir position. Calculate the principal point error Δx 01 and Δy 01 ;

[0056] The control point extraction unit is used to generate a simulated image of the reference band when the mirror is scanning at a constant speed using an image simulation method. Then, it uses an image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range, which are used as control points.

[0057] The offset curve generation unit is used to generate offset curves for the flight direction and the scanning direction by taking the median error of the control points between the original image and the simulated image, based on the image point offset of the control points between the original image and the simulated image. The scanning direction is the image column direction.

[0058] The fitting unit is used to fit the scanning angle variation model based on the characteristics of control point error changing with scanning angle, using the offset curve of the scanning direction and the sum of sine functions, i.e., the scanning non-stationary error model of the reference band; and to introduce the principal point error Δx estimated by principal point offset. 01 and Δy 01 As the initial phase offset, the scanning non-stationary error model θ(t2) of the band to be registered is then fitted. The difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered is then used to obtain the inter-band scanning non-stationary registration model Δθ.

[0059] The registration unit is used to perform pixel-by-pixel (x,y) offset correction between the other multispectral imaging bands and the reference band using the inter-band scanning non-stationary registration model. Obtain the offset image-side coordinates (x', y').

[0060] The present invention also provides a computer device comprising at least: one or more processors and a memory storing one or more computer programs;

[0061] The processor calls the computer program to implement:

[0062] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0063] The present invention also provides a computer-readable storage medium storing a computer program, which is invoked by a processor to implement the following:

[0064] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0065] Beneficial effects

[0066] Compared with existing methods, the advantages of the present invention are:

[0067] 1. The technical solution of this invention obtains the corresponding points of the current image point in adjacent frames. By introducing geolocation data and introducing corresponding points for resampling, it solves the problem of gray value discontinuity caused by the bowtie effect mentioned in the background technology and the gray value error introduced during the resampling process, ensuring that the gray value is not contaminated by the bowtie effect.

[0068] 2. In a further preferred embodiment of the present invention, the size of the intra-frame sampling kernel is adaptively determined based on the frame number of the image point and the interval between the edge of the upper and lower frames, which can effectively eliminate pixels with discontinuous gray values ​​introduced during the sampling point selection process of the sampling kernel and reduce gray value pollution.

[0069] 3. Through precise image matching, error calculation and analysis based on a rigorous imaging geometry model, the registration error between image bands can be effectively estimated. Taking into account the resampling method of the sweeping imaging mechanism, the registration accuracy can be improved, making the position of the image between bands more accurate, and providing more reliable data support for subsequent applications such as image fusion and target recognition.

[0070] 4. The method of the present invention fully considers the imaging characteristics and error sources of spaceborne oscillating cameras, can adapt to different imaging conditions and data characteristics, has strong versatility and adaptability, and can be widely applied to the processing of images from various spaceborne oscillating cameras. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of registration error estimation according to an embodiment of the present invention;

[0072] Figure 2 This is a schematic diagram of the registration error between images after registration according to the present invention;

[0073] Figure 3 This is a schematic diagram of the resampling accuracy test of the present invention. Figures a, b, c, and d correspond to the present invention, and the comparison results of existing cubic polynomial interpolation method, existing bilinear interpolation method, and resampling method.

[0074] Figure 4 This is a flowchart provided in an embodiment of the present invention. Detailed Implementation

[0075] This invention provides an image band registration method and system for imaging spectrometers, which effectively solves / reduces the problem of the bowtie effect affecting inter-band registration in the prior art, avoiding significant discontinuities in grayscale values ​​between adjacent frames. This lays the foundation for improving the image registration accuracy of spaceborne sweeping cameras and achieving high-precision registration between multispectral images from spaceborne sweeping cameras. One core aspect of this invention is the resampling of multispectral images acquired by the sweeping camera. Specifically, by acquiring corresponding points of the current image point in adjacent frames and introducing these corresponding points for resampling, the grayscale values ​​are ensured to be free from the bowtie effect. Another core aspect of this invention is the full consideration of the sources of registration errors between bands in multispectral images from sweeping cameras, specifically the principal point error of the focal plane and scanning instability error. Therefore, an inter-band scanning instability registration model is constructed to eliminate images with the aforementioned registration errors, thereby improving registration accuracy.

[0076] The technical approach for band registration of imaging spectrometer images acquired by a sweeping camera is as follows:

[0077] S1: Obtain the registration model parameters of the multispectral images of the oscillating camera (belonging to the imaging spectrometer) between bands, and perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters.

[0078] S2: Based on the image-side coordinates of the offset-corrected image points in the image to be registered, determine the frame number of the current image point and the interval from the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0079] S3: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points in adjacent frames;

[0080] S4: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive-size intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging.

[0081] In this process, the grayscale value of each pixel in the image to be registered is resampled according to steps S2-S4 above.

[0082] Furthermore, the registration error estimation method applied to images acquired by a sweeping camera is used in step S1 to obtain the registration model parameters and coordinate offset corrections for the multispectral images of the sweeping camera between bands. The technical idea of ​​the registration error estimation method is as follows:

[0083] Based on the sources of registration error between bands in multispectral images from a sweeping camera, an inter-band scanning non-stationary registration model is constructed. The sources of this registration error include at least: principal point error of the focal plane and scanning non-stationary error.

[0084] Select a reference band and construct a scanning non-stationary error model for the reference band and a scanning non-stationary error model for the band to be registered.

[0085] Finally, the difference between the scanning non-stationary error model of the reference band and the scanning non-stationary error model of the band to be registered is used to obtain the inter-band scanning non-stationary registration model, which is then substituted into... Δp is calculated in the model x and Δp y Furthermore, pixel-by-pixel (x,y) offset corrections are performed between the remaining multispectral imaging bands and the reference band:

[0086] The present invention will be further described below with reference to embodiments.

[0087] In this embodiment of the invention, the inter-band registration error of the multispectral image is preferably estimated first, and then the multispectral image is resampled. Therefore, the specific implementation process is as follows:

[0088] Step 1: Determine the sources of registration error and construct an inter-band scanning non-stationary registration model.

[0089] This embodiment uses L1B level (Level 1B, representing image data after preliminary processing) image data from a sweeping imaging payload. A rigorous imaging geometric model is constructed through payload design parameters to analyze the sources of registration errors between bands, which are mainly divided into principal point errors and scanning instability errors.

[0090] The specific analysis is as follows: The camera simultaneously samples light through the tilting mirror, which is then transmitted to the focal plane system via the optical path system and projected onto detectors in different spectral channels. During this process, the optical path system experiences a consistent influence; therefore, the registration error between bands mainly originates from the focal plane, including at least the principal point error. Furthermore, due to the non-uniform scanning speed of the tilting mirror, the light received in different bands exhibits a shift related to the scanning angle, as shown below:

[0091]

[0092] In the formula, Δp x and Δp y The image square error Δx represents the image plane error in the flight and scanning directions, respectively. 01 and Δy 01 The values ​​represent the principal point offset of the band to be registered on the focal plane, μ represents the magnitude of the load CCD (Charge-Coupled Device), f and Δf are the design value and change value of the principal distance, respectively, and x 01 The principal point position indicates the direction of flight, and Δθ represents the change in scanning angle caused by the non-uniform scanning speed. btb This indicates the angle scanned by the pendulum mirror within the sampling interval.

[0093] Step 2: Select the reference band and extract the overall registration error between the remaining multispectral imaging bands and the reference band in the multispectral images in the flight and scanning directions.

[0094] In this embodiment of the invention, a high-precision image matching phase correlation method is preferably used to extract the overall registration error between the image flight and scanning upward bands. Near-infrared or visible light bands are also preferably used as the reference bands.

[0095] Specifically, sweeping cameras are typically mounted on platforms such as satellites or aircraft, generating multispectral images under the combined influence of the flight direction (along the orbital direction) and the scanning direction (perpendicular to the orbital direction). This invention preferably selects multispectral image data captured by sweeping cameras over large, cloudless land areas. For the visible and near-infrared bands, based on the corresponding band images from the sweeping camera, the registration error between the reference band and the band to be registered is calculated using a phase correlation method. To achieve high-precision registration of multimodal images and avoid a decrease in registration accuracy due to radiometric inconsistencies between multimodal images, for other infrared bands... The method employs image simulation, such as for the thermal infrared band. Landsat 8 imagery is used as the base map data to simulate and obtain a simulated image for the thermal infrared band. The corresponding grayscale value is calculated pixel by pixel using geolocation data and a rigorous imaging geometric model. The registration error between the thermal infrared band and the reference band is extracted by using phase correlation between the simulated image and the original thermal infrared band image. In this way, the thermal infrared band image generated by simulation using accurate geolocation data can be regarded as an image that is consistent with the reference band registration, which can effectively avoid the impact of multimodal matching on registration accuracy.

[0096] In this embodiment, the overall registration error of the nadir point (the nadir point is the ground point directly below the satellite orbit) is used to infer the principal point error, and the principal point error parameter Δx is constructed. 01 and Δy 01 .

[0097] Step 3: Construct the scanning non-stationary error model of the reference band and the scanning non-stationary error model of the band to be registered. Then, calculate the difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered to obtain the scanning non-stationary registration model Δθ between the bands.

[0098] Taking the scanning non-stationarity error model of the reference band as an example:

[0099] Step 3-1: Generate a simulated image of the reference band when the mirror is scanning at a constant speed using an image simulation method. Then, use an image correlation matching algorithm combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range, which are used as control points.

[0100] This embodiment intends to use Landsat-8 global 250M resolution imagery as base map data, and perform pixel-by-pixel simulation based on the initial values ​​of a rigorous imaging geometric model with a reference band of 0.865μm imaging band. The high-precision matching points between the original imagery of the reference band and the simulated imagery of its corresponding wavelength range are obtained by using image correlation matching combined with the least squares matching algorithm.

[0101] Step 3-2: Based on the image point offset of the control points between the original image and the simulated image as the error, the median error of every eight control points in the scanning direction (image column direction) is used to generate the offset curves of the flight direction and the scanning direction.

[0102] Dense control points were extracted to represent the offset of matching points between two images. Obvious striping was observed from the scanning direction, caused by the non-uniform scanning speed of the oscillating mirror. Simultaneously, there was a significant deviation between the current geolocation data of the images and the actual geographical location of the sampling area. To reduce the impact of gross errors on the results, the median error of every eight columns of control points along the scanning direction (image column direction) was used to plot the offset curve (Δp) between the flight direction and the scanning direction. x Δp y Using a curve relating to image size, the main frequencies of the scanning angle error data in the offset curve of the scanning direction are obtained through Fast Fourier Transform. The sine function parameters are then quickly obtained, and the following scanning angle variation model is constructed, namely, the scanning non-stationary error model for the reference band:

[0103]

[0104] In the formula, θ(t) is the scanning angle, n is the number of sine functions, and ω i φ is the sample frequency. i Let a be the phase of the sine function. i Let t represent the amplitude of the sine function and t represent time.

[0105] For the scanning non-stationarity error model of the band to be registered:

[0106] Based on the band design on the camera's focal plane, the corresponding positions of the reference band and the band to be registered on the focal plane are determined, and the correction amount Δx obtained by principal point offset estimation is introduced. 01 and Δy 01 Through the initial phase offset (Δx) 01 and Δy 01 The scanning non-stationarity error model of the band to be registered is obtained. This model utilizes the initial phase offset (Δx). 01 and Δy 01 Determining the initial phase (inter-band focal plane position parameters) and fitting the scanning non-stationary error model of the band to be registered with reference to the aforementioned reference band is feasible with existing technology, therefore, it will not be described in detail.

[0107] Finally, the inter-band scanning non-stationary registration model Δθ is obtained by subtracting the reference band model θ(t1) from the band model θ(t2) to be registered.

[0108] Δθ(s)=θ(t1)-θ(t2)

[0109] In the formula, s represents the coordinates of the image in the scanning direction, and t1 and t2 represent the design time for starting scanning and imaging in the two bands, respectively.

[0110] Step 4: Using the inter-band scanning non-stationary registration model, perform pixel-by-pixel (x,y) offset correction between the remaining multispectral imaging bands and the reference band: Obtain the offset image-side coordinates (x', y');

[0111] In the formula, Δp x and Δp y It is based on It's confirmed.

[0112] Step 5: Based on the image coordinates of the image points in the image to be registered, determine the frame number of the current image point and the interval between the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0113] Specifically, this means transforming the coordinates (i,j) of the image point to be registered to the image-side coordinates (i,j) after registration based on the obtained inter-band principal point offset error parameters. tran ,j tran Then, the frame number of the current image point is determined using the load CCD design value, and the interval between the image point and the edge of the upper and lower frames is calculated.

[0114] When using a bicubic polynomial sampling kernel or a bilinear sampling kernel, the correspondence between the interval type and the intra-frame sampling kernel type is as follows:

[0115] Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 4*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 3*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 2*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel.

[0116] Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single pixel sampling kernel, that is, 2*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual pixel sampling kernel, that is, 1*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel.

[0117] Step 6: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points of the current image point in adjacent frames.

[0118] In this embodiment, if a bicubic polynomial sampling kernel is used, and the interval between the current pixel points is greater than 2 pixels; or if a bilinear sampling kernel is used, and the interval between the current pixel points is greater than 1 pixel, then no corresponding points are introduced to participate in resampling. That is, for the current pixel point, resampling is performed in the corresponding sampling kernel of the current frame at the offset-corrected pixel coordinates.

[0119] In addition, the image coordinates of the image point in the corresponding point in the adjacent frame are calculated. The calculation process is as follows:

[0120] Using initial geolocation data and based on the image coordinates (i) of the current image point tran ,j tran Within the current frame, perform linear interpolation on the geolocation data in both the flight and scan directions to obtain the latitude and longitude difference Δlat between the current pixel and its neighboring pixels in both directions. x Δlat y Δlon x Δlon y lat represents latitude, lon represents longitude, and xy corresponds to the x and y axes respectively;

[0121] Next, the object space coordinates of the current image point in the current frame are calculated using bilinear interpolation.

[0122] Using the object coordinates of the current image point The image-side coordinate correction value (Δx, Δy) is calculated with the image-side coordinates (x0, y0) of the initial image point in the adjacent frame. The image-side coordinates (x0, y0) of the initial image point in the adjacent frame are arbitrarily selected within the range of the adjacent frames.

[0123] Based on the initial image point image-space coordinates (x0, y0), the corresponding object-space coordinates lon(x0, y0) and lat(x0, y0) are calculated using bilinear interpolation. Simultaneously, the two-dimensional gradient change lon' of the object-space coordinates is calculated based on the object-space coordinates of adjacent image points. x (x0,y0),lat' x (x0,y0),lon' y (x0,y0),lat' y (x0, y0), abbreviated as lon' x ,lat' x ,lon' y ,lat' y ;

[0124] Through formula Calculate the differences in object coordinates Δlon and Δlat; then substitute the above parameters into the formula. Calculate the image-side coordinate corrections Δx and Δy;

[0125] Then, the image-side coordinates of the new iterative image point are obtained by replacing (x0, y0) with (x0+Δx, y0+Δy);

[0126] The process involves iteratively calculating each parameter according to the steps outlined above until the image-side coordinate correction value is less than 1, indicating that two image points are within one pixel. This process yields the final image-side coordinate correction value (Δx) at the end of the iteration. n ,Δy n Then, the image coordinates of the corresponding point (i) are linearly interpolated from the object-space two-dimensional gradient of the iterative image point and the given object-space coordinates. adjoin ,j adjoin ).

[0127] It should be understood that the present invention obtains the corresponding points of the current image point in adjacent frames, and resamples by introducing the corresponding points to ensure that the gray value is not contaminated by the bowtie effect.

[0128] Step 7: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging.

[0129] In this process, the grayscale value of each pixel in the image to be registered is resampled according to the steps described above.

[0130] Specifically, the sampling kernel type is adaptively determined based on the distance from the frame edge, and the same image points are also divided into the three cases mentioned above according to their position from the frame edge; when using a bicubic polynomial sampling kernel, for any image point within the sampling kernel, the weight is calculated according to the following formula based on the horizontal and vertical coordinates:

[0131]

[0132] In the formula, x represents the distance between the image point and the sampling point.

[0133] This invention is not only applicable to the current cubic polynomial sampling kernel, but also matches other sampling kernels such as the nearest neighbor method and the bilinear method.

[0134] Calculate the weighted gray values ​​for the pixel regions of the current frame and adjacent frames respectively, and then perform a weighted average:

[0135]

[0136] In the formula, g is the gray value of the resampled pixel, N1 and N2 represent the number of sampling kernel pixels in the current frame and the adjacent frame, respectively, and g i g i and g j ω represents the grayscale value of each sampling point within the two sampling kernels of the current frame and the adjacent frame.i and ω j All of these are the weights corresponding to the sampling points.

[0137] The above process continues until all pixels in the image have been resampled.

[0138] Furthermore, this embodiment of the invention preferably performs step 8: using the registered image and its corresponding precise geolocation data, a detailed search is conducted between adjacent frames based on image point coordinates and object coordinates. Specifically, focusing on each image point, it is checked whether there are identical object coordinates in the interpolation of adjacent frames, while ensuring that these coordinates do not exceed the predetermined range of the corresponding scan frame. Once the conditions are met, the grid cell coordinates are selected as corresponding points, and then the gray values ​​of these two points are extracted respectively, and a precise ratio calculation is performed to complete the accuracy verification of the resampling method.

[0139] It should be understood that in other feasible embodiments, the accuracy verification of the resampling method can be selectively performed.

[0140] Example

[0141] The following example uses the Fengyun-3 satellite as an example, such as... Figure 1 The image shows a registration error estimation plot. The vertical axis represents the color list corresponding to the registration error values, and the horizontal axis represents the registration error curve obtained by extracting the registration error of each of the 16 columns along the scanning direction using median filtering. The plot reveals obvious striping, and the median values ​​indicate constant offset between bands and registration errors due to scanning instability. All pixels in the image were resampled using the aforementioned method, resulting in the following... Figure 2 As shown in the image, the obvious stripe phenomenon has been eliminated, and the inter-band registration error has been corrected to about 0.03 pixels. Figure 3 The diagram shows a comparative analysis of various resampling methods. By selecting a large number of pixels with discontinuous gray levels between consecutive frames, the processing accuracy of various resampling methods for this type of pixel is compared, demonstrating the effectiveness of the resampling method of the present invention.

[0142] Example 2

[0143] This invention also provides a registration system based on the above registration method, comprising: a registration module, a sampling kernel determination module, a corresponding point processing module, and a resampling module.

[0144] The system comprises the following modules: a registration module for acquiring registration model parameters of multispectral images from a sweeping camera across bands, and correcting the image coordinate offset of the image points to be registered based on these parameters; a sampling kernel determination module for determining the frame number and distance from the edges of the current image point to the frame based on the offset-corrected image coordinates of the image points in the image to be registered, and then determining the intra-frame sampling kernel type of the current image point based on the interval type; a corresponding point processing module for acquiring the object-space coordinates of the current image point, and obtaining the image-space coordinates of corresponding points of the current image point in adjacent frames based on the object-space coordinates; and a resampling module for selecting a corresponding number of adjacent image points around the two image points as sampling points based on the image-space coordinates of the current image point and corresponding points, and an adaptively sized intra-frame sampling kernel, then assigning weights to the selected sampling points according to the resampling weight kernel function, and calculating the gray value of the current image point after resampling through weighted averaging; wherein, the gray value of each image point in the image to be registered is resampled according to the above process.

[0145] In some embodiments, the registration module includes a registration error analysis unit, a principal point error analysis unit, a control point extraction unit, an offset curve generation unit, a fitting unit, and a registration unit.

[0146] The registration error analysis unit is used to analyze the sources of registration errors between bands in multispectral images from a sweeping camera. These sources include at least the principal point error of the focal plane and scanning instability error. The principal point error analysis unit is used to select a reference band and extract the overall registration error between multispectral image bands using phase correlation methods, thereby obtaining the overall registration error of the nadir position. Calculate the principal point error Δx 01 and Δy 01The control point extraction unit generates a simulated image of the reference band during uniform scanning using image simulation methods. It then employs image correlation matching combined with a least-squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image within the corresponding wavelength range, serving as control points. The offset curve generation unit uses the image point offset of the control points between the original and simulated images as the error, and takes the median error of every eight columns of control points along the scanning direction to generate offset curves for the flight and scanning directions. The fitting unit, based on the characteristics of control point error variation with the scanning angle, uses the offset curves in the scanning direction and the sum of sine functions to fit the scanning angle variation model, i.e., the scanning non-stationary error model of the reference band; and introduces the principal point offset estimation errors Δx0 and Δy0 as the initial phase offset, thereby fitting the scanning non-stationary error model θ(t2) of the band to be registered. Finally, the difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered is used to obtain the inter-band scanning non-stationary registration model Δθ. The registration unit is used to perform pixel-by-pixel (x,y) offset correction between the other multispectral imaging bands and the reference band using the inter-band scanning non-stationary registration model. Obtain the offset image-side coordinates (x', y').

[0147] It should be understood that the specific implementation process of each module unit is described in the above method. This invention will not repeat it here. The above division of functional modules is only for illustrative purposes. In some embodiments, some functional modules can be merged and some functional modules can be split. Each functional module can be implemented in software, hardware, or a combination of software and hardware. The software and hardware devices include, but are not limited to, general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0148] Example 3

[0149] This invention provides a computer device, characterized in that it includes at least one or more processors and a memory storing one or more computer programs;

[0150] The processor calls the computer program to implement:

[0151] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0152] The specific method for image band registration using the imaging spectrometer is as follows:

[0153] S1: Obtain the registration model parameters of the multispectral images of the sweep camera between bands, and perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters.

[0154] S2: Based on the image-side coordinates of the offset-corrected image points in the image to be registered, determine the frame number of the current image point and the interval from the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0155] S3: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points of the current image point in adjacent frames;

[0156] S4: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive-size intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging.

[0157] In this process, the grayscale value of each pixel in the image to be registered is resampled according to steps S2-S4 above.

[0158] The specific method for estimating the registration error between image bands is as follows:

[0159] S11: Sources of registration error between bands in multispectral images based on a sweeping camera, wherein the sources of registration error include at least: principal point error of the focal plane and scanning instability error, with the following relationship:

[0160]

[0161] In the formula, Δp x and Δp y Δx represents the image square error in the flight and scanning directions, respectively. 01 and Δy 01 denoted by and respectively, represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; μ represents the magnitude of the load CCD (Charge-Coupled Device); f and Δf are the design value and change value of the principal distance, respectively; x 01 The principal point position indicates the direction of flight, and Δθ represents the change in scanning angle caused by the non-uniform scanning speed. btb This indicates the angle of the oscillating mirror scan within the sampling interval;

[0162] S12: Select the reference band and use the phase correlation method to extract the overall registration error between multispectral image bands, and then take the overall registration error of the nadir position. Calculate the principal point error Δx 01 and Δy 01 ;

[0163] S13: A simulated image of the reference band during uniform scanning of the swing mirror is generated by an image simulation method. Then, a high-precision matching point between the original image of the reference band and the simulated image of the corresponding wavelength range is obtained by using image correlation matching combined with the least squares matching algorithm, which serves as the control point.

[0164] S14: Based on the image point offset of the control points between the original image and the simulated image as the error, the median error of every eight columns of control points is taken in the scanning direction to make the offset curve of the flight direction and the scanning direction, wherein the scanning direction is the image column direction;

[0165] S15: Based on the characteristics of control point error changing with scanning angle, the scanning angle variation model is fitted using the sum of sine functions on the offset curve of the scanning direction, i.e., the scanning non-stationary error model of the reference band:

[0166]

[0167] In the formula, θ(t) is the scanning angle, n is the number of sine functions, and ω i φ is the sample frequency. i Let a be the phase of the sine function. i Let be the amplitude of the sine function, and t represent time.

[0168] S16: Introduce the principal point error Δx for principal point offset estimation 01 and Δy 01 As the initial phase offset, the scanning non-stationary error model θ(t2) of the band to be registered is then fitted. The difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered is then used to obtain the inter-band scanning non-stationary registration model Δθ.

[0169] Δθ(s)=θ(t1)-θ(t2)

[0170] In the formula, s represents the coordinates of the image in the scanning direction, and t1 and t2 represent the design time for starting scanning and imaging in the two bands, respectively.

[0171] S17: Using the aforementioned inter-band scanning non-stationary registration model, pixel-by-pixel (x,y) offset correction is performed between the remaining multispectral imaging bands and the reference band: Obtain the offset image-side coordinates (x', y').

[0172] Please refer to the explanation of the method above for the specific implementation process of each step.

[0173] It should be understood that, in the embodiments of the present invention, the processor may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. The memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of the memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0174] Example 4

[0175] This invention provides a computer-readable storage medium, characterized in that it stores a computer program, which is invoked by a processor to implement the steps of the above-described imaging spectrometer image band registration method or the above-described image band registration error estimation method.

[0176] The specific method for image band registration using the imaging spectrometer is as follows:

[0177] S1: Obtain the registration model parameters of the multispectral images of the sweep camera between bands, and perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters.

[0178] S2: Based on the image-side coordinates of the offset-corrected image points in the image to be registered, determine the frame number of the current image point and the interval from the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0179] S3: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points of the current image point in adjacent frames;

[0180] S4: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive-size intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging.

[0181] In this process, the grayscale value of each pixel in the image to be registered is resampled according to steps S2-S4 above.

[0182] The specific method for estimating the registration error between image bands is as follows:

[0183] S11: Sources of registration error between bands in multispectral images based on a sweeping camera, wherein the sources of registration error include at least: principal point error of the focal plane and scanning instability error, with the following relationship:

[0184]

[0185] In the formula, Δp x and Δp y Δx represents the image square error in the flight and scanning directions, respectively. 01 and Δy 01 denoted by and respectively, represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; μ represents the magnitude of the load CCD (Charge-Coupled Device); f and Δf are the design value and change value of the principal distance, respectively; x 01 The principal point position indicates the direction of flight, and Δθ represents the change in scanning angle caused by the non-uniform scanning speed. btb This indicates the angle of the oscillating mirror scan within the sampling interval;

[0186] S12: Select the reference band and use the phase correlation method to extract the overall registration error between multispectral image bands, and then take the overall registration error of the nadir position. Calculate the principal point error Δx 01 and Δy 01 ;

[0187] S13: A simulated image of the reference band during uniform scanning of the swing mirror is generated by an image simulation method. Then, a high-precision matching point between the original image of the reference band and the simulated image of the corresponding wavelength range is obtained by using image correlation matching combined with the least squares matching algorithm, which serves as the control point.

[0188] S14: Based on the image point offset of the control points between the original image and the simulated image as the error, the median error of every eight columns of control points is taken in the scanning direction to make the offset curve of the flight direction and the scanning direction, wherein the scanning direction is the image column direction;

[0189] S15: Based on the characteristics of control point error changing with scanning angle, the scanning angle variation model is fitted using the sum of sine functions on the offset curve of the scanning direction, i.e., the scanning non-stationary error model of the reference band:

[0190]

[0191] In the formula, θ(t) is the scanning angle, n is the number of sine functions, and ω i φ is the sample frequency. iLet a be the phase of the sine function. i Let be the amplitude of the sine function, and t represent time.

[0192] S16: Introduce the principal point error Δx for principal point offset estimation 01 and Δy 01 As the initial phase offset, the scanning non-stationary error model θ(t2) of the band to be registered is then fitted. The difference between the scanning non-stationary error model θ(t1) of the reference band and the scanning non-stationary error model θ(t2) of the band to be registered is then used to obtain the inter-band scanning non-stationary registration model Δθ.

[0193] Δθ(s)=θ(t1)-θ(t2)

[0194] In the formula, s represents the coordinates of the image in the scanning direction, and t1 and t2 represent the design time for starting scanning and imaging in the two bands, respectively.

[0195] S17: Using the aforementioned inter-band scanning non-stationary registration model, pixel-by-pixel (x,y) offset correction is performed between the remaining multispectral imaging bands and the reference band: Obtain the offset image-side coordinates (x', y').

[0196] Please refer to the explanation of the method above for the specific implementation process of each step.

[0197] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the hardware and software device described in any of the foregoing embodiments, such as the hard drive or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both internal storage units and external storage devices of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0198] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0200] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A method for image band registration in an imaging spectrometer, characterized in that: Applied to multispectral images acquired by a sweeping camera, the method includes the following steps: S1: Obtain the registration model parameters of the multispectral images of the sweep camera between bands, and perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters. S2: Based on the image-side coordinates of the offset-corrected image points in the image to be registered, determine the frame number of the current image point and the interval from the edge of the previous and next frames, and then determine the intra-frame sampling kernel type of the current image point based on the interval type. S3: Obtain the object coordinates of the current image point, and based on the object coordinates, obtain the image coordinates of the corresponding points of the current image point in adjacent frames; S4: Based on the image coordinates of the current image point and the corresponding image point, and the adaptive-size intra-frame sampling kernel, select a corresponding number of adjacent image points around the two image points as sampling points. Then, according to the resampling weight kernel function, assign weights to the selected sampling points and calculate the gray value of the current image point after resampling by weighted averaging. In this process, the grayscale value of each pixel in the image to be registered is resampled according to steps S2-S4 above. The adaptive-sized intra-frame sampling kernel is determined based on the frame number of the pixel and the interval from the edges of the preceding and following frames. Specifically, when using a bicubic polynomial sampling kernel or a bilinear sampling kernel, the correspondence between the interval type and the intra-frame sampling kernel type is as follows: Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 4*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 3*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 2*4 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel. Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single pixel sampling kernel, that is, 2*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual pixel sampling kernel, that is, 1*2 adjacent pixels are selected in the current frame to form the intra-frame sampling kernel. The resampling formula is: ； In the formula, The pixel grayscale value after resampling. These represent the number of sampled kernel pixels in the current frame and the adjacent frame, respectively. and This represents the grayscale value of each sampling point within the two sampling kernels of the current frame and the adjacent frame. and All of these are the weights corresponding to the sampling points.

2. The method according to claim 1, characterized in that: When using a bicubic polynomial sampling kernel, in step S4, the weights are calculated for the sampling points selected within their respective sampling kernels for the image point and the corresponding point. The resampling weight kernel function is as follows: ； In the formula, x The distance between an image point or a point of the same name and the sampling points within its respective sampling kernel. For weights.

3. The method according to claim 1, characterized in that: If a bicubic polynomial sampling kernel is used, and the interval between the current pixel points is greater than 2 pixels; or if a bilinear sampling kernel is used, and the interval between the current pixel points is greater than 1 pixel, then no corresponding points are introduced to participate in resampling. That is, for the current pixel point, step S3 is not executed, and in step S4, resampling is performed at the offset-corrected pixel coordinates within the corresponding sampling kernel of this frame. In addition, step S3 is performed in all cases, and the calculation process of the image coordinates of the image point in step S3 for the corresponding points in adjacent frames is as follows: Using initial geolocation data and based on the image coordinates of the current image point Within the current frame, linear interpolation is performed on the geolocation data in both the flight and scan directions to obtain the latitude and longitude differences between the current pixel and its neighboring pixels in both directions. , lat Indicates latitude, lon Indicates longitude, x and y correspond to the x and y axes respectively; Next, the object space coordinates of the current image point in the current frame are calculated using bilinear interpolation. ; Using the object coordinates of the current image point Image coordinates of initial image points in adjacent frames Calculate the image-side coordinate correction value The image-side coordinates of the initial image points in the adjacent frames These are image points arbitrarily selected within the range of adjacent frames; Based on the initial image point image square coordinates Calculate the corresponding object coordinates using bilinear interpolation. , Simultaneously, the two-dimensional gradient change of the object coordinates is calculated based on the object coordinates of adjacent image points of the initial image point. , , , abbreviation , , , ; Through formula Calculate the difference in object coordinates and Then substitute the above parameters into the formula. Calculate the image-side coordinate correction value and ; Then, through Obtain the image-side coordinates of the new iterative image points. ; The process involves iteratively calculating each parameter according to the steps outlined above until the image-side coordinate correction value is less than 1, indicating that two image points are within one pixel. This process yields the final image-side coordinate correction value at the end of the iteration. Then, the image coordinates of the corresponding points are linearly interpolated using the object-space 2D gradient of the iterative image point and the given object-space coordinates. .

4. A method for estimating image inter-band registration error based on the method described in any one of claims 1-3, characterized in that: The method for obtaining registration model parameters of multispectral images from a sweeping camera between bands and performing coordinate offset correction in step S1 includes the following steps: S11: Sources of registration error between bands in multispectral images based on a sweeping camera, wherein the sources of registration error include at least: principal point error of the focal plane and scanning instability error, with the following relationship: ； In the formula, and These represent the image-square errors in the flight and scanning directions, respectively. and These represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; Indicates the load CCD size. and These are the design value and the changed value of the principal distance, respectively. The location of the principal point indicating the direction of flight. This represents the change in scanning angle caused by a non-uniform scanning speed. This indicates the angle of the oscillating mirror scan within the sampling interval; S12: Select the reference band and use the phase correlation method to extract the overall registration error between multispectral image bands, and then take the overall registration error of the nadir position. Calculate principal point error and ; S13: A simulated image of the reference band during uniform scanning of the swing mirror is generated by an image simulation method. Then, a high-precision matching point between the original image of the reference band and the simulated image of the corresponding wavelength range is obtained by using image correlation matching combined with the least squares matching algorithm, which serves as the control point. S14: Based on the image point offset of the control points between the original image and the simulated image as the error, the median error of every eight columns of control points is taken in the scanning direction to make the offset curve of the flight direction and the scanning direction, wherein the scanning direction is the image column direction; S15: Based on the characteristics of control point error changing with scanning angle, the scanning angle variation model is fitted using the sum of sine functions on the offset curve of the scanning direction, i.e., the scanning non-stationary error model of the reference band: ； In the formula, Let n be the scanning angle, and n be the number of sine functions. For sample frequency, The phase of the sine function. Let be the amplitude of the sine function. t Indicates time; S16: Introduce principal point error for principal point offset estimation and The initial phase offset is then used to fit the scanning non-stationarity error model of the band to be registered. Then use the scanning non-stationarity error model of the reference band. Scanning non-stationarity error model with the band to be registered The difference yields the inter-band scanning non-stationary registration model. : ； In the formula, This indicates the coordinates of the image in the scanning direction. and These represent the design time for starting scanning and imaging in the two bands, respectively. S17: Using the aforementioned inter-band scanning non-stationary registration model, perform pixel-by-pixel registration between the remaining multispectral imaging bands and the reference band. Offset correction: Obtain the offset image coordinates ; In the formula, and It is based on The parameters of the registration model are determined.

5. A registration system based on the method of any one of claims 1-3, characterized in that, include: The registration module is used to obtain the registration model parameters of the multispectral images of the sweep camera between bands, and to perform image-side coordinate offset correction on the image points of the image to be registered based on the registration model parameters. The sampling kernel determination module is used to determine the frame number of the current image point and the interval from the edge of the previous and next frames based on the image-side coordinates of the offset-corrected image points in the image to be registered, and then determine the intra-frame sampling kernel type of the current image point based on the interval type. The corresponding point processing module is used to obtain the object-space coordinates of the current image point and, based on the object-space coordinates, obtain the image-space coordinates of corresponding points of the current image point in adjacent frames; The resampling module is used to select a corresponding number of neighboring image points around the current image point and the corresponding image point as sampling points based on the image coordinates of the current image point and the intra-frame sampling kernel of adaptive size. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the gray value of the current image point after resampling is obtained by weighted average calculation. In this process, the grayscale value of each pixel in the image to be registered is resampled according to the above procedure.

6. The system according to claim 5, characterized in that: The registration module includes: The registration error analysis unit is used to analyze the sources of registration errors between bands in multispectral images from a sweeping camera. These sources include at least: principal point error of the focal plane and scanning instability error, with the following relationship: ； In the formula, and These represent the image-square errors in the flight and scanning directions, respectively. and These represent the principal point offset of the band to be registered on the focal plane, i.e., the principal point error; Indicates the load CCD size. and These are the design value and the changed value of the principal distance, respectively. The location of the principal point indicating the direction of flight. This represents the change in scanning angle caused by a non-uniform scanning speed. This indicates the angle of the oscillating mirror scan within the sampling interval; The principal point error analysis unit is used to select a reference band and extract the overall registration error between multispectral image bands using the phase correlation method, and then obtain the overall registration error of the nadir position. Calculate principal point error and ; The control point extraction unit is used to generate a simulated image of the reference band when the mirror is scanning at a constant speed using an image simulation method. Then, it uses an image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range, which are used as control points. The offset curve generation unit is used to generate offset curves for the flight direction and the scanning direction by taking the median error of the control points between the original image and the simulated image, based on the image point offset of the control points between the original image and the simulated image. The scanning direction is the image column direction. The fitting unit is used to fit the scanning angle variation model based on the characteristics of control point error changing with scanning angle, using the offset curve of the scanning direction and the sum of sine functions, i.e., the scanning non-stationary error model of the reference band; and to introduce principal point offset estimation for principal point error. and The initial phase offset is then used to fit the scanning non-stationarity error model of the band to be registered. Then use the scanning non-stationarity error model of the reference band. Scanning non-stationarity error model with the band to be registered The difference yields the inter-band scanning non-stationary registration model. : The registration unit is used to perform pixel-by-pixel registration between the remaining multispectral imaging bands and the reference band using the inter-band scanning non-stationary registration model. Offset correction: Obtain the offset image coordinates .

7. A computer device, characterized in that: At least include: One or more processors; A memory that stores one or more computer programs; The processor calls the computer program to implement: The steps of the image band registration method of an imaging spectrometer according to any one of claims 1-3 or the image band registration error estimation method according to any one of claims 4.

8. A computer-readable storage medium, characterized in that: The computer program is stored and is invoked by the processor to implement: The steps of the image band registration method of an imaging spectrometer according to any one of claims 1-3 or the image band registration error estimation method according to any one of claims 4.

Images