Image waveband registration method and system of imaging spectrometer

Abstract

The invention discloses an imaging spectrometer image wave band registration method and system, and the method comprises the steps: extracting registration error initial values between other multispectral imaging wave bands and a reference wave band through a phase correlation method, and taking a registration error of a sub-satellite point position as a principal point error; based on the initial value of the rigorous imaging model, generating a simulation image during constant-speed scanning of the oscillating mirror through an image simulation method so as to establish a scanning instability error model; and finally, correcting an initial value of an inter-band registration error by fusing principal point offset and a scanning error, and on this basis, performing object space coordinate interpolation and homonymy point image space coordinate back calculation by adopting a dynamic frame edge interval detection mechanism and combining geographic positioning data. A homonymy point and a resampling weight kernel function are innovatively introduced to realize gray value weighted fusion, and image global image point error correction is completed through iterative processing. According to the technical scheme, the inter-band registration precision is effectively improved, and the method is suitable for space-borne remote sensing image processing of sweep imaging.

Description

Technical Field

[0001] The present invention belongs to the field of satellite image processing technology, and relates to image preprocessing of meteorological, oceanic and other satellite imaging systems, and specifically to an imaging spectrometer image band registration method and system based on a satellite-borne swing-scanning camera. Background Art

[0002] Through their unique swing-scanning method, pendulum-scanning cameras can capture large-area ground images across multiple spectral bands, providing a rich resource of multispectral image data for numerous applications such as topographic mapping, environmental monitoring, and agricultural remote sensing. While multispectral images acquired by spaceborne pendulum-scanning cameras contain rich and valuable information, the images of each spectral band are misaligned due to various factors, including geometric and radiometric differences, making them unsuitable for direct use in comprehensive analysis and quantitative applications.

[0003] Due to the unique oscillating scanning method of oscillating cameras, the image acquisition process produces a phenomenon known as the "bow-tie effect." This bow-tie effect arises from the change in sensor perspective caused by the oscillation of the scanning platform during the scanning process, resulting in geometric distortion between adjacent scanned strips. Specifically, the ground sampling areas in adjacent frames are affected by factors such as the scanning angle, the camera's oscillation trajectory, and the sensor's dynamic response. This causes overlap at the ends of the scan. This means that adjacent frames will contain the same ground area, resulting in significant discontinuities in the grayscale values between frames. Inter-image registration requires consistent grayscale texture. This grayscale discontinuity not only affects the radiometric consistency of the image but also significantly reduces the accuracy of inter-band registration, making it a challenge for oscillating imagery in multispectral data fusion and quantitative analysis. Therefore, effectively addressing this phenomenon and minimizing the impact of the bow-tie effect on inter-band registration is key to achieving high-precision multispectral image processing using oscillating cameras. Summary of the Invention

[0004] The present invention aims to address or mitigate the impact of the bow-tie effect on inter-band registration in the aforementioned prior art, avoiding significant discontinuities in grayscale values between adjacent frames. This approach improves the accuracy of image registration for spaceborne oscillating cameras, lays the foundation for high-precision registration of multispectral images from spaceborne oscillating cameras, and eliminates spatial inconsistencies between multispectral image bands. The present invention provides a method and system for band registration of imaging spectrometer images, which effectively improves the accuracy of multispectral registration for oscillating cameras and minimizes spatial inconsistencies and differences between multispectral images.

[0005] To this end, the present invention provides the following technical solutions:

[0006] The present invention provides an imaging spectrometer image band registration method, which is applied to multispectral images acquired by a swing-scanning camera. The method comprises the following steps:

[0007] S1: Acquire registration model parameters between bands of a multispectral image of a swing-scanning camera, and perform image coordinate offset correction on image points of the image to be registered based on the registration model parameters;

[0008] S2: Based on the image coordinates of the offset-corrected image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type;

[0009] S3: Acquire the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates;

[0010] S4: Based on the image coordinates of the current image point and the point with the same name and the adaptive size of the intra-frame sampling kernel, a corresponding number of adjacent image points are selected around the two image points as sampling points. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the grayscale value of the current image point after resampling is calculated by weighted average;

[0011] The grayscale value of each pixel in the image to be registered is resampled according to the above steps S2-S4.

[0012] Preferably, the adaptive size intra-frame sampling kernel is determined based on the frame number of the pixel and the interval from the upper and lower frame edges. When a bicubic polynomial sampling kernel or a bilinear sampling kernel is used, the correspondence between the interval type and the intra-frame sampling kernel type is as follows:

[0013] Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-image point sampling kernel, that is, 4*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 3*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 2*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel;

[0014] Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 2*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 1*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel;

[0015] The resampling formula is:

[0016]

[0017] Where g is the grayscale value of the pixel after resampling, N1 and N2 represent the number of sampling kernel pixels in the current frame and the adjacent frame respectively, and g is the grayscale value of the pixel after resampling. i and g j Represents the grayscale value of each sampling point in the two sampling kernels of the current frame and the adjacent frame, ω i and ω j are the weights corresponding to the sampling points.

[0018] Preferably, when a bicubic polynomial sampling kernel is used, weights are calculated for the sampling points selected for the image points and the points with the same name within their respective sampling kernels in step S4. The resampling weight kernel function is as follows:

[0019]

[0020] Where x is the distance between the image point or the same-name point and the sampling point in the respective sampling kernel, and ω(x) is the weight.

[0021] Preferably, if a bicubic polynomial sampling kernel is used and the interval corresponding to the current image point is greater than 2 pixels; or if a bilinear sampling kernel is used and the interval corresponding to the current image point is greater than 1 pixel, no points with the same name are introduced to participate in resampling, that is, step S3 is not performed for the current image point, and in step S4, resampling is performed within the sampling kernel corresponding to the current frame at the offset-corrected pixel coordinates;

[0022] In addition, step S3 is executed, and the image-side coordinates of the same-name points in adjacent frames in step S3 are calculated as follows:

[0023] Using the initial geolocation data and according to the image coordinates of the current image point (i tran ,j tran ) Perform linear interpolation on the geolocation data in the current frame range in the flight and scanning directions, and obtain the longitude and latitude difference Δlat between the current pixel and the adjacent pixel in the flight and scanning directions respectively. x , Δlat y , Δlon x , Δlon y , lat represents latitude, lon represents longitude, and xy corresponds to the xy axis respectively;

[0024] Next, the object coordinates of the current image point in this frame are calculated using bilinear interpolation

[0025] Using the object coordinates of the current image point Calculate the image space coordinate correction value (Δx, Δy) using the image space coordinates (x0, y0) of the initial image point in the adjacent frame, where the image space coordinates (x0, y0) of the initial image point in the adjacent frame are randomly selected within the range of the adjacent frame (generally, the upper left corner of the adjacent frame is selected);

[0026] According to the image coordinates of the initial image point (x0, y0), the corresponding object coordinates lon(x0, y0) and lat(x0, y0) are calculated using bilinear interpolation. At the same time, the two-dimensional gradient change lon' of the object coordinates is calculated based on the object coordinates of the adjacent image points of the initial image point. x (x0,y0),lat' x (x0,y0),lon' y (x0,y0),lat' y (x0,y0), abbreviated as lon' x ,lat' x ,lon' y ,lat' y ;

[0027] By formula Calculate the object coordinate differences Δlon and Δlat; then substitute the above parameters into the formula Calculate the image coordinate correction values Δx and Δy;

[0028] Then, the image coordinates of the new iterative image point are obtained by (x0+Δx,y0+Δy) instead of (x0,y0);

[0029] Here, each parameter is calculated iteratively according to the above steps until the image coordinate correction value is less than 1, that is, the two image points are in one pixel, and the image coordinate correction value (Δx n ,Δy n ), and then the image coordinates of the same name point are linearly interpolated from the object-space two-dimensional gradient of the iterative image point and the given object-space coordinates (i adjoin ,j adjoin ). Then, the image coordinates of the current image point and the image coordinates of the same-name point are finally transmitted and aligned.

[0030] In addition, the present invention provides a method for estimating image band registration errors based on the above-mentioned imaging spectrometer image band registration method, which is used to obtain the registration model parameters between the bands of the multispectral image of the swing-scanning camera in step S1 and perform coordinate offset correction. The method includes the following steps:

[0031] S11: The registration error sources of the multispectral image between the bands based on the swing-scanning camera include at least: the principal point error of the focal plane and the scanning instability error, and the relationship is:

[0032]

[0033] Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the payload CCD, f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval;

[0034] S12: Select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ;

[0035] S13: Generate a simulated image of the reference band when the oscillating mirror scans at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points;

[0036] S14: Based on the image point offset between the original image and the simulated image as the error, the median value of the error of each eight columns of control points in the scanning direction is taken to generate an offset curve in the flight direction and the scanning direction, where the scanning direction is the image column direction;

[0037] S15: Based on the characteristics of the control point error changing with the scanning angle, the offset curve of the scanning direction is used to fit the scanning angle change model using the sum of sine functions, that is, the scanning instability error model of the reference band:

[0038]

[0039] Where θ(t) is the scanning angle, n is the number of sine functions, ω i is the sample frequency, φ i is the phase of the sine function, a i is the amplitude of the sine function, t represents time;

[0040] S16: Introducing the principal point error Δx estimated by principal point offset 01 and Δy 01As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ:

[0041] Δθ(s)=θ(t1)-θ(t2)

[0042] Where s represents the coordinate of the image in the scanning direction, t1 and t2 represent the design time for the two bands to start scanning imaging respectively;

[0043] S17: Using the inter-band scanning unsteady registration model, perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band: Get the image coordinates (x', y') after the offset;

[0044] Where Δp x and Δp y is based on Determined, that is, the registration model parameters.

[0045] The technical solution of the present invention further provides a registration system based on the above registration method, comprising:

[0046] A registration module is used to obtain registration model parameters between bands of the multispectral image of the swing-scanning camera, and perform image coordinate offset correction on the image points of the image to be registered based on the registration model parameters;

[0047] The sampling kernel determination module is used to determine the frame number of the current image point and the interval from the upper and lower frame edges based on the image coordinates of the offset-corrected image point in the image to be registered, and then determine the intra-frame sampling kernel type of the current image point based on the interval type;

[0048] A homonymous point processing module is used to obtain the object-space coordinates of the current image point and obtain the image-space coordinates of the homonymous points of the current image point in adjacent frames based on the object-space coordinates;

[0049] The resampling module is used to select a corresponding number of adjacent image points around the current image point and the same-name point based on the image coordinates and the adaptively sized intra-frame sampling kernel. Then, based on the resampling weight kernel function, weights are assigned to the selected sampling points. The grayscale value of the current image point after resampling is calculated by weighted average.

[0050] The grayscale value of each pixel in the image to be registered is resampled according to the above process.

[0051] Preferably, the registration module of the system comprises:

[0052] The registration error analysis unit is used to analyze the registration error sources between bands of the multispectral image based on the swing-scanning camera. The registration error sources include at least the principal point error of the focal plane and the scanning instability error, and the relationship is:

[0053]

[0054] Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the payload CCD, f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval;

[0055] The principal point error analysis unit is used to select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ;

[0056] A control point extraction unit is used to generate a simulated image of the reference band when the oscillating mirror is scanning at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points;

[0057] an offset curve generating unit, configured to generate an offset curve in a flight direction and a scanning direction by taking the median of the errors of each eight columns of control points in a scanning direction based on the image point offset between the original image and the simulated image, wherein the scanning direction is the image column direction;

[0058] The fitting unit is used to fit the scanning angle variation model based on the characteristics of the control point error changing with the scanning angle, using the offset curve of the scanning direction and the sum of sine functions, that is, the scanning instability error model of the reference band; and introduce the main point error Δx estimated by the main point offset. 01 and Δy 01 As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ:

[0059] The registration unit is used to perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band using the inter-band scanning instability registration model: Get the offset image coordinates (x', y').

[0060] The technical solution of the present invention further provides a computer device comprising at least: one or more processors and a memory storing one or more computer programs;

[0061] The processor calls the computer program to implement:

[0062] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0063] The technical solution of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is called by a processor to implement:

[0064] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0065] Beneficial effects

[0066] Compared with the existing method, the advantages of the present invention are:

[0067] 1. The technical solution of the present invention obtains the same-name points of the current image point in adjacent frames. By introducing geographic positioning data and introducing same-name points for resampling, it solves the grayscale value discontinuity problem caused by the bow-tie effect mentioned in the background technology and the grayscale value error introduced during the resampling process, ensuring that the grayscale value is not contaminated by the bow-tie effect.

[0068] 2. In a further preferred embodiment of the present invention, the size of the sampling kernel within a frame is adaptively determined based on the frame number of the image point and the distance from the upper and lower frame edges. This can effectively eliminate pixels with discontinuous grayscale values introduced during the sampling kernel selection process, thereby reducing grayscale value contamination.

[0069] 3. Through precise image matching, error calculation and analysis based on a rigorous imaging geometry model, the registration error between image bands can be effectively estimated. The resampling method that takes into account the swing-scan imaging mechanism can improve the registration accuracy and make the position of the image between bands more accurate, providing more reliable data support for subsequent applications such as image fusion and target recognition.

[0070] 4. The method of the present invention fully considers the imaging characteristics and error sources of satellite-borne swing-scan cameras, can adapt to different imaging conditions and data characteristics, has strong versatility and adaptability, and can be widely used in the processing of images from various satellite-borne swing-scan cameras. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 is a schematic diagram of registration error estimation according to an embodiment of the present invention;

[0072] Figure 2 is a schematic diagram of the registration error between images after registration according to the present invention;

[0073] Figure 3 Schematic diagram of resampling accuracy test performed by the present invention, wherein a, b, c, and d correspond to the present invention, the existing cubic polynomial interpolation method, the existing bilinear interpolation method, and the resampling method comparison effect diagram respectively;

[0074] Figure 4 It is a schematic diagram of a process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The present invention provides an imaging spectrometer image band registration method and system, which can effectively solve / reduce the problem of the bow-tie effect affecting inter-band registration in the above-mentioned prior art, avoid the phenomenon of significant discontinuity in the grayscale values between adjacent frames of the image, and lay the foundation for improving the image registration accuracy of satellite-borne swing-scanning cameras and realizing high-precision registration between multispectral images of satellite-borne swing-scanning cameras. Among them, one of the cores of the present invention is to resample the multispectral images acquired by the swing-scanning camera, that is, by obtaining the same-name points of the current image point in the adjacent frames, and resampling by introducing the same-name points to ensure that the grayscale values are not contaminated by the bow-tie effect. Another core of the present invention is to fully consider the sources of registration errors between bands of the multispectral images of the swing-scanning camera, specifically from the principal point error of the focal plane and the scanning instability error, so as to construct an inter-band scanning instability registration model to eliminate the images with the above-mentioned registration errors and improve the registration accuracy.

[0076] The technical ideas of the band registration method for imaging spectrometer images acquired by a swing-scanning camera are as follows:

[0077] S1: Obtain the registration model parameters of the multispectral image of the swing-scanning camera (belonging to the imaging spectrometer) between the bands, and perform image coordinate offset correction on the image points to be registered based on the registration model parameters;

[0078] S2: Based on the image coordinates of the offset-corrected image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type;

[0079] S3: Obtain the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates;

[0080] S4: Based on the image coordinates of the current image point and the point with the same name and the adaptive size of the intra-frame sampling kernel, a corresponding number of adjacent image points are selected around the two image points as sampling points. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the grayscale value of the current image point after resampling is calculated by weighted average;

[0081] The grayscale value of each pixel in the image to be registered is resampled according to the above steps S2-S4.

[0082] In addition, the method for estimating the inter-band registration error of images acquired by the swing-scanning camera is used to obtain the registration model parameters and coordinate offset correction between the bands of the multispectral image acquired by the swing-scanning camera in step S1. The technical ideas of the registration error estimation method are as follows:

[0083] Based on the sources of registration errors between bands of multispectral images of a swing-scanning camera, an inter-band scanning instability registration model is constructed. The sources of registration errors include at least the principal point error of the focal plane and the scanning instability error.

[0084] Select a reference band, and construct a scanning instability error model for the reference band and a scanning instability error model for the band to be registered;

[0085] Finally, the difference between the scanning instability error model of the reference band and the scanning instability error model of the band to be registered is obtained to obtain the inter-band scanning instability registration model, and then substitute Δp is calculated in the model x and Δp y , and then perform pixel-by-pixel (x,y) offset correction between the remaining multispectral imaging bands and the reference band:

[0086] The present invention will be further described below with reference to the embodiments.

[0087] In the embodiment of the present invention, it is preferred to first estimate the inter-band registration error of the multispectral image and then resample the multispectral image. Therefore, the specific implementation process is as follows:

[0088] Step 1: Determine the sources of registration errors and construct an inter-band scanning instability registration model.

[0089] This embodiment uses Level 1B (preliminary processed) image data from a pendulum-scanning imaging payload. A rigorous imaging geometry model is constructed based on payload design parameters to analyze the sources of inter-band registration errors, which are primarily principal point errors and scanning instability errors.

[0090] The detailed analysis is as follows: The camera simultaneously samples light through an oscillating mirror, transmits it through the optical path system to the focal plane system, and projects it onto detectors in different spectral channels. During this process, the optical path system exerts a consistent influence. Therefore, the inter-band registration error mainly originates from the focal plane, including at least the principal point error. In addition, the non-uniform scanning speed of the oscillating mirror causes the light received by different bands to shift depending on the scanning angle, which manifests as follows:

[0091]

[0092] Where Δp x and Δp y Denotes the image error Δx in the flight and scanning directions respectively 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, μ represents the size of the load CCD (Charge-Coupled Device), f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle scanned by the oscillating mirror during the sampling interval.

[0093] Step 2: Select a reference band and extract the overall registration error between the reference band and the remaining multispectral imaging bands in the multispectral image in the flight and scanning directions.

[0094] In the embodiment of the present invention, a high-precision image matching phase correlation method is preferably used to extract the overall registration error between the image flight and scanning upward bands. A near-infrared or visible light band is also preferably used as a reference band.

[0095] Specifically, a swing-scan camera is usually carried on a platform such as a satellite or an airplane, and forms a multispectral image under the joint action of the flight direction (along the track direction) and the scanning direction (perpendicular to the track direction). The present invention preferably selects multispectral image data taken by a swing-scan camera in a cloudless area with a large land area. For visible light and near-infrared bands, the registration error between the reference band and the band to be registered is calculated by the phase correlation method based on the corresponding band image of the swing-scan camera; in order to achieve high-precision registration of multimodal images and avoid the decrease in registration accuracy caused by inconsistent radiation between multimodal images, for other infrared bands, the registration error between the reference band and the band to be registered is calculated by the phase correlation method. This section adopts the image simulation method, such as the thermal infrared band. The Landsat8 image is used as the base map data for simulation to obtain the simulated image of the thermal infrared band. The corresponding grayscale value is calculated pixel by pixel through the geographic positioning data and the rigorous imaging geometric model. The phase correlation is used to extract the registration error between the simulated image of the thermal infrared band and the original thermal infrared band image. Among them, the thermal infrared band image generated by simulation using precise geographic positioning data can be regarded as an image that is consistent with the benchmark band. This can effectively avoid the influence of multimodal matching on the registration accuracy.

[0096] In this embodiment, the principal point error is inferred from the overall registration error of the nadir point (the nadir point refers to the ground point directly below the satellite orbit) to construct the principal point error parameter Δx 01 and Δy 01 .

[0097] Step 3: Construct the scanning instability error model of the reference band and the scanning instability error model of the band to be registered, and then use the difference between the scanning instability error model θ(t1) of the reference band and the scanning instability error model θ(t2) of the band to be registered to obtain the inter-band scanning instability registration model Δθ.

[0098] Take the scanning instability error model of the reference band as an example:

[0099] Step 3-1: Generate a simulated image of the reference band when the oscillating mirror scans at a uniform speed through an image simulation method, and then use image correlation matching combined with the least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points.

[0100] This embodiment intends to use the Landsat-8 global 250M resolution image as the base map data, perform pixel-by-pixel simulation according to the initial value of the rigorous imaging geometry model with the reference band being the 0.865μm imaging band, and use image correlation matching combined with the least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of its corresponding wavelength range.

[0101] In step 3-2, based on the image point offset between the original image and the simulated image as the error, the median of the error of every eight columns of control points in the scanning direction (image column direction) is taken to generate the offset curves in the flight direction and the scanning direction.

[0102] Among them, dense control points are extracted to represent the offset of matching points between the two images. There is an obvious stripe phenomenon in the scanning direction of the image, which is caused by the non-uniform scanning of the swing mirror. At the same time, there is a significant deviation between the current geographic positioning data of the image and the actual geographic location of the sampling area. In order to reduce the influence of gross errors on the results, the median error of each eight columns of control points in the scanning direction (image column direction) is taken to make the offset curve (Δp x , Δp y The main frequency of the scanning angle error data in the offset curve of the scanning direction is obtained by fast Fourier transform, and the sine function parameters are quickly obtained to construct the following scanning angle change model, that is, the scanning instability error model of the reference band:

[0103]

[0104] Where θ(t) is the scanning angle, n is the number of sine functions, ω i is the sample frequency, φ i is the phase of the sine function, a i is the amplitude of the sine function, and t represents time.

[0105] Scanning instability error model for the band to be registered:

[0106] According to the band design on the camera focal plane, the corresponding positions of the reference band and the band to be registered on the focal plane are determined, and the correction Δx obtained by estimating the principal point offset is introduced. 01 and Δy 01 , by shifting the initial phase (Δx 01 and Δy 01 ) to obtain the scanning unsteady error model of the band to be registered. Among them, the initial phase offset (Δx 01 and Δy 01 ) determines the initial phase (inter-band focal plane position parameter), and refers to the scanning instability error model of the aforementioned reference band. Fitting the scanning instability error model of the band to be aligned is achievable in the existing technology, so it will not be described in detail.

[0107] Finally, the inter-band scanning instability registration model Δθ is obtained by taking the difference between the reference band model θ(t1) and the band model to be registered θ(t2).

[0108] Δθ(s)=θ(t1)-θ(t2)

[0109] Where s represents the coordinate of the image in the scanning direction, and t1 and t2 represent the design time for the two bands to start scanning imaging, respectively.

[0110] Step 4: Use the inter-band scanning unsteady registration model to perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band: Get the image coordinates (x', y') after the offset;

[0111] Where Δp x and Δp y is based on Sure.

[0112] Step 5: Based on the image coordinates of the image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type.

[0113] Specifically, the coordinates of the image point to be registered (i, j) are converted to the image coordinates after registration (i tran ,j tran ); then use the load CCD design value to determine the frame number of the current image point and calculate the distance between the image point and the upper and lower frame edges.

[0114] When using a bicubic polynomial sampling kernel or a bilinear sampling kernel, the correspondence between the interval type and the intra-frame sampling kernel type is as follows:

[0115] Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-image point sampling kernel, that is, 4*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 3*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 2*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel;

[0116] Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 2*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 1*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel;

[0117] Step 6: Obtain the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates.

[0118] In this embodiment, if a bicubic polynomial sampling kernel is used and the interval corresponding to the current image point is greater than 2 pixels; or if a bilinear sampling kernel is used and the interval corresponding to the current image point is greater than 1 pixel, no homonymous points are introduced for resampling. That is, resampling is performed for the current image point at the offset-corrected pixel coordinates within the sampling kernel corresponding to the current frame.

[0119] In addition, the image coordinates of the same-name points in adjacent frames are calculated. The calculation process is as follows:

[0120] Using the initial geolocation data and according to the image coordinates of the current image point (i tran ,j tran ) Perform linear interpolation on the geolocation data in the current frame range in the flight and scanning directions, and obtain the longitude and latitude difference Δlat between the current pixel and the adjacent pixel in the flight and scanning directions respectively. x , Δlat y , Δlon x , Δlon y , lat represents latitude, lon represents longitude, and xy corresponds to the xy axis respectively;

[0121] Next, the object coordinates of the current image point in this frame are calculated using bilinear interpolation

[0122] Using the object coordinates of the current image point Calculating an image space coordinate correction value (Δx, Δy) from the image space coordinates (x0, y0) of the initial image point in the adjacent frame, wherein the image space coordinates (x0, y0) of the initial image point in the adjacent frame are randomly selected image points within the range of the adjacent frames;

[0123] According to the image coordinates of the initial image point (x0, y0), the corresponding object coordinates lon(x0, y0) and lat(x0, y0) are calculated using bilinear interpolation. At the same time, the two-dimensional gradient change lon' of the object coordinates is calculated based on the object coordinates of the adjacent image points of the initial image point. x (x0,y0),lat' x (x0,y0),lon' y (x0,y0),lat' y (x0,y0), abbreviated as lon' x ,lat' x ,lon' y ,lat' y ;

[0124] By formula Calculate the object coordinate differences Δlon and Δlat; then substitute the above parameters into the formula Calculate the image coordinate correction values Δx and Δy;

[0125] Then, the image coordinates of the new iterative image point are obtained by (x0+Δx,y0+Δy) instead of (x0,y0);

[0126] Here, each parameter is calculated iteratively according to the above steps until the image coordinate correction value is less than 1, that is, the two image points are in one pixel, and the image coordinate correction value (Δx n ,Δy n ), and then the image coordinates of the same name point are linearly interpolated from the object-space two-dimensional gradient of the iterative image point and the given object-space coordinates (i adjoin ,j adjoin ).

[0127] It should be understood that the technology of the present invention obtains the same-name points of the current image point in adjacent frames, and resamples by introducing the same-name points to ensure that the grayscale value is not contaminated by the bow-tie effect.

[0128] Step 7: Based on the image-space coordinates of the current pixel and the point with the same name and the adaptively sized intra-frame sampling kernel, select a corresponding number of adjacent pixels around the two pixels as sampling points. Then, based on the resampling weight kernel function, assign weights to the selected sampling points, and calculate the grayscale value of the current pixel after resampling through weighted average calculation.

[0129] The grayscale value of each pixel in the image to be registered is resampled according to the above steps.

[0130] Specifically, the sampling kernel type is adaptively determined based on the distance from the frame edge. Pixels with the same name are also divided into the above three situations according to their distance from the frame edge. When a bicubic polynomial sampling kernel is used, the weight of any pixel within the sampling kernel is calculated based on the horizontal and vertical coordinates according to the following formula:

[0131]

[0132] Where x represents the distance between the image point and the sampling point

[0133] The present invention is not only applicable to the current cubic polynomial sampling kernel, but is also applicable to other sampling kernels such as the nearest neighbor method and the bilinear method.

[0134] Calculate the weighted grayscale values of the current frame and the adjacent frame pixel areas and perform weighted averaging:

[0135]

[0136] Where g is the grayscale value of the pixel after resampling, N1 and N2 represent the number of sampling kernel pixels in the current frame and the adjacent frame respectively, and g is the grayscale value of the pixel after resampling. i g i and g j Represents the grayscale value of each sampling point in the two sampling kernels of the current frame and the adjacent frame, ωi and ω j are the weights corresponding to the sampling points.

[0137] Follow the above process until all image points in the image are resampled.

[0138] Furthermore, embodiments of the present invention preferably perform step 8: utilizing the registered image and its corresponding precise geolocation data, a detailed search is performed between adjacent frames based on image point coordinates and object-space coordinates. Specifically, each image point is examined for identical object-space coordinates in the interpolated values of adjacent frames, ensuring that these coordinates do not exceed the specified range of the corresponding scanned frame. Once these conditions are met, the grid cell coordinates are selected as the same-name point, and the grayscale values of these two points are extracted separately, and a precise ratio calculation is performed to verify the accuracy of the resampling method.

[0139] It should be understood that in other feasible embodiments, the accuracy verification of the resampling method may be selectively performed.

[0140] Examples

[0141] The following examples take the Fengyun-3 satellite as an example. Figure 1 The registration error estimation diagram is shown in Figure 1. The ordinate is the color list corresponding to the registration error value, and the abscissa is the registration error curve obtained by extracting the registration error of every 16 columns in the scanning direction using median filtering. From the figure, we can see that there are obvious stripes, and the constant offset between bands and the registration error of unstable scanning can be seen in the median value representation. According to the above method, all the image points in the image are resampled to obtain the following Figure 2 As can be seen from the image shown, the obvious stripe phenomenon has been eliminated and the inter-band registration error has been corrected to about 0.03 pixels. Figure 3 This is a comparative analysis chart of various resampling methods. By selecting a large number of pixels with discontinuous grayscale between consecutive frames, the processing accuracy levels of various resampling methods for these pixels are compared, which illustrates the effectiveness of the resampling method of the present invention.

[0142] Example 2

[0143] An embodiment of the present invention further provides a registration system based on the above registration method, comprising: a registration module, a sampling kernel determination module, a homonymous point processing module, and a resampling module.

[0144] Among them, the registration module is used to obtain the registration model parameters of the multispectral image of the swing-scanning camera between bands, and perform image coordinate offset correction on the image points of the image to be registered based on the registration model parameters; the sampling kernel determination module is used to determine the frame number of the current image point and the interval from the upper and lower frame edges based on the image coordinates of the offset-corrected image point in the image to be registered, and then determine the intra-frame sampling kernel type of the current image point based on the interval type; the homonymous point processing module is used to obtain the object coordinates of the current image point, and obtain the image coordinates of the homonymous point of the current image point in the adjacent frame based on the object coordinates; the resampling module is used to select a corresponding number of adjacent image points around the two image points according to the image coordinates of the current image point and the homonymous point and the adaptively sized intra-frame sampling kernel, and then assign weights to the selected sampling points according to the resampling weight kernel function, and obtain the grayscale value of the resampled current image point by weighted average calculation; wherein, the grayscale value of each image point in the image to be registered is resampled according to the above process.

[0145] In some embodiments, the registration module includes a registration error analysis unit, a principal point error analysis unit, a control point extraction unit, an offset curve generation unit, a fitting unit, and a registration unit.

[0146] Among them, the registration error analysis unit is used to analyze the registration error sources between bands of the multispectral image based on the swing-scanning camera. The registration error sources include at least: the principal point error of the focal plane and the scanning instability error. The principal point error analysis unit is used to select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01The control point extraction unit is used to generate a simulated image of the reference band when the oscillating mirror is scanning at a constant speed through an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points. The offset curve generation unit is used to use the image point offset between the original image and the simulated image as the error, and take the median of the error of each eight columns of control points in the scanning direction to generate an offset curve in the flight direction and scanning direction. The fitting unit is used to fit the scanning angle variation model, i.e., the scanning instability error model of the reference band, using the offset curve in the scanning direction and the sum of sine functions based on the characteristics of the control point error varying with the scanning angle. The principal point errors Δx0 and Δy0 estimated from the principal point offset are introduced as the initial phase offset to fit the scanning instability error model θ(t2) of the band to be registered. The scanning instability error model θ(t1) of the reference band is then subtracted from the scanning instability error model θ(t2) of the band to be registered to obtain the inter-band scanning instability registration model Δθ. The registration unit is used to perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band using the inter-band scanning instability registration model: Get the offset image coordinates (x', y').

[0147] It should be understood that the specific implementation process of each module unit please refer to the above method content, the present invention will not go into details here, and the division of the above functional modules is only for example illustration. In some embodiments, some functional modules can be merged, and some functional modules can be split. Each functional module can be implemented in software or hardware or a combination of software and hardware. Among them, the software and hardware equipment includes but is not limited to general-purpose computer equipment, programmable gate arrays, digital signal processors, microprocessors and their corresponding programming or burning software.

[0148] Example 3

[0149] An embodiment of the present invention provides a computer device, characterized in that it comprises at least: one or more processors and a memory storing one or more computer programs;

[0150] The processor calls the computer program to implement:

[0151] The steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0152] Among them, the specific implementation method of imaging spectrometer image band registration is:

[0153] S1: Acquire registration model parameters between bands of a multispectral image of a swing-scanning camera, and perform image coordinate offset correction on image points of the image to be registered based on the registration model parameters;

[0154] S2: Based on the image coordinates of the offset-corrected image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type;

[0155] S3: Acquire the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates;

[0156] S4: Based on the image coordinates of the current image point and the point with the same name and the adaptive size of the intra-frame sampling kernel, a corresponding number of adjacent image points are selected around the two image points as sampling points. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the grayscale value of the current image point after resampling is calculated by weighted average;

[0157] The grayscale value of each pixel in the image to be registered is resampled according to the above steps S2-S4.

[0158] Among them, the specific method for estimating the image band registration error is:

[0159] S11: The registration error sources of the multispectral image between the bands based on the swing-scanning camera include at least: the principal point error of the focal plane and the scanning instability error, and the relationship is:

[0160]

[0161] Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the load CCD (Charge-Coupled Device), f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval;

[0162] S12: Select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ;

[0163] S13: Generate a simulated image of the reference band when the oscillating mirror scans at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points;

[0164] S14: Based on the image point offset between the original image and the simulated image as the error, the median value of the error of each eight columns of control points in the scanning direction is taken to generate an offset curve in the flight direction and the scanning direction, where the scanning direction is the image column direction;

[0165] S15: Based on the characteristics of the control point error changing with the scanning angle, the offset curve of the scanning direction is used to fit the scanning angle change model using the sum of sine functions, that is, the scanning instability error model of the reference band:

[0166]

[0167] Where θ(t) is the scanning angle, n is the number of sine functions, ω i is the sample frequency, φ i is the phase of the sine function, a i is the amplitude of the sine function, t represents time;

[0168] S16: Introducing the principal point error Δx estimated by principal point offset 01 and Δy 01 As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ:

[0169] Δθ(s)=θ(t1)-θ(t2)

[0170] Where s represents the coordinate of the image in the scanning direction, t1 and t2 represent the design time for the two bands to start scanning imaging respectively;

[0171] S17: Using the inter-band scanning unsteady registration model, perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band: Get the offset image coordinates (x', y').

[0172] For the specific implementation process of each step, please refer to the description of the above method.

[0173] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0174] Example 4

[0175] An embodiment of the present invention provides a computer-readable storage medium, characterized in that: a computer program is stored, and the computer program is called by a processor to implement: the steps of the above-mentioned imaging spectrometer image band registration method or the above-mentioned image band registration error estimation method.

[0176] Among them, the specific implementation method of imaging spectrometer image band registration is:

[0177] S1: Acquire registration model parameters between bands of a multispectral image of a swing-scanning camera, and perform image coordinate offset correction on image points of the image to be registered based on the registration model parameters;

[0178] S2: Based on the image coordinates of the offset-corrected image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type;

[0179] S3: Acquire the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates;

[0180] S4: Based on the image coordinates of the current image point and the point with the same name and the adaptive size of the intra-frame sampling kernel, a corresponding number of adjacent image points are selected around the two image points as sampling points. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the grayscale value of the current image point after resampling is calculated by weighted average;

[0181] The grayscale value of each pixel in the image to be registered is resampled according to the above steps S2-S4.

[0182] Among them, the specific method for estimating the image band registration error is:

[0183] S11: The registration error sources of the multispectral image between the bands based on the swing-scanning camera include at least: the principal point error of the focal plane and the scanning instability error, and the relationship is:

[0184]

[0185] Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the load CCD (Charge-Coupled Device), f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval;

[0186] S12: Select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ;

[0187] S13: Generate a simulated image of the reference band when the oscillating mirror scans at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points;

[0188] S14: Based on the image point offset between the original image and the simulated image as the error, the median value of the error of each eight columns of control points in the scanning direction is taken to generate an offset curve in the flight direction and the scanning direction, where the scanning direction is the image column direction;

[0189] S15: Based on the characteristics of the control point error changing with the scanning angle, the offset curve of the scanning direction is used to fit the scanning angle change model using the sum of sine functions, that is, the scanning instability error model of the reference band:

[0190]

[0191] Where θ(t) is the scanning angle, n is the number of sine functions, ω i is the sample frequency, φ iis the phase of the sine function, a i is the amplitude of the sine function, t represents time;

[0192] S16: Introducing the principal point error Δx estimated by principal point offset 01 and Δy 01 As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ:

[0193] Δθ(s)=θ(t1)-θ(t2)

[0194] Where s represents the coordinate of the image in the scanning direction, t1 and t2 represent the design time for the two bands to start scanning imaging respectively;

[0195] S17: Using the inter-band scanning unsteady registration model, perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band: Get the offset image coordinates (x', y').

[0196] For the specific implementation process of each step, please refer to the description of the above method.

[0197] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the software and hardware device described in any of the aforementioned embodiments, such as a hard disk or memory of a controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard disk equipped on the controller, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Furthermore, the readable storage medium can also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or is to be output.

[0198] Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0199] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is a flow chart according to the method, device (system), and computer program product of the embodiment of the present application and / or the instructions executed by the processor to generate a device for realizing the function specified in one flow chart or multiple flows and / or one box or multiple boxes of the block diagram. These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a product comprising an instruction device, which realizes the function specified in one flow chart or multiple flows and / or one box or multiple boxes of the block diagram. These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0200] It should be emphasized that the examples described in the present invention are illustrative rather than restrictive. Therefore, the present invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solution of the present invention that do not depart from the purpose and scope of the present invention, whether modified or replaced, also fall within the scope of protection of the present invention.

Claims

1. A method for band registration of imaging spectrometer images, characterized by: Applied to multispectral images acquired by a swing-scanning camera, the method comprises the following steps: S1: Acquire registration model parameters between bands of a multispectral image of a swing-scanning camera, and perform image coordinate offset correction on image points of the image to be registered based on the registration model parameters; S2: Based on the image coordinates of the offset-corrected image point in the image to be registered, determine the frame number of the current image point and the interval from the upper and lower frame edges, and then determine the intra-frame sampling kernel type of the current image point based on the interval type; S3: Acquire the object space coordinates of the current image point, and obtain the image space coordinates of the same-name points of the current image point in adjacent frames based on the object space coordinates; S4: Based on the image coordinates of the current image point and the point with the same name and the adaptive size of the intra-frame sampling kernel, a corresponding number of adjacent image points are selected around the two image points as sampling points. Then, according to the resampling weight kernel function, weights are assigned to the selected sampling points, and the grayscale value of the current image point after resampling is calculated by weighted average; The grayscale value of each pixel in the image to be registered is resampled according to the above steps S2-S4.

2. The method according to claim 1, wherein: The adaptive intra-frame sampling kernel is determined based on the frame number of the pixel and the interval from the upper and lower frame edges. When using a bicubic polynomial sampling kernel or a bilinear sampling kernel, the correspondence between the interval type and the intra-frame sampling kernel type is as follows: Bicubic polynomial sampling kernel: If the interval is greater than or equal to 2 pixels, the intra-frame sampling kernel is a single-image point sampling kernel, that is, 4*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 2 pixels but greater than or equal to 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 3*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-image point sampling kernel, that is, 2*4 adjacent image points are selected in the current frame to form the intra-frame sampling kernel; Bilinear sampling kernel: If the interval is greater than or equal to 1 pixel, the intra-frame sampling kernel is a single-pixel sampling kernel, that is, 2*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel; if the interval is less than 1 pixel, the intra-frame sampling kernel is a cross-frame dual-pixel sampling kernel, that is, 1*2 adjacent pixels are selected within the current frame to form the intra-frame sampling kernel; The resampling formula is: Where g is the grayscale value of the pixel after resampling, N1 and N2 represent the number of sampling kernel pixels in the current frame and the adjacent frame respectively, and g is the grayscale value of the pixel after resampling. i and g j Represents the grayscale value of each sampling point in the two sampling kernels of the current frame and the adjacent frame, ω i and ω j are the weights corresponding to the sampling points.

3. The method according to any one of claims 1 or 2, characterized in that: When a bicubic polynomial sampling kernel is used, weights are calculated for the sampling points selected for the image points and the points with the same name within their respective sampling kernels in step S4. The resampling weight kernel function is as follows: Where x is the distance between the image point or the same-name point and the sampling point in the respective sampling kernel, and ω(x) is the weight.

4. The method according to claim 2, wherein: If a bicubic polynomial sampling kernel is used and the interval corresponding to the current image point is greater than 2 pixels; or if a bilinear sampling kernel is used and the interval corresponding to the current image point is greater than 1 pixel, no points with the same name are introduced for resampling, that is, step S3 is not performed for the current image point, and resampling is performed in step S4 at the offset-corrected pixel coordinates within the sampling kernel corresponding to the current frame; In addition, step S3 is executed, and the image-side coordinates of the same-name points in adjacent frames in step S3 are calculated as follows: Using the initial geolocation data and according to the image coordinates of the current image point (i tran ,j tran ) Perform linear interpolation on the geolocation data in the current frame range in the flight and scanning directions, and obtain the longitude and latitude difference Δlat between the current pixel and the adjacent pixel in the flight and scanning directions respectively. x , Δlat y , Δlon x , Δlon y , lat represents latitude, lon represents longitude, and xy corresponds to the xy axis respectively; Next, the object coordinates of the current image point in this frame are calculated using bilinear interpolation Using the object coordinates of the current image point Calculating an image space coordinate correction value (Δx, Δy) from the image space coordinates (x0, y0) of the initial image point in the adjacent frame, wherein the image space coordinates (x0, y0) of the initial image point in the adjacent frame are randomly selected image points within the range of the adjacent frames; According to the image coordinates of the initial image point (x0, y0), the corresponding object coordinates lon(x0, y0) and lat(x0, y0) are calculated using bilinear interpolation. At the same time, the two-dimensional gradient change lon' of the object coordinates is calculated based on the object coordinates of the adjacent image points of the initial image point. x (x0,y0),lat' x (x0,y0),lon' y (x0,y0),lat' y (x0,y0), abbreviated as lon' x ,lat' x ,lon' y ,lat' y ; By formula Calculate the object coordinate differences Δlon and Δlat; then substitute the above parameters into the formula Calculate the image coordinate correction values Δx and Δy; Then, the image coordinates of the new iterative image point are obtained by (x0+Δx,y0+Δy) instead of (x0,y0); Here, each parameter is calculated iteratively according to the above steps until the image coordinate correction value is less than 1, that is, the two image points are in one pixel, and the image coordinate correction value (Δx n ,Δy n ), and then the image coordinates of the same name point are linearly interpolated from the object-space two-dimensional gradient of the iterative image point and the given object-space coordinates (i adjoin ,j adjoin ).

5. A method for estimating image band registration errors based on the method according to any one of claims 1 to 4, characterized in that: The method is used to obtain the registration model parameters of the multispectral image of the swing-scanning camera between the bands and perform coordinate offset correction in step S1, and includes the following steps: S11: The registration error sources of the multispectral image between the bands based on the swing-scanning camera include at least: the principal point error of the focal plane and the scanning instability error, and the relationship is: Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the payload CCD, f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval; S12: Select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ; S13: Generate a simulated image of the reference band when the oscillating mirror scans at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points; S14: Based on the image point offset between the original image and the simulated image as the error, the median value of the error of each eight columns of control points in the scanning direction is taken to generate an offset curve in the flight direction and the scanning direction, where the scanning direction is the image column direction; S15: Based on the characteristics of the control point error changing with the scanning angle, the offset curve of the scanning direction is used to fit the scanning angle change model using the sum of sine functions, that is, the scanning instability error model of the reference band: Where θ(t) is the scanning angle, n is the number of sine functions, ω i is the sample frequency, φ i is the phase of the sine function, a i is the amplitude of the sine function, t represents time; S16: Introducing the principal point error Δx estimated by principal point offset 01 and Δy 01 As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ: Δθ(s)=θ(t1)-θ(t2) Where s represents the coordinate of the image in the scanning direction, t1 and t2 represent the design time for the two bands to start scanning imaging respectively; S17: Using the inter-band scanning unsteady registration model, perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band: Get the image coordinates (x', y') after the offset; Where Δp x and Δp y is based on Determined, that is, the registration model parameters.

6. A registration system based on the method according to any one of claims 1 to 4, characterized in that: include: A registration module is used to obtain registration model parameters between bands of the multispectral image of the swing-scanning camera, and perform image coordinate offset correction on the image points of the image to be registered based on the registration model parameters; The sampling kernel determination module is used to determine the frame number of the current image point and the interval from the upper and lower frame edges based on the image coordinates of the offset-corrected image point in the image to be registered, and then determine the intra-frame sampling kernel type of the current image point based on the interval type; A homonymous point processing module is used to obtain the object-space coordinates of the current image point and obtain the image-space coordinates of the homonymous points of the current image point in adjacent frames based on the object-space coordinates; The resampling module is used to select a corresponding number of adjacent image points around the current image point and the same-name point based on the image coordinates and the adaptively sized intra-frame sampling kernel. Then, based on the resampling weight kernel function, weights are assigned to the selected sampling points. The grayscale value of the current image point after resampling is calculated by weighted average. The grayscale value of each pixel in the image to be registered is resampled according to the above process.

7. The system according to claim 6, characterized in that: The registration module includes: The registration error analysis unit is used to analyze the registration error sources between bands of the multispectral image based on the swing-scanning camera. The registration error sources include at least the principal point error of the focal plane and the scanning instability error, and the relationship is: Where Δp x and Δp y Denote the image errors in the flight and scanning directions, Δx 01 and Δy 01 They represent the principal point offset of the band to be registered on the focal plane, that is, the principal point error; μ represents the size of the payload CCD, f and Δf are the design value and change value of the principal distance, respectively, and x 01 represents the main point position of the flight direction, Δθ represents the change in scanning angle caused by non-uniform scanning speed, and θ btb Indicates the angle of the swing mirror scanning during the sampling interval; The principal point error analysis unit is used to select the reference band and use the phase correlation method to extract the overall registration error between the multispectral image bands, and then obtain the overall registration error of the sub-satellite point position. Calculate the principal point error Δx 01 and Δy 01 ; A control point extraction unit is used to generate a simulated image of the reference band when the oscillating mirror is scanning at a uniform speed by using an image simulation method, and then use image correlation matching combined with a least squares matching algorithm to obtain high-precision matching points between the original image of the reference band and the simulated image of the corresponding wavelength range as control points; an offset curve generating unit, configured to generate an offset curve in a flight direction and a scanning direction by taking the median of the errors of each eight columns of control points in a scanning direction based on the image point offset between the original image and the simulated image, wherein the scanning direction is the image column direction; The fitting unit is used to fit the scanning angle variation model based on the characteristics of the control point error changing with the scanning angle, using the offset curve of the scanning direction and the sum of sine functions, that is, the scanning instability error model of the reference band; and introduce the main point error Δx estimated by the main point offset. 01 and Δy 01 As the initial phase offset, the scanning unsteady error model θ(t2) of the band to be registered is fitted, and then the scanning unsteady error model θ(t1) of the reference band is used to calculate the difference with the scanning unsteady error model θ(t2) of the band to be registered to obtain the inter-band scanning unsteady registration model Δθ: The registration unit is used to perform pixel-by-pixel (x, y) offset correction between the remaining multispectral imaging bands and the reference band using the inter-band scanning instability registration model: Get the offset image coordinates (x', y').

8. A computer device, characterized in that: At least: one or more processors; a memory storing one or more computer programs; The processor calls the computer program to implement: The steps of the imaging spectrometer image band registration method described in any one of claims 1 to 4 or the image band registration error estimation method described in any one of claim 5.

9. A computer-readable storage medium, characterized in that: A computer program is stored, which is called by a processor to implement: The steps of the imaging spectrometer image band registration method described in any one of claims 1 to 4 or the image band registration error estimation method described in any one of claim 5.

Images