Spatiotemporal spectral fusion method for hyperspectral and multispectral images

Abstract

The present application relates to the technical field of space-time spectrum remote sensing data fusion, and especially relates to a space-time spectrum fusion method for hyperspectral images and multispectral images, which reserves the hyperspectral resolution of a reference time and the multispectral high spatial resolution of a prediction time by preprocessing GF1WFV images and GF5AHSI images and analyzing the spatial resolution of the spectral images and the spectral resolution of the hyperspectral images, constructs a space-time spectrum comprehensive prediction model, obtains a new time-phase high spatial high spectral image, and verifies the new time-phase high spatial high spectral image by using the multispectral image of the prediction time. The present application has the beneficial effect that when a hyperspectral image at a certain time cannot be obtained, the multispectral image and the hyperspectral image at a past time can be used for prediction to obtain a hyperspectral image with high spatial resolution at a required time, thereby solving the problem of data sources and serving numerous hyperspectral application fields.

Description

Technical Field

[0001] This invention relates to the field of spatiotemporal spectral remote sensing data fusion technology, and in particular to a spatiotemporal spectral fusion method for hyperspectral and multispectral images. Background Technology

[0002] In recent years, with the rapid development of hyperspectral imaging technology, hyperspectral remote sensing data has achieved significant progress and breakthroughs in various fields of application research. Hyperspectral remote sensing data has the advantages of multiple bands and high spectral resolution, but it also suffers from disadvantages such as low spatial resolution, limited satellite types leading to low coverage frequency, and the inability of a single sensor to simultaneously meet the high spatial, hyperspectral, and temporal characteristics of similar data, greatly restricting the utilization of hyperspectral data. Multi-sensor data fusion can effectively solve this problem and significantly improve the utilization efficiency of spectral data.

[0003] Spatiotemporal spectral remote sensing data fusion is the collaborative combination of temporal, spatial, and spectral information from two or more image datasets. The concept and methods of data fusion developed in the 1970s, with documented remote sensing data fusion originating in the 1990s. Since the 21st century, numerous scholars have conducted research on fusion technologies and applications, resulting in nearly ten thousand research papers both domestically and internationally. Many scholars have also combined classical methods with emerging hyperspectral data for testing and verification, providing valuable guidance for the selection of fusion methods. Current fusion methods can be categorized into three types based on index enhancement: spatial dimension enhancement (component substitution method, multi-resolution analysis method), spectral dimension enhancement (sparse unmixing method, artificial intelligence method), and temporal dimension enhancement (weighting function method, linear optimization decomposition method, artificial intelligence method), thus giving rise to research on spatial-spectral, spatiotemporal, and spatiotemporal spectral fusion. Spatial-spectral fusion often employs model optimization methods to address the problem of mixed pixels in low-resolution pixels. These methods include sharpening-based approaches (Gram-Smidt transform, Brovey algorithm, color space transformation, principal component analysis, and spatial-spectral sampling modeling methods such as sparse representation and neural networks), imaging model-based approaches (based on mixed pixel decomposition, tensor decomposition, coupled nonnegative matrix factorization (CNMF), and low-rank decomposition (LRF)), tensor-based fusion methods, and deep network-based methods such as introducing residual attention networks to enhance spatial and spectral fusion information. However, current research on the fusion of hyperspectral data with other data has not considered the temporal variations of hyperspectral inversion features. Spatiotemporal fusion mainly considers injecting change information from time-series images into reference images. More than 100 methods have emerged to date, including spatially weighted STARFM, spatially unmixed STDAF, and various combinations thereof, improved GLP spatiotemporal fusion methods, and a series of derivative research methods. Current spatiotemporal research mainly focuses on spatiotemporal fusion simulation between multispectral and multispectral images (such as MODIS and Landsat, GF1 PMS and GF1 WFV). There are relatively few literature records on the application of spatial and temporal scaling of hyperspectral images.

[0004] Hyperspectral images typically have low spatial resolution, and pixel mixing is a significant issue. Therefore, research on hyperspectral fusion must consider the decomposition of mixed pixels. The Flexible Spatial-Temporal Data Fusion (FSDAF) model, which combines weighting, unmixing, and interpolation, utilizes a weight function to combine the unmixing results with spatial interpolation, achieving good robustness and suitability for highly heterogeneous and variable scenes. This invention uses the FSDAF method for spatiotemporal fusion prediction of hyperspectral images. Based on GF5 ASHI and GF1 WFV images at time T0, and GF1 WFV images at times T1 and T2, the fusion yields simulated GF5 ASHI images and classification results at times T1 and T2. These results are compared with those from traditional methods, solving the problems of difficult acquisition and low temporal frequency of hyperspectral satellite data. This provides a method for spatiotemporal dynamic prediction research that fully leverages the characteristics of hyperspectral data. Summary of the Invention

[0005] To address the shortcomings of the existing technologies, the present invention aims to provide a spatiotemporal spectral fusion method for hyperspectral and multispectral images, which solves the problems of difficulty in acquiring hyperspectral satellite data and low temporal frequency, and provides a method for spatiotemporal dynamic prediction research that fully utilizes the characteristics of hyperspectral data.

[0006] To address the aforementioned problems, this invention discloses a spatiotemporal spectral fusion method for hyperspectral and multispectral images. In this method, GF5 AHSI provides spectral information, and GF1WFV provides temporal and spatial information. The reference time and prediction time are defined as T1 and T2, respectively, and the GF5 AHSI and GF1 WFV images are defined as HSI and MSI, respectively. The method includes the following steps:

[0007] S1. Perform mixed pixel decomposition on the HSI image at time T1 to obtain endmember category and abundance maps;

[0008] S2. Estimate the time variation of each type of image from T1 to T2;

[0009] S3. Assuming that the endmember categories do not change from T1 to T2, add the time-varying residual to the HSI image at time T1 to obtain the predicted value of the HSI image at time T2.

[0010] S4. Use the thin plate spline interpolation method (TPS) to interpolate each band of the MSI image at time T2 to predict the HSI image at time T2, and then assign the residuals to the predicted HSI image.

[0011] S5. Use neighborhood information to obtain the HSI image at time T2.

[0012] As a preferred option, the specific method of S1 is as follows:

[0013] predefined (x) i ,y i ) is the index of each cell; i is the cell index of the MSI; j is the cell index of the corresponding MSI image in each HSI cell, j = 1...n; M1(x i ,y i,b ) and M2(x i ,y i,b (x) represents the time intervals T1 and T2, respectively, when the multispectral image in the b band is at (x) i ,y i The pixel value at position H1(x) ij ,y ij,b H2(x) and H2(x) ij ,y ij,b (x) represents the time intervals T1 and T2, respectively, when the multispectral image in the b band is at (x) i ,y i The pixel value at (); h c (x i ,y i ) indicates that in a hyperspectral image, the endmember category c is in (x i ,y i The pixel value at position ); ΔM(x i ,y i,b () represents times T1 and T2, and the multispectral image in band b is in (x i ,y i The change value at the pixel; ΔH(c) ,b () represents the change value of land cover end-member category c in the hyperspectral image at times T1 and T2;

[0014] The low spatial resolution of hyperspectral images leads to the generation of mixed pixels. First, the mixed pixels of the HSI image at time T1 are decomposed to obtain the land cover endmember categories m and the abundance H of each land cover category within the pixel. m (x i ,y i ):

[0015] Define H(x) i ,y i Let ) represent the i-th cell of GF5 ASHI, and n be the value of H(x). i ,y i If the number of endmembers in the i-th pixel is given, then the abundance of each class in the i-th pixel is represented as:

[0016]

[0017] In the formula, a j For ground feature c j In f(x)i ,y i The abundance in ) is ε, where ε is the model error, which may be caused by differences in bandwidth, solar geometry and observation angle;

[0018] The endmember classes in an MSI image can be calculated by the number of classes contained in each pixel of the HSI:

[0019] H m (x i ,y i ) = N c (x i y i ) / m (2)

[0020] N c (x i ,y i ) is (x i ,y i The number of HSI image pixels belonging to class c within the MSI at ().

[0021] As a preferred method, S2 uses the following approach: Assuming no change in land cover type during the prediction time, the temporal change of land cover type in the HSI can be expressed as ΔH(c, b), and the temporal change of the MSI image is:

[0022] ΔM(x i ,y i b) = M n (x i ,y i ,b)-M m (x i ,y i b) (3)

[0023] Combining the hybrid pixel decomposition model, it can also be expressed as:

[0024]

[0025] In the formula, n (n>l) hyperspectral pixels are selected to form a mixed pixel equation system to solve ΔH(class, b), and high-purity pixels in each category are selected as much as possible.

[0026] As a preferred option, the specific method of S3 is as follows:

[0027] Predicting events that change land cover type: Assuming the land cover type does not change from T0 to TP, the temporal variation residual is added to the T0 multispectral image to obtain the temporal predicted value H of the multispectral image at time TP. tp ;

[0028] H n-tp (xij ,y ij b) = H1(x ij ,y ij ,b)+ΔH(class,b) (5)

[0029] In the formula,

[0030] Each MSI pixel is equal to the sum of the HSI pixels and the system error φ. The MSI images at times T0 and TP are represented as follows:

[0031]

[0032]

[0033] In the formula, Each MSI image pixel contains m HSI image pixels.

[0034] As a preferred option, the specific method of S4 is as follows: residual calculation and allocation. The previous prediction of time changes did not take into account the case of large changes in land type. Therefore, it is necessary to introduce residual R to characterize the differences in categories.

[0035]

[0036]

[0037] Spatial variation information was obtained from the MSI image at time TP, and thin-plate spline interpolation was performed on each band image b:

[0038]

[0039] in, r i 2 =(xx) i ) 2 +(yy i ) 2 N is the known number of pixels.

[0040] The coefficients in the above formula are optimized by minimizing them.

[0041]

[0042] The spatial prediction of the HSI image at time TP (denoted by the subscript sp) is as follows:

[0043] H n-sp (x ij ,y ij b) = f TPS-b (x ij ,y ij(12)

[0044] If the changes in the image are homogeneous, then the predicted HSI image spatial features can be considered to have no error compared to the true image. The error in predicting an HSI image based on time variations is expressed as:

[0045] E ho (x ij ,y ij b) = H n-sp (x ij ,y ij ,b)-H n-tp (x ij ,y ij b) (13)

[0046] If the land cover types in the image are heterogeneous, assuming that each MSI pixel corresponds to an HSI pixel with the same error, then

[0047] E he (x ij ,y ij b) = R(x i ,y i b) (14)

[0048] Introducing HI as a metric to represent homogeneity, if the k-th HSI pixel within the window is similar to the center pixel (x... ij ,y ij If the land types are the same, then let I... k =1, otherwise 0;

[0049] E he (x ij ,y ij b) = R(x i ,y i b) (15)

[0050] Combining homogeneous and heterogeneous errors, the overall weight can be expressed as:

[0051] weight(x ij ,y ij b) = E ho (x ij ,y ij b)×HI(x ij ,y ij )+E he (x ij ,y ij ,b)×(1-HI(x ij ,y ij (16)

[0052] Normalizing the weight yields the weight, and the residuals assigned to the HSI pixels are:

[0053] r(x ij ,y ij b) = m × r(x i ,y i b)×WEIGHT(x ij ,y ij b) (17)

[0054] All changes in HSI pixels are represented as follows:

[0055]

[0056] As a preferred method, S5 is as follows: For the final prediction, considering both the spatial distance and spectral similarity between neighboring pixels k and the target center pixel, a weight Wk is derived. The final HSI pixel prediction formula is then expressed as:

[0057]

[0058] Preferably, the following steps are also included:

[0059] S6. Model Validation: A spatiotemporal spectral fusion model of hyperspectral and multispectral images was established through analysis. The multispectral data of Gaofen-1 and Gaofen-5 were processed, and the signal-to-noise ratio, image quality, and information entropy of the predicted image were validated using Gaofen-1 multispectral image as the validation image.

[0060] Due to the adoption of the above technical solution, the beneficial effects of the present invention are as follows:

[0061] A spatiotemporal spectral fusion method for hyperspectral and multispectral images is proposed. Based on the high spatial resolution of multispectral images and the combined spatial and spectral advantages of hyperspectral images, it combines a hybrid pixel decomposition model for data fusion. Addressing the current challenges of limited global hyperspectral satellites, difficulty in data acquisition, and lack of comprehensive coverage, this method calculates hyperspectral images at the required time, solving the application needs of hyperspectral remote sensing data in various industries. It also represents a technological trend in hyperspectral development and spatiotemporal fusion, and can solve problems related to environmental change monitoring, agricultural development, and changes in land surface types. Attached Figure Description

[0062] Figure 1 Technical flowchart of the present invention;

[0063] Figure 2 The data preprocessing flowchart of this invention;

[0064] Figure 3 Pixel maps of hyperspectral and multispectral images. Detailed Implementation

[0065] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0066] like Figure 1 As shown, this invention provides a spatiotemporal spectral fusion method for hyperspectral and multispectral images. In the spatiotemporal spectral fusion, GF5 AHSI provides spectral information, and GF1WFV provides temporal and spatial information. The reference time and prediction time are defined as T1 and T2, respectively, and the GF5 AHSI and GF1 WFV images are defined as HSI and MSI, respectively. The hyperspectral spatiotemporal prediction model is corrected based on the FSDAF method. The main goal is to obtain the hyperspectral information of the HSI image and the high spatial and temporal scale information of the MSI image, including the following steps:

[0067] S1. Perform mixed pixel decomposition on the HSI image at time T1 to obtain endmember category and abundance maps;

[0068] Mixed pixels: These are the basic units that make up remote sensing images. If a pixel contains multiple coverage types, it is called a mixed pixel.

[0069] Endmember: The basic building block of each mixed pixel;

[0070] Abundance: The proportion of each endmember in its own pixel. HSI images have low spatial resolution, and a pixel contains many land cover categories. Each category is a unit, and the proportion of each category in a pixel is the abundance.

[0071] S2. Estimate the time variation of each type of image from T1 to T2;

[0072] S3. Assuming that the endmember categories do not change from T1 to T2, add the time variation residual (the residual between the actual value and the observed value of the abundance ratio change of each endmember category) to the HSI image at time T1 to obtain the predicted value of the HSI image at time T2.

[0073] S4. Using thin plate spline interpolation (TPS: a commonly used 2D spatial interpolation method, often used for image deformation, etc., which can drive image changes with a small number of control points), interpolate each band of the MSI image at time T2 to predict the HSI image at time T2, and assign the residuals to the predicted HSI image;

[0074] S5. Use neighborhood information to obtain the HSI image at time T2;

[0075] S6. Model Validation: A spatiotemporal spectral fusion model of hyperspectral and multispectral images was established through analysis. The multispectral data of Gaofen-1 and Gaofen-5 were processed, and the multispectral image of Gaofen-1 was used as the validation image to verify the signal-to-noise ratio, image quality, information entropy, etc. of the predicted image.

[0076] Furthermore, the specific method of S1 is as follows:

[0077] predefined (x) i ,y i ) is the index of each cell; i is the cell index of the MSI; j is the cell index of the corresponding MSI image in each HSI cell, j = 1...n; M1(x i ,y i b) and M2(x i ,y i b) represent the multispectral images at times T1 and T2, respectively, in band b (x i ,y i The pixel value at position H1(x) ij ,y ij b) and H2(x) ij ,y ij b) represent the multispectral images at times T1 and T2, respectively, in band b (x i ,y i The pixel value at (); h c (x i ,y i ) indicates that in a hyperspectral image, the endmember category c is in (x i ,y i The pixel value at position ); ΔM(x i ,y i b) represents times T1 and T2, and the multispectral image in band b is in (x i ,y i The change value at the pixel; ΔH(c) ,b () represents the change value of land cover end-member category c in the hyperspectral image at times T1 and T2;

[0078] The low spatial resolution of hyperspectral images leads to the generation of mixed pixels. First, the mixed pixels of the HSI image at time T1 are decomposed to obtain the land cover endmember categories m and the abundance H of each land cover category within the pixel. m (x i ,y i ):

[0079] like Figure 3 As shown, the left -GF5 ASHI corresponds to one pixel, and the right -GF1 WFV corresponds to a mixed pixel. H(x) is defined as follows: i ,y iLet ) represent the i-th cell of GF5 ASHI, and n be the value of H(x). i ,y i If the number of endmembers in the i-th pixel is given, then the abundance of each class in the i-th pixel is represented as:

[0080]

[0081] In the formula, a j For ground feature c j In f(x) i ,y i The abundance in ) is ε, where ε is the model error, which may be caused by differences in bandwidth, solar geometry, and observation angle.

[0082] The endmember classes in an MSI image can be calculated by the number of classes contained in each pixel of the HSI:

[0083] H m (x i ,y i ) = N c (x i y i ) / m (2)

[0084] N c (x i ,y i ) is (x i ,y i The number of HSI image pixels belonging to class c within the MSI at () is where m is the land feature end-member category.

[0085] The specific method of S2 is as follows: During the prediction of changes in time, it is assumed that the land cover type does not change.

[0086] Therefore, the temporal variation of land cover categories in HSI can be represented as ΔH(c, b). The temporal variation of the MSI image is:

[0087] ΔM(x i ,y i b) = M n (x i ,y i ,b)-M m (x i ,y i b) (3)

[0088] Combining the hybrid pixel decomposition model, it can also be expressed as:

[0089]

[0090] In the formula, n (n>l) hyperspectral pixels are selected to form a mixed pixel equation system to solve ΔH(class, b), and high-purity pixels in each category are selected as much as possible.

[0091] The specific method for S3 is as follows: Spatial prediction (predicting events that change land cover types): Assuming that the land cover type does not change from T0 to TP, the temporal variation residual is added to the T0 multispectral image to obtain the temporal prediction value H of the multispectral image at time TP. tp .

[0092] H n-tp (x ij ,y ij b) = H1(x ij ,y ij ,b)+ΔH(class,b) (5)

[0093] In the formula,

[0094] Each MSI pixel equals the sum of the HSI pixels and the system error φ. The MSI images at times T0 and TP are represented as follows:

[0095]

[0096]

[0097] In the formula, Each MSI image pixel contains m HSI image pixels;

[0098] The specific method for S4 is as follows: residual calculation and allocation.

[0099] The previous predictions included time-related changes, but did not account for significant changes in land type. Therefore, residuals R are needed to characterize the differences between categories.

[0100]

[0101]

[0102] Spatial variation information was obtained from the MSI image at time TP, and thin-plate spline interpolation was performed on each band image b:

[0103]

[0104] in, r i 2 =(xx) i ) 2 +(yy i ) 2 N is the known number of pixels.

[0105] The coefficients in the above equation are optimized by minimizing them.

[0106]

[0107] The spatial prediction of the HSI image at time TP (denoted by the subscript sp) is as follows:

[0108] H n-sp (x ij ,y ij b) = f TPS-b (x ij ,y ij (12)

[0109] If the changes in the image are homogeneous, then the predicted HSI image spatial features can be considered to have no error compared to the true image. The error in predicting HSI images based on time variations is expressed as:

[0110] E ho (x ij ,y ij b) = H n-sp (x ij ,y ij ,b)-H n-tp (x ij ,y ij b) (13)

[0111] If the land cover types in the image are heterogeneous, assuming that each MSI pixel corresponds to an HSI pixel with the same error, then

[0112] E he (x ij ,y ij b) = R(x i ,y i b) (14)

[0113] Introducing HI as a metric to represent homogeneity, if the k-th HSI pixel within the window is similar to the center pixel (x... ij ,y ij If the land types are the same, then let I... k =1, otherwise 0.

[0114] E he (x ij ,y ij b) = R(x i ,y i b) (15)

[0115] Combining homogeneous and heterogeneous errors, the overall weight can be expressed as:

[0116] weight(x ij ,y ij b) = E ho (x ij ,y ij b)×HI(x ij ,y ij )+E he (x ij ,y ij ,b)×(1-HI(x ij ,y ij (16)

[0117] We obtain the weight by normalizing the weight. Then the residuals assigned to the HSI pixels are...

[0118] r(x ij ,y ij b) = m × r(x i ,y i b)×WEIGHT(x ij ,y ij b) (17)

[0119] All changes in HSI pixels are represented as follows:

[0120]

[0121] The specific method for S5 is as follows: Final prediction. Taking into account the spatial distance and spectral similarity between neighboring pixels k and the target center pixel, a weight Wk is derived. The final HSI pixel prediction formula is then expressed as:

[0122]

[0123] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A spatiotemporal spectral fusion method for hyperspectral and multispectral images, characterized in that, In spatiotemporal spectral fusion, GF5 AHSI provides spectral information, and GF1WFV provides temporal and spatial information. The reference time and predicted time are defined as T1 and T2, respectively, and the GF5 AHSI and GF1 WFV images are defined as HSI and MSI, respectively. The process includes the following steps: S1. Perform mixed pixel decomposition on the HSI image at time T1 to obtain endmember category and abundance maps; S2. Estimate the time variation of each type of image from T1 to T2; S3. Assuming that the endmember categories do not change from T1 to T2, add the time-varying residual to the HSI image at time T1 to obtain the predicted value of the HSI image at time T2. S4. Use the thin plate spline interpolation method (TPS) to interpolate each band of the MSI image at time T2 to predict the HSI image at time T2, and then assign the residuals to the predicted HSI image. S5. Use neighborhood information to obtain the HSI image at time T2.

2. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 1, characterized in that, The specific method for S1 is as follows: predefined (x) i ,y i ) is the index of each cell; i is the cell index of the MSI; j is the cell index of the corresponding MSI image in each HSI cell, j=1...n; M1(x i ,y i,b ) and M2(x i ,y i,b (x) represent the times T1 and T2, respectively, when the multispectral image in the b band is at (x) i ,y i The pixel value at (x); H1(x) ij ,y ij,b H2(x) and H2(x) ij ,y ij,b (x) represent the hyperspectral images at times T1 and T2, respectively, in the b band at (x) i ,y i The pixel value at (); h c (x) i ,y i ) indicates that in a hyperspectral image, the endmember category c is in (x i ,y i The pixel value at position ); ΔM(x i ,y i,b () represents times T1 and T2, and the multispectral image in band b is in (x i ,y i The change value at the pixel; ΔH(c) ,b () represents the change value of land cover end-member category c in the hyperspectral image at times T1 and T2; The low spatial resolution of hyperspectral images leads to the generation of mixed pixels. First, the mixed pixels of the HSI image at time T1 are decomposed to obtain the land cover endmember categories m and the abundance H of each land cover category within the pixel. m (x) i ,y i ): Define H(x) i ,y i Let be the i-th pixel of GF5 ASHI, and n be H(x). i ,y i If the number of endmembers in the i-th pixel is given, then the abundance of each class in the i-th pixel is represented as: (1)； In the formula, a j For ground feature c j In f(x) i ,y i The abundance in ) is ε, where ε is the model error, which is caused by differences in bandwidth, solar geometry, and observation angle; The endmember classes in an MSI image can be calculated by the number of classes contained in each pixel of the HSI: (2)； N c (x i , y i ) is (x i , y i The number of HSI image pixels belonging to class c within the MSI at ().

3. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 2, characterized in that, The specific method for S2 is as follows: Assuming no change in land cover type during the prediction of time changes, the time change of land cover type in HSI can be expressed as follows: The time variation of the MSI image is as follows: (3)； express Time-series multispectral images at pixels ( ), the pixel value of the b-th band; express Multispectral images at different times in pixels ( ), the pixel value of the b-th band; Combining the hybrid pixel decomposition model, it can also be expressed as: （4）； In the formula, Indicates the abundance of endmember categories for land cover type c; In the formula, n (n>l) hyperspectral pixels are selected to form a set of mixed pixel equations. Solve by selecting the high-purity pixels in each category.

4. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 3, characterized in that, The specific method for S3 is as follows: Predicting events that change land cover type: Assuming the land cover type remains unchanged from T0 to TP, the temporal variation residual is added to the T0 multispectral image to obtain the temporal predicted value of the multispectral image at time TP. ; (5)； In the formula, ( ):express Hyperspectral image at time of pixel ( Abundance of band b; ; Each MSI pixel is equal to the sum of the HSI pixels and the system error φ. The MSI images at times T0 and TP are represented as follows: (6)； (7)； In the formula, Each MSI image pixel contains m HSI image pixels.

5. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 4, characterized in that, The specific method of S4 is as follows: residual calculation and allocation, introducing residual R to characterize class differences; (8)； (9)； Spatial variation information was obtained from the MSI image at time TP, and thin-plate spline interpolation was performed on each band image b: (10)； These are the coefficients to be solved, representing the center point of the i-th known point, i.e., the i-th coarse-resolution pixel. The degree of influence on the final interpolated surface, As basis functions, The pixels to be predicted With the i-th known coarse pixel center point The square of the Euclidean distance between them; where, , N is the known number of pixels. The coefficients in the above formula are optimized by minimizing them. (11)； The spatial prediction of the HSI image at time TP is represented as follows: (12)； If the changes in the image are homogeneous, the error in predicting the HSI image based on time changes is expressed as: (13)； If the land cover types in the image are heterogeneous, assuming that each MSI pixel corresponds to an HSI pixel with the same error, then (14)； HI is introduced as a factor to characterize the degree of homogeneity; Combining homogeneous and heterogeneous errors, the overall weight can be expressed as: (16)； Normalizing the weight yields the weight, and the residuals assigned to the HSI pixels are: (17)； All changes in HSI pixels are represented as follows: , (18)。 6. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 5, characterized in that, The specific method of S5 is as follows: For the final prediction, the spatial distance and spectral similarity between the neighboring pixel k and the target center pixel are comprehensively considered to obtain the weight Wk. The final HSI pixel prediction formula is then expressed as: (19)。 7. The spatiotemporal spectral fusion method for hyperspectral and multispectral images according to claim 1, characterized in that, It also includes the following steps: S6. Model Validation: A spatiotemporal spectral fusion model of hyperspectral and multispectral images was established through analysis. The multispectral data of Gaofen-1 and Gaofen-5 were processed, and the signal-to-noise ratio, image quality, and information entropy of the predicted image were validated using Gaofen-1 multispectral image as the validation image.

Images