Near-real-time reconstruction method, device, equipment and medium for multispectral remote sensing time sequence images in whole growth season of crops

Abstract

The invention discloses a near-real-time reconstruction method, device and equipment for multispectral remote sensing time sequence images in the whole growth season of crops and a medium, and relates to the field of remote sensing data reconstruction. The method comprises the following steps: acquiring historical time sequence remote sensing image spectral data of a target ground feature type, constructing a spectral reference curve of the target ground feature type in a whole growing season by adopting a ground feature change detection algorithm and a harmonic function fitting algorithm, and further constructing a vegetation index reference curve of the target ground feature type in the whole growing season; according to the vegetation index reference curve and the near-real-time vegetation index data, utilizing an extended Kalman filtering algorithm to reconstruct a vegetation index to obtain a vegetation index reconstruction result; and according to the spectrum reference curve, the near real-time remote sensing image spectrum data and the vegetation index reconstruction result, performing remote sensing image spectrum data reconstruction by using an extended Kalman filtering algorithm to obtain a remote sensing image spectrum data reconstruction result. According to the invention, reconstruction of near-real-time remote sensing image spectral data is realized.

Description

Technical Field

[0001] The present application relates to the field of remote sensing data reconstruction, and in particular to a method, device, equipment and medium for near real-time reconstruction of multispectral remote sensing time series images of crops throughout their growing season. Background Art

[0002] Accurate and real-time crop mapping, growth monitoring information and yield forecast information are important guarantees for agricultural management and food security.

[0003] Accurate and timely intra-season crop mapping and distribution are necessary for near real-time (NRT) crop monitoring. However, due to the difficulty in obtaining near-real-time data, most current crop mapping research is post-season mapping, and the timeliness is difficult to meet the current agricultural production needs. For example, the existing more mature crop mapping products, the annual cropland data layer (CDL) produced by the United States Department of Agriculture, need to be produced at least four months after the harvest of the current growing season. Near-real-time crop intra-season mapping is challenging. The main point is that crop phenology changes rapidly in a relatively short time scale, so remote sensing data with high temporal frequency is required. However, the current remote sensing data is greatly affected by interference such as clouds and rain during the growing season, so it is difficult to accurately perform near-real-time intra-season crop mapping. Therefore, it is of great significance to reconstruct optical remote sensing data during the crop growing season in near real time.

[0004] In response to the problem of insufficient availability of intra-season crop mapping data, scholars often map based on time series images after removing abnormal interference such as clouds and rain. Based on the reconstructed time series images, we can make full use of the phenological change laws of different crops and better extract the temporal characteristics of different crops. At present, there are two main ideas for time series image reconstruction. One is to map intra-season crops based on multi-remote sensing image spectral data synthesized from one or a few periods, which is a more commonly used method (Chen et al., 2021). Although this method is fast and simple, it also has obvious shortcomings. When the remote sensing images in the area are disturbed by clouds and rain for a long time, it is difficult to obtain image data for the best phenological period, and there are obvious deficiencies in the extraction of key features of the crop growth period. The other is a reconstruction method based on time series filtering. Based on the continuity and volatility of long-term remote sensing observation data in time, a corresponding mathematical model is established on the basis of summarizing the change law of remote sensing data. The reconstruction of long-term remote sensing data is achieved by filtering the time series data as a whole, such as Savitzky-Golay (SG) filtering (Chen et al., 2004; Gu et al., 2024), Whittaker filtering, etc. (Kong et al., 2019). However, this type of method requires time series images of one to two months before and after the target reconstruction image as the time series filtering window. To reconstruct high-quality images, this type of method has a time lag.

[0005] In summary, previous time series reconstruction technologies often reconstruct historical cloud-free time series images based on continuous time series observation data, which cannot process cloud pollution remote sensing images acquired in real time and cannot provide timely data services for crop identification and disaster monitoring. Summary of the invention

[0006] The purpose of this application is to provide a method, device, equipment and medium for near-real-time reconstruction of multi-spectral remote sensing time series images of crops throughout the growing season, which can realize the reconstruction of near-real-time remote sensing image spectral data, obtain real-time cloud-free time series images, and provide timely data services for crop identification and disaster monitoring.

[0007] To achieve the above objectives, this application provides the following solutions:

[0008] In a first aspect, the present application provides a near real-time reconstruction method for multispectral remote sensing time series images of crops throughout the growing season, comprising:

[0009] Acquire historical time-series remote sensing image spectral data of target object types;

[0010] Based on the historical time-series remote sensing image spectral data, a spectral reference curve of the full growing season of the target land object type is constructed by using a land object change detection algorithm and a harmonic wave function fitting algorithm;

[0011] Based on the spectral reference curve, construct a vegetation index reference curve for the target object type throughout the growing season;

[0012] According to the vegetation index reference curve and the near-real-time vegetation index data, the vegetation index is reconstructed using the extended Kalman filter algorithm to obtain the vegetation index reconstruction result;

[0013] According to the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results, the extended Kalman filter algorithm is used to reconstruct the remote sensing image spectral data to obtain the remote sensing image spectral data reconstruction results.

[0014] In a second aspect, the present application provides a near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season, wherein the near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season is applied to the above-mentioned near-real-time reconstruction method for multispectral remote sensing time series images of crops throughout the growing season, and the near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season comprises:

[0015] The historical remote sensing image spectral data acquisition module is used to obtain the historical time-series remote sensing image spectral data of the target object type;

[0016] A spectral reference curve construction module is used to construct a spectral reference curve of the target ground object type throughout the growing season by using a ground object change detection algorithm and a harmonic wave function fitting algorithm based on the historical time series remote sensing image spectral data;

[0017] A vegetation index reference curve construction module, used to construct a vegetation index reference curve of the target object type throughout the growing season based on the spectral reference curve;

[0018] The vegetation index reconstruction module is used to reconstruct the vegetation index according to the vegetation index reference curve and the near-real-time vegetation index data using the extended Kalman filter algorithm to obtain the vegetation index reconstruction result;

[0019] The remote sensing image spectral data reconstruction module is used to reconstruct the remote sensing image spectral data based on the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results, and use the extended Kalman filter algorithm to obtain the remote sensing image spectral data reconstruction results.

[0020] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned near-real-time reconstruction method of multispectral remote sensing time series images of crops throughout their growing season.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned near-real-time reconstruction method of multispectral remote sensing time series images of crops throughout their growing season.

[0022] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0023] The present application provides a method, device, equipment and medium for near-real-time reconstruction of multi-spectral remote sensing time series images of crops throughout the growing season. First, based on the spectral data of historical time series remote sensing images, the spectral reference curve of the target object type throughout the growing season is obtained by using the ground object change detection algorithm and the harmonic wave function fitting algorithm. Secondly, the spectral reference curve is migrated in combination with the near-real-time image, and the near-real-time vegetation index is estimated through the time series trend, and the vegetation index reconstruction result is migrated to the remote sensing image spectral data reconstruction process. Finally, the extended Kalman filter method (EKF) is used to simulate the nonlinear change trend of vegetation characteristics, and the prior estimation value and the near-real-time observation value obtained by the migrated spectral reference curve are assimilated, and finally the near-real-time vegetation index reconstruction result and the remote sensing image spectral data reconstruction result are obtained. The present application realizes the reconstruction of near-real-time remote sensing image spectral data, and can obtain real-time cloud-free time series images, providing effective data support for the accurate identification and parameter inversion of crops throughout the growing season. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flowchart of a method for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season provided in one embodiment of the present application;

[0026] Figure 2 A schematic diagram of historical data migration and reconstruction of near real-time data provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of an algorithm testing area in the agricultural area of ​​Coleambally, Australia, provided for an embodiment of the present application;

[0028] Figure 4 A schematic diagram of an algorithm testing area in an agricultural area of ​​Tai'an, China, provided for an embodiment of the present application;

[0029] Figure 5 A simulated distribution diagram of cloud tags provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram comparing the vegetation index reconstruction result provided in one embodiment of the present application with a typical algorithm;

[0031] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] The key scientific problem of near-real-time time series reconstruction is the lack of timely remote sensing information. Historical optical data can help in the near-real-time information reasoning in near-real-time remote sensing data reconstruction. Currently, the spatiotemporal correlation between historical optical data and near-real-time optical remote sensing images is not clear, and it is urgent to establish a near-real-time time series reconstruction model for remote sensing data.

[0035] Based on this, this application explores how to make full use of historical optical remote sensing information to reasonably fill in the missing information of near-real-time remote sensing data, produce high-quality cloud-free near-real-time remote sensing data, and provide data for high-precision identification and inversion of crops.

[0036] In an exemplary embodiment, Figure 1 As shown, a method for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season is provided, and the test method includes the following steps 101 to 105. Among them:

[0037] Step 101, obtaining historical time-series remote sensing image spectral data of the target object type.

[0038] Step 102: construct a spectral reference curve of the target land object type throughout the growing season by using a land object change detection algorithm and a harmonic wave function fitting algorithm based on the historical time series remote sensing image spectral data.

[0039] Step 103: constructing a vegetation index reference curve for the entire growing season of the target feature type based on the spectral reference curve.

[0040] Step 104 , reconstructing the vegetation index according to the vegetation index reference curve and the near real-time vegetation index data using an extended Kalman filter algorithm to obtain a vegetation index reconstruction result.

[0041] Step 105 , based on the spectral reference curve, the near real-time remote sensing image spectral data and the vegetation index reconstruction result, the extended Kalman filter algorithm is used to reconstruct the remote sensing image spectral data to obtain the remote sensing image spectral data reconstruction result.

[0042] The reconstruction method provided in the above steps 101 to 105 realizes the reconstruction of near-real-time remote sensing image spectral data, and can obtain real-time cloud-free time-series images, providing effective data support for accurate identification and parameter inversion of crops throughout the growing season.

[0043] As an implementation method of time series data reconstruction, the amount of historical remote sensing multispectral data needs to be limited. In terms of the historical data period, since long-term stable historical data is required to obtain a reference curve, it is recommended that the historical input data should be greater than 1 year. Since time migration is performed based on historical reference curves to estimate near-real-time multispectral data, the spatial resolution and spectral band settings of the historical input data and the remote sensing image data to be estimated need to be unified.

[0044] The above step 102 mainly considers the curve change problem caused by the change of the ground object type in the historical process, and mainly uses the ground object change detection algorithm (CCDC) and the harmonic wave function fitting (HANTS) method to obtain the spectral reference curve.

[0045] Among them, the steps of the CCDC algorithm mainly include four steps: data preprocessing, time series segmentation, change vector analysis and change detection.

[0046] (1) Data preprocessing: The input historical time series remote sensing image spectral data needs to be preprocessed, including atmospheric correction, radiation correction, cloud mask, etc. This can eliminate the noise in the image data and improve the data quality.

[0047] (2) Time series segmentation: Split the continuous historical time series remote sensing image spectral data into different time periods. This can decompose the long time series historical time series remote sensing image spectral data into multiple shorter time periods, which is convenient for subsequent change analysis.

[0048] (3) Change vector analysis: Change vector analysis is performed on the time series remote sensing image spectral data of each time period. By calculating the change vector in each time period, the change of surface cover can be obtained. The change vector includes information such as change amplitude, change direction and change time.

[0049] (4) Change detection: Change detection is performed based on the results of the change vector. By setting a threshold or using statistical methods, the changes in the historical land feature types of the surface are judged and the historical time series data period corresponding to the land feature types that are stable and highly reliable in the historical process is selected as the target time series remote sensing image spectral data.

[0050] The harmonic function fitting method is mainly divided into two steps: first, the amplitude and phase of non-zero frequency are obtained through Fourier transform, and then the target time series remote sensing image spectral data is fitted with the least squares. Secondly, by comparing the target time series remote sensing image spectral data with the fitting curve, those points that are obviously lower than the fitting curve are regarded as cloud pollution points and their weights are assigned to zero to refuse to participate in the curve fitting. A new curve fitting is performed on the remaining points, and the spectral reference curve is obtained through this repeated iterative process.

[0051] In another exemplary embodiment, the following steps 201 and 202 may be used to replace the step 102 in which the spectral reference curve of the target object type in the whole growing season is constructed based on the historical time series remote sensing image spectral data by using the object change detection algorithm and the harmonic wave function fitting algorithm.

[0052] Step 201, using a ground object change detection algorithm, selects time-series remote sensing image spectral data of the entire growing season that meets stability and credibility conditions from historical time-series remote sensing image spectral data as target time-series remote sensing image spectral data, specifically including:

[0053] The historical time series remote sensing image spectral data is divided into time series remote sensing image spectral data of multiple time periods according to the full growing season of the target land object type.

[0054] The variation of the time series remote sensing image spectral data in each time period is analyzed to obtain the variation vector of the time series remote sensing image spectral data in each time period; the variation vector includes: variation amplitude, variation direction and variation time.

[0055] Determine the time series remote sensing image spectral data of the time period in which the change vector meets the preset change conditions as the target time series remote sensing image spectral data; the preset change conditions are that the change amplitude is within the preset amplitude range, the change direction is within the preset direction range, and the change time is within the preset time range.

[0056] Step 202, using a harmonic function fitting algorithm to fit the target time series remote sensing image spectral data to obtain the spectral reference curve specifically includes:

[0057] Let the value of n be 1, and initialize the target time series remote sensing image spectral data to the time series remote sensing image spectral data after the 0th update.

[0058] The least square fitting is performed on the time series remote sensing image spectral data updated for the n-1th time to obtain the fitting curve obtained by the nth fitting.

[0059] The time series remote sensing image spectral data after the n-1th update is compared with the fitting curve obtained by the nth fitting, and the remote sensing image spectral data whose deviation distance from the fitting curve obtained by the nth fitting is greater than the distance threshold is removed to obtain the time series remote sensing image spectral data after the nth update, and the value of n is increased by 1, and the step of "performing a least square fitting on the time series remote sensing image spectral data after the n-1th update to obtain the fitting curve obtained by the nth fitting" is returned until the deviation of the target time series remote sensing image spectral data from the fitting curve obtained by the nth fitting is less than the deviation threshold, and the fitting curve obtained by the nth fitting is output as the spectral reference curve.

[0060] In the data reconstruction process in the above step 104 and step 105, the Extended Kalman Filter (EKF) algorithm is used.

[0061] The EKF algorithm is implemented in two steps. It is divided into two different sets of equations: time update (prediction) and real-time update (correction). Both sets of equations are applied to the k-th (real-time) state. The time update propagates the estimate of the state at the previous moment and its credibility based on the nonlinear state transition model. The real-time update is calculated as a linear combination of the prior model estimate and the weighted difference between the observed value and the prior estimate of the model state, and is updated based on new real-time observations.

[0062] The process of reconstructing the vegetation index and calculating its credibility in the above embodiment is: first, reconstructing the vegetation index data based on the vegetation index time series data calculated from the historical data, and secondly, based on the vegetation index reconstruction, taking the vegetation data as prior knowledge, and then reconstructing the multispectral data. Based on the extended Kalman filter framework, the input is the credibility of the vegetation index data observation value obtained by the basic reference curve obtained based on the historical data, the variance of the reference curve, and the deviation between the observed value and the reference curve. Based on the prior knowledge of the above three inputs, the EKF method is used for assimilation and fusion to obtain the near real-time reconstruction data of the vegetation index.

[0063] In another exemplary embodiment, step 104 reconstructs the vegetation index based on the vegetation index reference curve and the near real-time vegetation index data using an extended Kalman filter algorithm to obtain a vegetation index reconstruction result, which specifically includes the following steps 301 to 303.

[0064] Step 301 : solving a first state transition matrix according to fitting values ​​of vegetation indices at moments k and k-1 in near real time in a vegetation index reference curve.

[0065] In the embodiment of the present application, the state transition matrix is ​​defined as the gradient of the reference curve in the posterior state. Further, the solution formula of the first state transition matrix is:

[0066]

[0067] Among them, A is the first state transition matrix, is the gradient of the vegetation index reference curve, yscale is the range stretching coefficient, x is time, and tshift is the offset before and after time.

[0068] Step 302: Based on the first state transition matrix and the observed value of the vegetation index at time k-1 in the near real-time vegetation index data, the vegetation index is estimated a priori using the following formula to obtain the a priori estimated value of the vegetation index at time k and the a priori estimated value of the vegetation index credibility.

[0069]

[0070]

[0071] in, is the prior estimate of the vegetation index at time k, that is, the rough estimate before the measurement update and correction. A is the first state transition matrix, and B is the first control matrix. Because there is no external interference, it is set to 0. u k is the vegetation index control value at time k, is the posterior estimate of the vegetation index at time k-1, which is obtained based on the observed value of the vegetation index at time k-1 in the near-real-time vegetation index data and the prior estimate of the vegetation index at time k-1. is the prior estimate of the credibility of the vegetation index at time k, P k-1 is the posterior estimate of the credibility of the vegetation index at time k-1, Q is the first covariance matrix, because for the vegetation index time series curve, the change of the curve has only one independent state feature, so Q is defined as the variance of the vegetation index reference curve, and the superscript T represents transposition.

[0072] Step 303, based on the prior estimation value of the vegetation index at time k, the prior estimation value of the vegetation index credibility and the observed value of the vegetation index at time k in the near real-time vegetation index data, the vegetation index is estimated a posteriori using the following formula to obtain the a posteriori estimation value of the vegetation index at time k and the a posteriori estimation value of the vegetation index credibility.

[0073]

[0074]

[0075]

[0076] Among them, K k is the first Kalman gain at time k, H is the measurement matrix representing the relationship between the true value and the observed value of the vegetation index, R is the third covariance matrix representing the variance of the relationship between the true value and the observed value of the vegetation index, is the posterior estimate of the vegetation index at time k, Z k is the observed value of the vegetation index at time k in the near-real-time vegetation index data, P k is the posterior estimate of the credibility of the vegetation index at time k, and I is the unit matrix.

[0077] and All are required for the k+1 (future) estimate. The Kalman gain calculated in this application is required for the current reconstruction data estimate, which is also the most important part of this set of equations. k =Hx k +v k , which is the real-time observation value. H is used to convert the true state (x k ) and the observed value Z k H is the real state (x k ) is converted to a measured value (Z k ) can be set to 1.

[0078] The process of reconstructing the remote sensing image spectral data in the above embodiment is: based on the vegetation index reconstruction result, the vegetation data is used as prior knowledge to reconstruct the remote sensing image spectral data. Based on the extended Kalman filter framework, the input is the credibility of the instant observation value obtained based on the spectral reference curve obtained based on the historical multi-spectral data and the variance of the spectral reference curve (as prior knowledge, this input is the credibility calculated based on the deviation between the vegetation index observation value and the reference curve). Based on the prior knowledge of the above three inputs, the EKF method is used for assimilation and fusion to obtain the near real-time reconstruction data of the multi-spectrum.

[0079] In another exemplary embodiment, step 105 reconstructs the remote sensing image spectral data based on the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results using an extended Kalman filter algorithm to obtain the remote sensing image spectral data reconstruction results, which specifically includes the following steps 401-403.

[0080] Step 401, solving a second state transition matrix according to the fitting values ​​of the remote sensing image spectral data at time k and k-1 in near real time in the spectral reference curve.

[0081] Step 402, based on the second state transition matrix and the observed value of the remote sensing image spectral data at time k-1 in the near real-time remote sensing image spectral data, the remote sensing image spectral data is estimated a priori using the following formula to obtain the a priori estimated value of the remote sensing image spectral data at time k and the a priori estimated value of the remote sensing image spectral data credibility.

[0082]

[0083]

[0084] in, is the prior estimate of the spectral data at time k, A' is the second state transition matrix, B' is the second control matrix, u' k is the spectral control quantity at time k, is the posterior estimate of the remote sensing image spectral data at time k-1, is the prior estimate of the credibility of the remote sensing image spectral data at time k, P' k-1 is the posterior estimate of the credibility of the remote sensing image spectral data at time k-1, Q' is the second covariance matrix, specifically the covariance matrix of the change from time k-1 to time k, which is defined as the uncertainty of the predicted value corresponding to time k-1, and is represented by the variance of all historical values ​​at time k-1.

[0085] Step 403, based on the prior estimation value of the remote sensing image spectral data at time k, the prior estimation value of the credibility of the remote sensing image spectral data and the observed value of the remote sensing image spectral data at time k in the near real-time remote sensing image spectral data, use the following formula to perform a posteriori estimation on the remote sensing image spectral data to obtain the a posteriori estimation value of the remote sensing image spectral data at time k and the a posteriori estimation value of the credibility of the remote sensing image spectral data.

[0086]

[0087]

[0088] P' k =P k .

[0089] Among them, K' k is the second Kalman gain at time k, H' is the measurement matrix that characterizes the relationship between the prior estimate and the observed value of the remote sensing image spectral data, R' is the fourth covariance matrix that characterizes the variance of the relationship between the prior estimate and the observed value of the remote sensing image spectral data, is the posterior estimate of the remote sensing image spectral data at time k, Z' k is the observed value of the remote sensing image spectral data at time k in the near real-time remote sensing image spectral data, P' kis the posterior estimate of the credibility of the remote sensing image spectral data at time k, P k is the posterior estimate of the credibility of the vegetation index at time k.

[0090] In order to illustrate the effectiveness of the above method, the present embodiment also conducted an accuracy evaluation, specifically including:

[0091] Based on the vegetation index and multispectral data reconstruction results obtained after the near-real-time reconstruction framework, this application evaluates the reconstruction effect of multispectral data from the two aspects of spectrum and texture of the reconstructed data itself, and selects a total of 4 accuracy indicators in spectrum and texture for evaluation. Root mean square error (RMSE), mean absolute error (MAE), structural similarity (SSIM), and local binary pattern (LBP) are selected as accuracy evaluation indicators for near-real-time remote sensing data reconstruction. The specific examples are as follows:

[0092] Step 1, such as Figure 2 As shown in the figure, Sentinel-2 multispectral remote sensing data is taken as an example. The historical input data is all available images from January 1, 2019 to the date of the image to be reconstructed. The input images of the Tai'an study area in Shandong are from January 1, 2019 to March 2023, and the input images of the Australian study area are from January 1, 2019 to December 2023. Figure 3 and Figure 4 As shown, Figure 3 (a) is a panoramic view of study area 1 (Coleambally, Australia). Figure 3 (a1) is a local detail map of study area 1. Figure 3 Middle (a2) is the full growing season phenology curve of typical vegetation in study area 1; Figure 4 (b), (b1), and (b2) are respectively the panorama, local detail map, and full-growing-season phenological curve of typical vegetation in Study Area 2 (Tai'an, China). In terms of spatial resolution, the projection coordinate system, geographic coordinate system, and spatial resolution of each data must be consistent for subsequent calculations. In this embodiment, the Sentinel-2 multispectral data is derived from the Google EarthEngine Google Cloud Platform, with a spatial resolution of 10m and a temporal resolution of 5 days.

[0093] Step 2, such as Figure 2 As shown, the CCDC algorithm is used to detect time series changes in the data from 2019 to 2022. The mutation time point of the landform in the time series can be detected, and whether the landform type after the mutation is stable or not can be judged. The data of the time period corresponding to the stable landform type (2020-2022) is input into the harmonic function fitting method to obtain the reference curve produced by historical data, namely, the spectral reference curve.

[0094] Step 3, based on the vegetation index reference curve obtained in the historical stable feature time period, the vegetation index in this embodiment takes the normalized difference vegetation index (NDVI) as an example, migrates the spectral reference curve (2020-2022) to the time to be reconstructed in 2023, and reconstructs the vegetation index based on the EKF method, so that near real-time vegetation index reconstruction data can be obtained, such as Figure 2 shown.

[0095] Step 4: Using the vegetation index reconstruction result as a reference, calculate the deviation between the simulated NDVI and the actual NDVI as the credibility of the actual observation data, and use it as the current data weight of the EKF reconstruction framework of the multispectral data to be reconstructed. Reconstruction and correction can obtain near-real-time multispectral data.

[0096] Step 5: For the vegetation index and multispectral data reconstruction results obtained after the near-real-time reconstruction framework, the reconstruction effect of the multispectral data is evaluated from the spectrum and texture of the reconstructed data itself, and a total of 4 accuracy indicators in the spectrum and texture are selected for evaluation. The root mean square error (RMSE), mean absolute error (MAE), structural similarity (SSIM), and local binary pattern (LBP) are selected as the accuracy evaluation indicators for near-real-time remote sensing data reconstruction.

[0097] In this embodiment, cloud tag simulation data with real observation data areas is first constructed, such as Figure 5 As shown. Secondly, the accuracy is evaluated at the cloud tag simulation. In this embodiment, the root mean square error and mean absolute error are selected as two accuracy indicators to calculate the deviation between the original image observation value and the reconstructed image observation value in the cloud simulation area. The results are shown in Figure 6 As shown, Figure 6 (a) is the real image of the study area. Figure 6 (b) to (f) are respectively the reconstructed panoramic images of the study area obtained by using the method NRTR of the present application and the classical algorithms (SG, Whittaker, RTSR and DCT-PLS); Figure 6 (a1)~(f1) in the figure are the local details corresponding to (a)~(f) respectively; Figure 6 (b2) to (f2) are the differences between (b1) to (f1) and (a1), respectively, which represent the difference between the reconstruction result and the real image. Compared with the classic SG filtering method, the near-real-time remote sensing data reconstruction method NRTR of this application shows higher accuracy and better reconstruction effect.

[0098] This application, supported by migration models such as historical data and extended Kalman filtering, uses the ground object change detection algorithm (CCDC) and the harmonic wave function fitting (HANTS) algorithm to obtain the benchmark historical reference curve based on historical multispectral time series data. Secondly, the historical phenological curve information is migrated in combination with near-real-time images, and the credibility of the near-real-time vegetation feature observation data is estimated through the time series trend, and the credibility of the vegetation feature observation data is migrated as the observation credibility of the multi-band. Finally, the extended Kalman filter method is used to simulate the nonlinear change trend of vegetation characteristics, and the simulated data estimated by the migrated historical reference curve and the near-real-time observation data are assimilated, and finally the near-real-time vegetation index data and multispectral data are obtained, which improves the near-real-time observation capability of the entire growing season of crops, and is of great significance to the promotion of accurate identification of crops throughout the growing season, parameter inversion, and timely and accurate crop monitoring.

[0099] The near-real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season of the present application can utilize historical multispectral time series data for migration and reconstruct near-real-time multispectral time series data. Compared with the previous method that requires a long period of time (generally greater than 1 month) of data before and after the image to be estimated as a window for data reconstruction, the present application can better reconstruct multispectral data within 1 to 2 satellite revisit cycles after the image to be reconstructed, providing effective data support for accurate identification of crops throughout the growing season and parameter inversion.

[0100] Based on the same inventive concept, the embodiment of the present application also provides a device for realizing the near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more devices for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season provided below can be referred to the limitations of the method for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season above, and will not be repeated here.

[0101] In an exemplary embodiment, a near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season is provided, comprising:

[0102] The historical remote sensing image spectral data acquisition module is used to obtain the historical time-series remote sensing image spectral data of the target object type.

[0103] The spectral reference curve construction module is used to construct a spectral reference curve of the target land object type throughout the growing season by using a land object change detection algorithm and a harmonic wave function fitting algorithm based on the historical time series remote sensing image spectral data.

[0104] The vegetation index reference curve construction module is used to construct a vegetation index reference curve of the target object type throughout the growing season based on the spectral reference curve.

[0105] The vegetation index reconstruction module is used to reconstruct the vegetation index according to the vegetation index reference curve and the near real-time vegetation index data using the extended Kalman filter algorithm to obtain the vegetation index reconstruction result.

[0106] The remote sensing image spectral data reconstruction module is used to reconstruct the remote sensing image spectral data based on the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results, and use the extended Kalman filter algorithm to obtain the remote sensing image spectral data reconstruction results.

[0107] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for remote sensing image spectral data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a near-real-time reconstruction method for multi-spectral remote sensing time series images of crops throughout the growing season is realized.

[0108] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] In an exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the above-mentioned embodiment of the near-real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season are implemented.

[0110] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, which, when executed by a processor, implements the steps in the above-mentioned method embodiment for near-real-time reconstruction of multispectral remote sensing time series images of crops throughout their growing season.

[0111] In an exemplary embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in the above-mentioned embodiment of the near-real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season.

[0112] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0113] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, etc., but is not limited thereto.

[0114] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0115] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A near real-time reconstruction method for multispectral remote sensing time series images of crops throughout the growing season, characterized in that: include: Acquire historical time-series remote sensing image spectral data of target object types; Based on the historical time-series remote sensing image spectral data, a spectral reference curve of the full growing season of the target land object type is constructed by using a land object change detection algorithm and a harmonic wave function fitting algorithm; Based on the spectral reference curve, construct a vegetation index reference curve for the target object type throughout the growing season; According to the vegetation index reference curve and the near-real-time vegetation index data, the vegetation index is reconstructed using the extended Kalman filter algorithm to obtain the vegetation index reconstruction result; According to the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results, the extended Kalman filter algorithm is used to reconstruct the remote sensing image spectral data to obtain the remote sensing image spectral data reconstruction results.

2. The near real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season according to claim 1 is characterized in that: According to the historical time series remote sensing image spectral data, a spectral reference curve of the full growing season of the target object type is constructed by using a ground object change detection algorithm and a harmonic wave function fitting algorithm, specifically including: Using the ground object change detection algorithm, the time series remote sensing image spectral data of the whole growing season that meets the stability and credibility conditions are selected from the historical time series remote sensing image spectral data as the target time series remote sensing image spectral data; The spectral data of the target time series remote sensing image are fitted by using a harmonic function fitting algorithm to obtain the spectral reference curve.

3. The near real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season according to claim 2 is characterized in that: Using the ground object change detection algorithm, the time series remote sensing image spectral data of the whole growing season that meets the stability and credibility conditions are selected from the historical time series remote sensing image spectral data as the target time series remote sensing image spectral data, including: Dividing the historical time series remote sensing image spectral data into time series remote sensing image spectral data of multiple time periods according to the full growing season of the target land object type; Performing a change analysis on the time series remote sensing image spectral data of each time period to obtain a change vector of the time series remote sensing image spectral data of each time period; the change vector includes: change amplitude, change direction and change time; Determine the time series remote sensing image spectral data of the time period in which the change vector meets the preset change conditions as the target time series remote sensing image spectral data; the preset change conditions are that the change amplitude is within the preset amplitude range, the change direction is within the preset direction range, and the change time is within the preset time range.

4. The near real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season according to claim 2, characterized in that: The harmonic function fitting algorithm is used to fit the target time series remote sensing image spectral data to obtain the spectral reference curve, which specifically includes: Let the value of n be 1, and initialize the target time series remote sensing image spectral data to the time series remote sensing image spectral data after the 0th update; Perform the least square fitting on the time series remote sensing image spectral data updated for the n-1th time to obtain the fitting curve obtained by the nth fitting; The time series remote sensing image spectral data after the n-1th update is compared with the fitting curve obtained by the nth fitting, and the remote sensing image spectral data whose deviation distance from the fitting curve obtained by the nth fitting is greater than the distance threshold is removed to obtain the time series remote sensing image spectral data after the nth update, and the value of n is increased by 1, and the step of "performing a least square fitting on the time series remote sensing image spectral data after the n-1th update to obtain the fitting curve obtained by the nth fitting" is returned until the deviation of the target time series remote sensing image spectral data from the fitting curve obtained by the nth fitting is less than the deviation threshold, and the fitting curve obtained by the nth fitting is output as the spectral reference curve.

5. The near real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season according to claim 1, characterized in that: The vegetation index reconstruction result includes a posteriori estimated values ​​of the vegetation index at different moments in near real time and a posteriori estimated values ​​of the credibility of the vegetation index; According to the vegetation index reference curve and near real-time vegetation index data, the vegetation index is reconstructed using the extended Kalman filter algorithm to obtain the vegetation index reconstruction results, including: Solve the first state transition matrix according to the fitted values ​​of the vegetation index at the k and k-1 moments in the near real time in the vegetation index reference curve; According to the first state transition matrix and the observed value of the vegetation index at time k-1 in the near real-time vegetation index data, the vegetation index is estimated a priori using the following formula to obtain the a priori estimated value of the vegetation index at time k and the a priori estimated value of the vegetation index credibility; in, is the prior estimate of the vegetation index at time k, A is the first state transition matrix, B is the first control matrix, u k is the vegetation index control value at time k, is the posterior estimate of the vegetation index at time k-1, which is obtained based on the observed value of the vegetation index at time k-1 in the near-real-time vegetation index data and the prior estimate of the vegetation index at time k-1. is the prior estimate of the credibility of the vegetation index at time k, P k-1 is the posterior estimate of the credibility of the vegetation index at time k-1, Q is the first covariance matrix, and the superscript T indicates transposition; According to the prior estimation value of the vegetation index at time k, the prior estimation value of the vegetation index credibility and the observed value of the vegetation index at time k in the near real-time vegetation index data, the vegetation index is estimated a posteriori using the following formula to obtain the a posteriori estimation value of the vegetation index at time k and the a posteriori estimation value of the vegetation index credibility; Among them, K k is the first Kalman gain at time k, H is the measurement matrix that characterizes the relationship between the true value and the observed value of the vegetation index, R is the third covariance matrix, is the posterior estimate of the vegetation index at time k, Z k is the observed value of the vegetation index at time k in the near-real-time vegetation index data, P k is the posterior estimate of the credibility of the vegetation index at time k, and I is the unit matrix.

6. The near real-time reconstruction method of multispectral remote sensing time series images of crops throughout the growing season according to claim 1, characterized in that: According to the spectral reference curve, near-real-time remote sensing image spectral data and vegetation index reconstruction results, the extended Kalman filter algorithm is used to reconstruct the remote sensing image spectral data to obtain the remote sensing image spectral data reconstruction results, including: According to the fitting values ​​of the remote sensing image spectral data at the k and k-1 moments in near real time in the spectral reference curve, the second state transition matrix is ​​solved; According to the second state transition matrix and the observed value of the remote sensing image spectral data at time k-1 in the near real-time remote sensing image spectral data, the remote sensing image spectral data is estimated a priori using the following formula to obtain the a priori estimated value of the remote sensing image spectral data at time k and the a priori estimated value of the remote sensing image spectral data credibility; in, is the prior estimate of the spectral data at time k, A' is the second state transition matrix, B' is the second control matrix, u' k is the spectral control quantity at time k, is the posterior estimate of the remote sensing image spectral data at time k-1, is the prior estimate of the credibility of the remote sensing image spectral data at time k, P' k-1 is the posterior estimate of the credibility of the remote sensing image spectral data at time k-1, Q' is the second covariance matrix, and the superscript T indicates transposition; According to the prior estimation value of the remote sensing image spectral data at time k, the prior estimation value of the remote sensing image spectral data credibility and the observed value of the remote sensing image spectral data at time k in the near real-time remote sensing image spectral data, the remote sensing image spectral data is estimated a posteriori using the following formula to obtain the a posteriori estimation value of the remote sensing image spectral data at time k and the a posteriori estimation value of the remote sensing image spectral data credibility; P' k =P k ; Among them, K' k is the second Kalman gain at time k, H' is the measurement matrix that characterizes the relationship between the prior estimate and the observed value of the remote sensing image spectral data, R' is the fourth covariance matrix, is the posterior estimate of the remote sensing image spectral data at time k, Z' k is the observed value of the remote sensing image spectral data at time k in the near real-time remote sensing image spectral data, P' k is the posterior estimate of the credibility of the remote sensing image spectral data at time k, P k is the posterior estimate of the credibility of the vegetation index at time k.

7. A near real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season, characterized in that: The near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season is applied to the near-real-time reconstruction method for multispectral remote sensing time series images of crops throughout the growing season according to any one of claims 1 to 6, and the near-real-time reconstruction device for multispectral remote sensing time series images of crops throughout the growing season comprises: The historical remote sensing image spectral data acquisition module is used to obtain the historical time-series remote sensing image spectral data of the target object type; A spectral reference curve construction module is used to construct a spectral reference curve of the target ground object type throughout the growing season by using a ground object change detection algorithm and a harmonic wave function fitting algorithm based on the historical time series remote sensing image spectral data; A vegetation index reference curve construction module, used to construct a vegetation index reference curve of the target object type throughout the growing season based on the spectral reference curve; The vegetation index reconstruction module is used to reconstruct the vegetation index according to the vegetation index reference curve and the near-real-time vegetation index data using the extended Kalman filter algorithm to obtain the vegetation index reconstruction result; The remote sensing image spectral data reconstruction module is used to reconstruct the remote sensing image spectral data based on the spectral reference curve, near real-time remote sensing image spectral data and vegetation index reconstruction results, and use the extended Kalman filter algorithm to obtain the remote sensing image spectral data reconstruction results.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the near real-time reconstruction method for multispectral remote sensing time series images of crops throughout the growing season as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for near real-time reconstruction of multispectral remote sensing time series images of crops throughout the growing season according to any one of claims 1 to 6 is implemented.

Images