Remote sensing crop drawing-oriented systematic classification sample purification and expansion method

Abstract

The invention discloses a remote sensing crop drawing-oriented systematic classification sample purification and expansion method. The method comprises the steps of open source remote sensing crop classification product data set acquisition, data preprocessing and availability condition analysis, sample purification based on a multi-source data space-time filtering algorithm, and sample expansion in combination with remote sensing feature enhancement and self-supervised deep learning. According to the method, high-quality crop samples which are high in representativeness, sufficient in number and balanced in space and category distribution can be efficiently obtained on the basis of multi-source crop classification product data, sample data support is provided for remote sensing crop classification research and application, and compared with traditional crop sample obtaining methods such as field collection and visual interpretation, the method has the advantage that the method is more efficient. The method is more efficient, objective and automatic, compared with an existing sample generation method based on simple processing of remote sensing crop classification product data, the method is more comprehensive, systematic and universal, and the generated samples have more advantages in the aspects of accuracy, representativeness and balance.

Description

Technical Field

[0001] The invention relates to a sample purification and expansion method, in particular to a systematic classification sample purification and expansion method for remote sensing crop mapping, belonging to the technical field of crop remote sensing mapping. Background Art

[0002] Timely and accurate "one map" of crop distribution can provide a basic spatiotemporal information base for digital agriculture, precision agriculture and smart agriculture, and has a wide range of applications in crop planting area estimation, growth monitoring and yield estimation, and is of great significance for adjusting crop planting structure, optimizing agricultural production management, and ensuring national food security. Remote sensing technology has become the main means of crop distribution mapping due to its advantages such as macroscopicity, timeliness and accuracy.

[0003] With the continuous enrichment of remote sensing data sources and the continuous development of remote sensing information extraction technology, remote sensing crop mapping has gradually transitioned from traditional visual interpretation to data-driven machine learning and deep learning, and has continued to progress in the direction of "data-knowledge" dual drive. At present, statistical remote sensing index feature threshold method, machine learning and deep learning algorithms have become the mainstream methods of remote sensing crop mapping. In the process of constructing and implementing these methods, high-quality crop sample data is required as support without exception. The quality and quantity of crop samples have become important factors limiting the improvement of the accuracy and generalization ability of remote sensing crop classification model algorithms. How to efficiently obtain a large number of high-quality samples has become an unavoidable key issue in remote sensing crop mapping research.

[0004] Generally speaking, crop samples are often initially obtained through field sampling and remote sensing visual interpretation. However, field sampling requires a lot of manpower and material resources, and remote sensing visual interpretation is highly subjective, making it difficult to obtain a sufficient number of high-quality samples. In recent years, remote sensing big data and cloud computing have continued to develop, and many open-source large-area remote sensing crop classification product data have emerged, becoming an ideal source of sample data for subsequent remote sensing crop classification research and application. However, various crop classification data products have errors to a certain extent, and the temporal and spatial coverage and included crop categories are uneven, so they cannot be used directly as sample data and need to undergo a series of optimization processes. The existing technology usually processes existing crop classification product data in a rough way, often using simple set operations or statistical analysis strategies such as intersection, union, complement, and difference. The generated samples often have defects in accuracy, representativeness, and balance, which is not conducive to the smooth implementation of remote sensing crop classification tasks. Therefore, it is urgent to design a systematic sample purification and expansion method to make full use of the existing crop classification product data to generate high-quality samples with sufficient quantity, strong representativeness, and balanced categories and spatial distribution to meet the needs of subsequent remote sensing crop classification research and application. Summary of the invention

[0005] The present invention aims to solve the problems of small number, poor quality and difficulty in obtaining samples in the field of crop remote sensing mapping. In order to solve this problem, the present invention proposes a systematic classification sample purification and expansion method for remote sensing crop mapping to efficiently obtain a sufficient number of high-quality crop classification samples.

[0006] The present invention achieves the above-mentioned purpose through the following technical scheme: a systematic classification sample purification and expansion method for remote sensing crop mapping, the method specifically comprising the following steps:

[0007] Step 1: Obtain a dataset of open source remote sensing crop classification products;

[0008] Step 2: Data preprocessing and availability analysis based on the data set to analyze the coverage of crop classification product data in the region;

[0009] Step 3: Sample purification based on multi-source data spatiotemporal filtering algorithm: Using the multi-source data spatiotemporal filtering algorithm, sample purification is performed from three perspectives: space, time, and data source, to obtain high-confidence crop samples in spatiotemporally stable areas;

[0010] Step 4: Combine remote sensing feature enhancement with sample expansion using self-supervised deep learning. Use the purified crop samples (if there is crop sample data collected on the spot, it can also be added to complete sample expansion together with the purified crop samples). Based on the sliding time window remote sensing feature extraction, perform data enhancement by random perturbation, and use the self-supervised deep learning algorithm to assign category labels to the generated unlabeled samples to achieve crop sample expansion.

[0011] Step 5: Obtain high-quality remote sensing crop classification samples that are sufficient in number, representative, and balanced in spatial and category distribution.

[0012] As a further solution of the present invention: in step 3, the multi-source data spatiotemporal filtering algorithm for sample purification in different scenarios includes:

[0013] (1) Spatial filtering of a single data source and a single temporal phase: When there is only one single-temporal remote sensing crop classification product in a region, spatial filtering is performed on a single data source at a specific time point to remove noise and improve data quality.

[0014] (2) Time series filtering of multiple time phases from a single data source: In the case where there is a multi-temporal remote sensing crop classification product in the region, the product data from the same data source at different time points is used to remove short-term fluctuations and extract real and stable samples through time series analysis;

[0015] (3) Joint filtering among multi-source data: This is aimed at the situation where there are multiple remote sensing crop classification products in the region.

[0016] As a further solution of the present invention: based on the correlation between crop planting patterns in space and time, the multi-source data spatiotemporal filtering algorithm is expanded on the basis of the inverse distance weighted method, extended to the time dimension, and three-dimensional convolution filtering is used to extract high confidence areas of remote sensing crop classification data products;

[0017] The three-dimensional convolution filter includes two spatial dimensions and one time dimension. The size of the spatial dimension corresponds to the window size of the two-dimensional plane, which is used to adjust the spatial range of the convolution operation to consider the stability of the category label within a certain spatial range. The size of the time dimension corresponds to the number of years of crop classification product data, which can consider the stability of the category label in time. If the window size on the spatial plane is m×m, and t years of crop classification data are used, the structure of the three-dimensional convolution kernel is m×m×t. Its specific calculation expression is:

[0018]

[0019] Where: d i is the distance from the sample point (other points in the convolution kernel) i to the point to be estimated (the center point of the convolution kernel) x; p is a parameter used to set the distance type, which is set to "Euclidean distance" in this algorithm; Z(x) is the calculation result of the center point x to be estimated; Z i is the value of sample point i; w i is the weight of sample point i for estimating center point x; n is the number of sample points involved in the calculation.

[0020] As a further solution of the present invention: the multi-source data spatiotemporal filtering algorithm specifically includes the following steps during calculation:

[0021] First, the multi-temporal crop classification product data is read and integrated into a three-dimensional array consisting of two spatial dimensions and one time dimension;

[0022] Secondly, the data is convolved using the three-dimensional convolution kernel constructed using the improved IDW algorithm. During the convolution, the boundaries can be filled with 0 so that the pixels on the boundaries can participate in the calculation correctly.

[0023] Then, the spatiotemporal stability discrimination threshold is set for each crop category according to the actual situation; according to the category code of each crop, the convolution operation result of each crop under the most stable condition can be calculated, and the value range can be set according to a certain ratio;

[0024] Finally, determine whether the convolution operation result of each pixel in the original crop classification layer falls within the value range of spatiotemporal stability of the corresponding crop category; retain the spatiotemporal stable pixels within the value range and remove other pixels;

[0025] The above operations simultaneously realize spatial filtering of a single data source in a single phase and temporal filtering of multiple phases of a single data source; on this basis, the above process is repeated for crop classification product data from different sources to compare whether the results are consistent, and inconsistent areas are eliminated to achieve joint filtering among multi-source data; the raster data obtained by the above filtering is converted into vector data, and areas with smaller areas are screened and eliminated to finally obtain a high-confidence sample data set, completing the entire sample purification process.

[0026] As a further solution of the present invention: in step 4, sample expansion specifically includes the following steps:

[0027] (1) Sliding time window remote sensing feature extraction: Extracting time series remote sensing features through sliding time windows to capture the differences in crop remote sensing phenology in the region;

[0028] (2) Random perturbation enhancement of remote sensing feature vectors: The random perturbation algorithm of feature vectors is used to expand the remote sensing feature space of each type of crop samples, thereby capturing the intra-class differences in remote sensing features of the same crop.

[0029] (3) Sample expansion under the self-supervised deep learning strategy: Using the self-supervised deep learning strategy, on the basis of using the purified and enhanced samples for model pre-training, the unlabeled samples generated by fishing net sampling are assigned high-confidence category labels, and then the expanded high-confidence samples are added to the original labeled samples for model pre-training again. The above process is repeated until the samples can meet the requirements of spatial and crop category balance.

[0030] As a further solution of the present invention: sliding time window remote sensing feature extraction specifically includes the following steps:

[0031] First, collect macro-phenological information in the region to determine the time window corresponding to the entire crop growing season, and determine the step length of the time series synthesis of remote sensing feature data according to the actual situation in the region;

[0032] Secondly, based on the original growing season time window, the time axis is slid forward and backward by one step to take into account the differences in crop phenology within the region;

[0033] Finally, the original growing season time window and the two time windows obtained after sliding were used to extract the time series remote sensing features of crop samples. Each sample point corresponds to three high-dimensional remote sensing feature vectors, which enhances the phenological information of crop remote sensing features.

[0034] As a further solution of the present invention: random perturbation enhancement of remote sensing feature vectors is used to simulate intra-class differences of crop remote sensing features, and remote sensing features are enhanced from the dimension of feature value size; in view of the fact that the feature values ​​of crop remote sensing features may have a certain fluctuation range, Gaussian random functions are introduced to simulate the differences of crop features at various stages under natural conditions, specifically including the following steps:

[0035] First, the characteristic values ​​of each feature of each crop category were sorted in ascending order, and the largest and smallest outliers were removed using the percentile method;

[0036] Then, the mean and standard deviation of the feature data are calculated;

[0037] Finally, according to the calculation results, a number of data that conform to the Gaussian distribution (i.e. normal distribution) of crop sample remote sensing characteristics are generated to achieve random perturbation enhancement of feature vectors;

[0038] Assume that the characteristic value (random variable) X of the remote sensing characteristics of crops in a single phase follows a normal distribution with a mean of μ and a standard deviation of σ, that is, Its probability density function f(x) is expressed as:

[0039]

[0040] Among them: μ is the mean value of the remote sensing characteristics of each sample, which determines the center position of the distribution; σ is the standard deviation of the remote sensing characteristics of each sample, which determines the width of the distribution (the degree of dispersion of the distribution); σ 2 is the variance of the remote sensing characteristics of each sample, reflecting the degree of data fluctuation; x is the value of the remote sensing characteristic random variable;

[0041] If μ = 0 and σ = 1, the distribution is called the standard normal distribution, which can be simplified to:

[0042]

[0043] In summary, the random perturbation enhancement of remote sensing feature vectors is based on the assumption that the eigenvalues ​​of each remote sensing feature of crops in a single phase obey the standard normal distribution. The above-mentioned random perturbation enhancement of remote sensing feature vectors is performed to achieve the expansion of remote sensing feature values ​​under limited crop sample labels, so as to capture the intra-class differences of crops and enhance the information of crop remote sensing features in the eigenvalue dimension.

[0044] As a further solution of the present invention: sample expansion under the self-supervised deep learning strategy expands the number of crop sample points on the basis of purification and remote sensing feature enhancement to achieve the purpose of balanced distribution of crop samples in terms of space and crop categories. Specifically, it includes the following steps:

[0045] First, the region is divided into grids of a certain size and overlaid with the cultivated land range data to screen out the grids that intersect with the cultivated land range. A method for generating sample points based on a fishing net is used to generate a certain number of unlabeled sample points in each grid to ensure the balance of the spatial distribution of the sample points.

[0046] Secondly, based on the Transformer-Encoder deep learning model architecture, the sample label discrimination model is pre-trained using the purified and remote sensing feature enhanced labeled crop classification samples. It is applied to the label discrimination of crop classification samples. Its multi-head self-attention mechanism can focus on different feature dimensions of the data at the same time, ensuring that complex feature patterns are captured when generating label probabilities.

[0047] Then, the generated unlabeled sample points are used for remote sensing feature extraction, and the probability of each crop category label is input into the pre-trained model. The discrimination conditions of high-confidence labels are set according to the actual situation, and samples with high-confidence labels are screened out according to the probability threshold defined by the discrimination conditions.

[0048] Finally, the generated high-confidence samples are added to the original labeled samples, the model is pre-trained again, and the crop category labels of the regenerated unlabeled samples are determined again; the above process is repeated until the crop samples meet the usage requirements in terms of spatial distribution balance, crop category balance and quantity, so as to achieve the purpose of sample expansion.

[0049] The beneficial effects of the present invention are:

[0050] 1) The present invention designs a multi-source data spatiotemporal filtering algorithm, which can comprehensively consider the information of three dimensions of space, time and data source to extract stable areas, obtain high-confidence areas from crop classification products, and achieve the purpose of sample purification;

[0051] 2) The present invention proposes a sliding time window remote sensing feature extraction strategy and a remote sensing feature vector random perturbation enhancement strategy, which respectively realize data enhancement from the time dimension (crop phenology difference information) and the feature value dimension (crop feature category difference information), so that one sample label can correspond to multiple remote sensing crop features, thereby improving its ability to express crop features without changing the number of sample points, and improving the representativeness of samples when the sample labels are limited;

[0052] 3) The present invention constructs a sample expansion method under a self-supervised deep learning strategy to obtain crop samples with balanced spatial distribution and number of categories, trains a deep learning model based on labeled crop samples after purification and remote sensing feature enhancement, and discriminates the crop categories of unlabeled samples automatically generated by fishing nets under a self-supervised strategy. The identified high-confidence samples are continuously added to the model training to continue to assign category labels to the newly generated unlabeled samples, and after multiple cycles of iteration, the expanded samples can meet the needs of remote sensing crop classification research and application.

[0053] 4) By using the systematic remote sensing crop classification sample purification and expansion method proposed in the present invention, it is possible to efficiently obtain high-quality crop samples with strong representativeness, sufficient quantity, and balanced spatial and category distribution based on multi-source crop classification product data, providing sample data support for remote sensing crop classification research and application;

[0054] 5) Compared with traditional crop sample acquisition methods such as field collection and visual interpretation, this method is more efficient, objective and automated. Compared with the current sample generation method based on simple processing of remote sensing crop classification product data (such as set operations or statistical analysis such as intersection, union, complement and difference), this method is more comprehensive, systematic and universal, and the generated samples have more advantages in accuracy, representativeness and balance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a schematic diagram of the overall technical process of the present invention;

[0056] Figure 2 Schematic diagram of three-dimensional convolution filtering of the present invention;

[0057] Figure 3 This is a schematic diagram of the sliding time window of the present invention;

[0058] Figure 4 This is a schematic diagram of random disturbance enhancement of the present invention;

[0059] Figure 5 Schematic diagram of the self-supervised deep learning sample expansion process of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Examples, such as Figure 1As shown, a systematic classification sample purification and expansion method for remote sensing crop mapping, the systematic classification sample purification and expansion method includes:

[0062] First: Get a dataset of open source remote sensing crop classification products.

[0063] Second: Data preprocessing and availability analysis based on the data set to analyze the coverage of crop classification product data in the region.

[0064] Third: Sample purification based on multi-source data spatiotemporal filtering algorithm. Using multi-source data spatiotemporal filtering algorithm, sample purification is performed from three perspectives: space, time and data source to obtain high-confidence crop samples in spatiotemporally stable areas.

[0065] Among them, the multi-source data spatiotemporal filtering algorithm is used to purify samples in different scenarios, including:

[0066] (1) Spatial filtering of a single data source and a single temporal phase: When there is only one single-temporal remote sensing crop classification product in a region, spatial filtering is performed on a single data source at a specific time point to remove noise and improve data quality.

[0067] (2) Time series filtering of multiple time phases from a single data source: In the case where there is a multi-temporal remote sensing crop classification product in the region, the product data from the same data source at different time points is used to remove short-term fluctuations and extract real and stable samples through time series analysis;

[0068] (3) Joint filtering among multi-source data; for situations where there are multiple remote sensing crop classification products in the region.

[0069] The specific principles of the three types of sample deposits are as follows:

[0070] Based on the correlation between crop planting patterns in space and time, the multi-source data spatiotemporal filtering algorithm is extended on the basis of the Inverse Distance Weighting (IDW) method and extended to the time dimension. Three-dimensional convolution filtering (two spatial dimensions and one temporal dimension) is used to extract high confidence (high stability) areas of remote sensing crop classification data products.

[0071] The three-dimensional convolution filter includes two spatial dimensions and one temporal dimension. The size of the spatial dimension corresponds to the window size of the two-dimensional plane, which is used to adjust the spatial range of the convolution operation to consider the stability of the category label within a certain spatial range. The size of the temporal dimension corresponds to the number of years of crop classification product data, which can consider the stability of the category label over time.

[0072] If the window size on the spatial plane is m×m (in order to make the convolution kernel have a central value, m is usually an odd number and the size can be set according to the actual situation), and the crop classification data of t years is used, the structure of the three-dimensional convolution kernel is m×m×t (such as Figure 2 shown);

[0073] In practical applications, if it is necessary to purify the samples of a certain year, the center value of the year in the convolution kernel is set to a large positive integer to increase the label weight of this position. The weight values ​​of other positions in the convolution kernel are calculated based on the IDW method based on the above center position.

[0074] In the calculation process, the distance between adjacent pixels in space in the same year is set to 1, and the distance between adjacent years in time at the same location is also set to 1. The specific calculation expression is:

[0075]

[0076] Where: d i is the distance from the sample point (other points in the convolution kernel) i to the point to be estimated (the center point of the convolution kernel) x; p is a parameter used to set the distance type, which is set to "Euclidean distance" in this algorithm; Z(x) is the calculation result of the center point x to be estimated; Z i is the value of sample point i; w i is the weight of sample point i for estimating center point x; n is the number of sample points involved in the calculation.

[0077] The multi-source data spatiotemporal filtering algorithm specifically includes the following steps during calculation:

[0078] First, the multi-temporal crop classification product data is read and integrated into a three-dimensional array consisting of two spatial dimensions and one time dimension;

[0079] Secondly, the data is convolved using the three-dimensional convolution kernel constructed using the improved IDW algorithm, such as Figure 2 As shown, the border can be filled with 0 during convolution so that the pixels on the border can participate in the operation correctly;

[0080] Then, according to the actual situation, the spatiotemporal stability discrimination threshold is set for each crop category. According to the category code of each crop, the convolution operation result of each crop in the most stable case (that is, the crop category code corresponding to each position of the convolution kernel is the same) can be calculated, and the value range is set according to a certain ratio;

[0081] Finally, determine whether the convolution operation result of each pixel in the original crop classification layer falls within the value range of the spatiotemporal stability of the corresponding crop category; retain the spatiotemporal stable pixels within the value range and eliminate other pixels.

[0082] The above operations simultaneously realize spatial filtering of a single data source in a single phase and temporal filtering of multiple phases of a single data source; on this basis, the above process is repeated for crop classification product data from different sources to compare whether the results are consistent, and inconsistent areas are eliminated to achieve joint filtering among multi-source data; the raster data obtained by the above filtering is converted into vector data, and areas with smaller areas are screened and eliminated to finally obtain a high-confidence sample data set, completing the entire sample purification process.

[0083] Data enhancement is achieved from the time dimension (crop phenology difference information) and the feature value dimension (crop feature category difference information) respectively, so that one sample label can correspond to multiple remote sensing crop characteristics, improving its ability to express crop characteristics without changing the number of sample points, and improving the representativeness of samples when sample labels are limited.

[0084] Fourth: Combine remote sensing feature enhancement with sample expansion through self-supervised deep learning. Use purified crop samples (if there is crop sample data collected in the field, it can also be added to complete sample expansion together with the purified crop samples). Based on the remote sensing feature extraction of the sliding time window, perform data enhancement through random perturbation. Use a self-supervised deep learning algorithm to assign category labels to the generated unlabeled samples to achieve crop sample expansion.

[0085] Through sample expansion, the representativeness of sample data can be enhanced, and the number of samples can be made more balanced in terms of spatial distribution and crop categories, so as to reduce the risk of model overfitting in crop classification applications, improve the generalization ability of the model, and better serve remote sensing crop classification research.

[0086] Sample expansion specifically includes the following steps:

[0087] (1) Sliding time window remote sensing feature extraction: Extracting time series remote sensing features through sliding time windows to capture the differences in crop remote sensing phenology in the region;

[0088] (2) Random perturbation enhancement of remote sensing feature vectors: The random perturbation algorithm of feature vectors is used to expand the remote sensing feature space of each type of crop samples, thereby capturing the intra-class differences in remote sensing features of the same crop.

[0089] (3) Sample expansion under the self-supervised deep learning strategy: Using the self-supervised deep learning strategy, on the basis of using the purified and enhanced samples for model pre-training, the unlabeled samples generated by fishing net sampling are assigned high-confidence category labels, and then the expanded high-confidence samples are added to the original labeled samples for model pre-training again. The above process is repeated until the samples can meet the requirements of spatial and crop category balance.

[0090] Furthermore, sliding time window remote sensing feature extraction, considering that there may be certain differences in crop planting and growth and development time in the region, a single time window is difficult to fully describe the various situations of crop phenology in the region. Therefore, the present invention proposes a sliding time window method for remote sensing feature extraction, so that the sample features can capture the difference information of crop phenology in the study area, which specifically includes the following steps (such as Figure 3 shown):

[0091] First, collect macro-phenological information in the region to determine the time window corresponding to the entire crop growing season, and determine the step length of the time series synthesis of remote sensing feature data (usually 10-15 days, or it can be determined according to the quantile of the entire time window) according to the actual situation of the region (including remote sensing image coverage and crop growth rate, etc.);

[0092] Secondly, based on the original growing season time window, slide forward and backward along the time axis by one step to take into account the differences in crop phenology in the region (because not all crops are sown at the same time, there is a certain order of precedence);

[0093] Finally, the original growing season time window and the two time windows obtained after sliding were used to extract the time series remote sensing features of crop samples. Each sample point corresponds to three high-dimensional remote sensing feature vectors, which enhances the crop remote sensing features in terms of phenological information (time dimension).

[0094] Random perturbation enhancement of remote sensing feature vectors is used to simulate the intra-class differences of crop remote sensing features and enhance remote sensing features from the dimension of feature value size; given that the feature values ​​of crop remote sensing features may have a certain fluctuation range, Gaussian random functions are introduced to simulate the differences of crop features at various stages under natural conditions, specifically including the following steps (such as Figure 4 shown):

[0095] First, the characteristic values ​​of each feature of each crop category were sorted in ascending order, and the largest and smallest outliers were removed using the percentile method;

[0096] Secondly, calculate the mean and standard deviation of the feature data;

[0097] Finally, according to the calculation results, a number of data that conform to the Gaussian distribution (ie normal distribution) law of remote sensing characteristics of crop samples are generated to achieve random disturbance enhancement of feature vectors.

[0098] Assume that the characteristic value (random variable) X of the remote sensing characteristics of crops in a single phase follows a normal distribution with a mean of μ and a standard deviation of σ, that is, Its probability density function f(x) is expressed as:

[0099]

[0100] Among them: μ is the mean value of the remote sensing characteristics of each sample, which determines the center position of the distribution; σ is the standard deviation of the remote sensing characteristics of each sample, which determines the width of the distribution (the degree of dispersion of the distribution); σ 2 is the variance of the remote sensing characteristics of each sample, reflecting the degree of data fluctuation; x is the value of the remote sensing characteristic random variable;

[0101] If μ = 0 and σ = 1, the distribution is called the standard normal distribution, which can be simplified to:

[0102]

[0103] In summary, the random perturbation enhancement of remote sensing feature vectors is based on the assumption that the eigenvalues ​​of each remote sensing feature of crops in a single phase obey the standard normal distribution. The above-mentioned random perturbation enhancement of remote sensing feature vectors is performed to achieve the expansion of remote sensing feature values ​​under limited crop sample labels, so as to capture the intra-class differences of crops and enhance the information of crop remote sensing features in the eigenvalue dimension.

[0104] Although the crop samples after purification and remote sensing feature enhancement have high confidence and representativeness, they may not meet the requirements of balanced distribution of sample quantity in space and category. This is because feature enhancement only uses limited samples to enhance the representativeness and diversity of its remote sensing features, and does not change the spatial distribution and quantity of samples. Based on this, a sample expansion method under a self-supervised deep learning strategy is proposed.

[0105] Sample expansion under the self-supervised deep learning strategy expands the number of crop sample points based on purification and remote sensing feature enhancement to achieve a balanced distribution of crop samples in terms of space and crop categories. Specifically, it includes the following steps, such as Figure 5 As shown:

[0106] First, the region is divided into grids of a certain size and overlaid with the cultivated land range data to screen out the grids that intersect with the cultivated land range. A method for generating sample points based on a fishing net is used to generate a certain number of unlabeled sample points in each grid to ensure the balance of the spatial distribution of the sample points.

[0107] Secondly, based on the Transformer-Encoder deep learning model architecture, the sample label discrimination model is pre-trained using labeled crop classification samples after purification and remote sensing feature enhancement; Transformer-Encoder is the encoder part of the Transformer model architecture. Compared with traditional methods, it can process long sequence data more efficiently. It is applied to the label discrimination of crop classification samples. Its multi-head self-attention mechanism can focus on different feature dimensions of the data at the same time, ensuring that complex feature patterns are captured when generating label probabilities;

[0108] Then, the generated unlabeled sample points are used for remote sensing feature extraction, and the pre-trained model is input to discriminate the probability of each crop category label. The discrimination conditions of high-confidence labels are set according to the actual situation (for example, the probability value corresponding to the category label with the highest probability should be no less than twice the probability value of other category labels, etc.), and samples with high-confidence labels are screened out according to the probability threshold defined by the discrimination conditions;

[0109] Finally, the generated high-confidence samples are added to the original labeled samples, the model is pre-trained again, and the crop category labels of the regenerated unlabeled samples are determined again; the above process is repeated until the crop samples meet the usage requirements in terms of spatial distribution balance, crop category balance and quantity, so as to achieve the purpose of sample expansion.

[0110] Fifth: Obtain high-quality remote sensing crop classification samples that are sufficient in number, representative, and balanced in spatial and category distribution. Make full use of open source remote sensing crop classification product data sets, purify samples based on multi-source data spatiotemporal filtering algorithms, and expand samples by combining remote sensing feature enhancement with self-supervised deep learning. While ensuring sample purity, expand the sample quantity, improve the representativeness and balance of the samples, and better meet the needs of remote sensing crop classification research and application.

[0111] By using the systematic remote sensing crop classification sample purification and expansion method proposed in the present invention, it is possible to efficiently obtain high-quality crop samples with strong representativeness, sufficient quantity, and balanced spatial and category distribution based on multi-source crop classification product data, providing sample data support for remote sensing crop classification research and application. Compared with traditional crop sample acquisition methods such as field collection and visual interpretation, this method is more efficient, objective, and automated. Compared with the current sample generation method based on simple processing of remote sensing crop classification product data (such as set operations or statistical analysis such as intersection, union, complement, and difference), this method is more comprehensive, systematic, and universal, and the generated samples have more advantages in accuracy, representativeness, and balance.

[0112] Working principle: Analyze the coverage of crop classification product data in the area, and purify samples from three perspectives: space, time and data source based on a multi-source data spatiotemporal filtering algorithm to obtain high-confidence samples; use the purified crop samples (if there is crop sample data collected on the spot, it can also be added to complete sample expansion together with the purified crop samples), based on the sliding time window remote sensing feature extraction, perform data enhancement through random perturbations, and use a self-supervised deep learning algorithm to assign category labels to the generated unlabeled samples to achieve crop sample expansion, and obtain a high-quality sample set that is sufficient in number, representative, and balanced in category and spatial distribution.

[0113] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0114] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A systematic classification sample purification and expansion method for remote sensing crop mapping, characterized in that: The method for systematic classification sample purification and expansion comprises the following steps: Step 1: Obtain a dataset of open source remote sensing crop classification products; Step 2: Data preprocessing and availability analysis based on the data set to analyze the coverage of crop classification product data in the region; Step 3: Sample purification based on multi-source data spatiotemporal filtering algorithm: Using the multi-source data spatiotemporal filtering algorithm, sample purification is performed from three perspectives: space, time, and data source, to obtain high-confidence crop samples in spatiotemporally stable areas; Step 4: Combine remote sensing feature enhancement with sample expansion of self-supervised deep learning. Using the purified crop samples, based on the sliding time window remote sensing feature extraction, data enhancement is performed by random perturbation. The self-supervised deep learning algorithm is used to assign category labels to the generated unlabeled samples to achieve crop sample expansion. Step 5: Obtain high-quality remote sensing crop classification samples that are sufficient in number, representative, and balanced in spatial and category distribution.

2. The method for systematic classification sample purification and expansion according to claim 1, characterized in that: In step 3, the multi-source data spatiotemporal filtering algorithm for sample purification in different scenarios includes: (1) Spatial filtering of a single data source and a single temporal phase: When there is only one single-temporal remote sensing crop classification product in a region, spatial filtering is performed on a single data source at a specific time point to remove noise and improve data quality. (2) Time series filtering of multiple time phases from a single data source: In the case where there is a multi-temporal remote sensing crop classification product in the region, the product data from the same data source at different time points is used to remove short-term fluctuations and extract real and stable samples through time series analysis; (3) Joint filtering among multi-source data: This is aimed at the situation where there are multiple remote sensing crop classification products in the region.

3. The method for systematic classification sample purification and expansion according to claim 2, characterized in that: Based on the correlation between crop planting patterns in space and time, the multi-source data spatiotemporal filtering algorithm is extended on the basis of the inverse distance weighted method, extended to the time dimension, and uses three-dimensional convolution filtering to extract high confidence areas of remote sensing crop classification data products; The three-dimensional convolution filter includes two spatial dimensions and one temporal dimension; wherein the size of the spatial dimension corresponds to the window size of the two-dimensional plane, and is used to adjust the spatial range of the convolution operation to consider the stability of the category label within the spatial range; the size of the temporal dimension corresponds to the number of years of the crop classification product data, and can consider the temporal stability of the category label; If the window size on the spatial plane is m×m, and crop classification data for t years is used, the structure of the three-dimensional convolution kernel is m×m×t; its specific calculation expression is: Where: d i is the distance from the sample point i to the estimated point x; p is a parameter used to set the distance type, which is set to "Euclidean distance" in this algorithm; Z(x) is the calculation result of the estimated center point x; Z i is the value of sample point i; w i is the weight of sample point i for estimating center point x; n is the number of sample points involved in the calculation.

4. The method for systematic classification sample purification and expansion according to claim 2, characterized in that: The multi-source data spatiotemporal filtering algorithm is specifically calculated in The following steps are involved: First, the multi-temporal crop classification product data is read and integrated into a three-dimensional array consisting of two spatial dimensions and one time dimension; Secondly, the data is convolved using the three-dimensional convolution kernel constructed using the improved IDW algorithm. During the convolution, the boundaries are filled with 0 so that the pixels on the boundaries can participate in the calculation correctly. Then, according to the actual situation, a spatiotemporal stability discrimination threshold is set for each crop category. According to the category code of each crop, the convolution operation result of each crop under the most stable condition is calculated, and the value range is set according to the preset ratio. Finally, determine whether the convolution operation result of each pixel in the original crop classification layer falls within the value range of the spatiotemporal stability of the corresponding crop category, retain the spatiotemporal stable pixels within the value range, and eliminate other pixels; The above operations simultaneously realize spatial filtering of a single data source in a single phase and temporal filtering of multiple phases of a single data source; on this basis, the above process is repeated for crop classification product data from different sources to compare whether the results are consistent, and inconsistent areas are eliminated to achieve joint filtering among multi-source data; the raster data obtained by the above filtering is converted into vector data, and areas with smaller areas are screened and eliminated to finally obtain a high-confidence sample data set, completing the entire sample purification process.

5. The method for systematic classification sample purification and expansion according to claim 1, characterized in that: In step 4, sample expansion specifically includes the following steps: (1) Sliding time window remote sensing feature extraction: Extracting time series remote sensing features through sliding time windows to capture the differences in crop remote sensing phenology in the region; (2) Random perturbation enhancement of remote sensing feature vectors: The random perturbation algorithm of feature vectors is used to expand the remote sensing feature space of each type of crop samples, thereby capturing the intra-class differences in remote sensing features of the same crop. (3) Sample expansion under the self-supervised deep learning strategy: Using the self-supervised deep learning strategy, on the basis of using the purified and enhanced samples for model pre-training, the unlabeled samples generated by fishing net sampling are assigned high-confidence category labels, and then the expanded high-confidence samples are added to the original labeled samples for model pre-training again. The above process is repeated until the samples can meet the requirements of spatial and crop category balance.

6. The method for systematic classification sample purification and expansion according to claim 5, characterized in that: The sliding time window remote sensing feature extraction specifically includes the following steps: 1) Collect macro-phenological information in the region to determine the time window corresponding to the entire crop growing season, and determine the step size of the time series synthesis of remote sensing feature data according to the actual situation in the region; 2) Based on the original growing season time window, slide forward and backward along the time axis by one step to take into account the differences in crop phenology in the region; 3) The original growing season time window and the two time windows obtained after sliding are used to extract the time series remote sensing features of crop samples. Each sample point corresponds to three high-dimensional remote sensing feature vectors, which enhances the phenological information of crop remote sensing features.

7. The method for systematic classification sample purification and expansion according to claim 5, characterized in that: The random disturbance enhancement of remote sensing feature vectors is used to simulate the intra-class differences of crop remote sensing features and enhance remote sensing features from the dimension of feature value size. In view of the fluctuation range of feature values ​​of crop remote sensing features, the Gaussian random function is introduced to simulate the differences of crop features at various stages under natural conditions, which specifically includes the following steps: First, the characteristic values ​​of each feature of each crop category were sorted in ascending order, and the largest and smallest outliers were removed using the percentile method; Then, the mean and standard deviation of the feature data are calculated; Finally, according to the calculation results, a number of data that conform to the Gaussian distribution law of crop sample remote sensing characteristics are generated to achieve random perturbation enhancement of feature vectors; Assume that the characteristic value X of the remote sensing characteristic of a single phase of crop follows a normal distribution with a mean of μ and a standard deviation of σ, that is, Its probability density function f(x) is expressed as: Where: μ is the mean value of the remote sensing feature of each sample; σ is the standard deviation of the remote sensing feature of each sample; σ 2 is the variance of the remote sensing characteristics of each sample; x is the value of the remote sensing characteristic random variable; If μ = 0 and σ = 1, then the distribution is called the standard normal distribution, which simplifies to: In summary, the random perturbation enhancement of remote sensing feature vectors is based on the assumption that the eigenvalues ​​of each remote sensing feature of crops in a single phase obey the standard normal distribution. The above-mentioned random perturbation enhancement of remote sensing feature vectors is performed to achieve the expansion of remote sensing feature values ​​under limited crop sample labels, so as to capture the intra-class differences of crops and enhance the information of crop remote sensing features in the eigenvalue dimension.

8. The method for systematic classification sample purification and expansion according to claim 5, characterized in that: The sample expansion under the self-supervised deep learning strategy expands the number of crop sample points on the basis of purification and remote sensing feature enhancement to achieve a balanced distribution of crop samples in terms of space and crop categories. The specific steps include: First, the region is divided into grids and overlaid with the cultivated land range data to screen out the grids that intersect with the cultivated land range. A fishing net-based sample point space generation method is used to generate a certain number of unlabeled sample points in each grid to ensure the balance of the spatial distribution of the sample points. Secondly, based on the Transformer-Encoder deep learning model architecture, the sample label discrimination model is pre-trained using the purified and remote sensing feature-enhanced labeled crop classification samples, and then applied to the label discrimination of crop classification samples. Its multi-head self-attention mechanism focuses on different feature dimensions of the data at the same time, ensuring that complex feature patterns are captured when generating label probabilities. Then, the generated unlabeled sample points are used for remote sensing feature extraction, and the probability of each crop category label is input into the pre-trained model. The discrimination conditions of high-confidence labels are set according to the actual situation, and samples with high-confidence labels are screened out according to the probability threshold defined by the discrimination conditions. Finally, the generated high-confidence samples are added to the original labeled samples, the model is pre-trained again, and the crop category labels of the regenerated unlabeled samples are identified again. The above process is repeated until the crop samples meet the usage requirements in terms of spatial distribution balance, crop category balance and quantity, so as to achieve the purpose of sample expansion.

Images