Crop classification method and device based on time sequence remote sensing image, medium and equipment

Abstract

The invention discloses a crop classification method and device based on a time sequence remote sensing image, a medium and equipment. The method comprises the following steps: acquiring an original remote sensing time sequence image of a to-be-classified crop; preprocessing the original remote sensing time sequence image to obtain a preprocessed original remote sensing time sequence image; normalizing the preprocessed original remote sensing time sequence image to obtain a normalized remote sensing time sequence image; performing time-spectrum feature space conversion on the normalized remote sensing time sequence image to obtain a time-spectrum feature space map; constructing a crop classification model, and performing training; and inputting the time-spectrum feature space diagram into a trained crop classification model so as to classify the to-be-classified crops. According to the invention, the classification precision and robustness can be improved.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing satellite technology, and in particular relates to a crop classification method, device, medium and equipment based on time-series remote sensing images. Background Art

[0002] Crop mapping is the process of monitoring agricultural land to obtain information about crop types and their spatial distribution. Currently, satellite remote sensing is the most widely used crop mapping method. Its principle is that crops have clear phenology, unique seasonal changes, and sowing and harvesting dates. As crops grow and develop, vegetation canopy structure, pigment type and concentration, etc. will undergo periodic changes, which contribute to the changes in spectral characteristics during growth. Therefore, effectively characterizing the temporal information characteristics of crops is of great value for crop identification. Traditional methods are mainly based on hand-made time series features designed based on relevant agronomic knowledge. Not only does it rely heavily on a deep understanding of the target crop, but it also takes a lot of time and effort to design and verify effective features. From the current needs of crop mapping, the recognition accuracy of this method can no longer meet current expectations. Data-driven deep learning technology has demonstrated the ability to automatically extract deep features for specific tasks from input data. This technology can achieve or even exceed the classification accuracy of traditional methods without in-depth knowledge of the target crop.

[0003] The motivation for better feature extraction has led to the continuous updating of deep learning models with different principles and architectures. The increase in model complexity has led to higher sampling requirements: sufficiently representative samples are needed to avoid model overfitting due to a large number of parameters, and to have robust performance when reasoning over large areas, reducing the impact of crop phenology and growth differences caused by environmental factors and human practices. However, extensive field sampling is time-consuming and labor-intensive, and it is impractical to obtain sufficient and representative samples when mapping large areas. In addition, the lack of surface observations (especially key phenological periods) due to cloud and haze pollution exacerbates the difficulty of crop identification. Therefore, it is necessary to propose a new method for crop identification.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0005] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a crop classification method, device, medium and equipment based on time-series remote sensing images, which can improve the accuracy of crop classification.

[0006] To achieve the above objectives, the present disclosure provides the following technical solutions:

[0007] A crop classification method based on time-series remote sensing images comprises the following steps: obtaining original remote sensing time-series images of crops to be classified; preprocessing the original remote sensing time-series images to obtain preprocessed original remote sensing time-series images; normalizing the preprocessed original remote sensing time-series images to obtain normalized remote sensing time-series images; performing time-spectrum feature space conversion on the normalized remote sensing time-series images to obtain a time-spectrum feature space graph;

[0008] A crop classification model is constructed and trained; the time-spectrum feature space graph is input into the trained crop classification model to classify the crop to be classified. Optionally, the original remote sensing time series image is preprocessed, including the following steps: cloud masking the original remote sensing time series image; median synthesis of the original remote sensing time series image after cloud masking; linear interpolation of the original remote sensing time series image after median synthesis.

[0009] Optionally, the normalizing the preprocessed original remote sensing time series images includes:

[0010] The preprocessed original remote sensing time series images are respectively subjected to spatial normalization and temporal normalization.

[0011] Optionally, the normalized remote sensing time series image is converted into a time-spectrum feature space, comprising the following steps: constructing a time-spectrum feature space; rasterizing the time-spectrum feature space to generate a grid time-spectrum feature space; and stacking the grid time-spectrum feature space to generate a time-spectrum feature space map of pixels.

[0012] Optionally, the crop classification model includes: a feature extraction module, used to extract features from the time-spectrum feature space graph; and a category prediction module, used to predict the category of the crop based on the features extracted by the feature extraction module.

[0013] The present disclosure also provides a crop classification device based on time-series remote sensing images, including: an acquisition module, used to acquire original remote sensing time-series images of crops to be classified; a preprocessing module, used to preprocess the original remote sensing time-series images to obtain preprocessed original remote sensing time-series images; a normalization module, used to normalize the preprocessed original remote sensing time-series images to obtain normalized remote sensing time-series images; a conversion module, used to perform time-spectrum feature space conversion on the normalized remote sensing time-series images to obtain a time-spectrum feature space map; a model construction and training module, used to construct a crop classification model and perform training; a classification module, used to input the time-spectrum feature space map into the trained crop classification model to classify the crops to be classified.

[0014] The present disclosure also provides a storage medium comprising instructions, which, when executed on a computer, enables the computer to execute the method as described in any one of the preceding items.

[0015] The present disclosure also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] The present invention generates two types of time-spectrum feature space graphs through time normalization and space normalization processing, which effectively enhances the discrimination of crop characteristics, reduces the influence of environmental factors, reduces intra-class differences, and increases inter-class differences. Using these two types of time-spectrum feature space graphs as inputs for subsequent models helps the model learn the complementary information provided by these two types of data, thereby better classifying crop types and improving classification accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flowchart of a crop classification method based on time-series remote sensing images provided by an embodiment of the present disclosure;

[0019] Figure 2 is a structural schematic diagram of a crop classification model provided by another embodiment of the present disclosure;

[0020] Figure 3 is a schematic diagram of an original remote sensing time series image and a preprocessed image provided by another embodiment of the present disclosure, wherein part (a) is an original remote sensing time series image, part (b) is an original remote sensing time series image after cloud masking; part (c) is an original remote sensing time series image after median synthesis; part (d) is an original remote sensing time series image after linear interpolation;

[0021] Figure 4 It is a normalized remote sensing time series image after time-spectrum feature space conversion provided by another embodiment of the present disclosure, wherein part (a) is a preprocessed remote sensing time series image, part (b) is a spatially normalized remote sensing time series image, part (c) is a temporally normalized remote sensing time series image, part (d) is a second type of time-spectrum feature space map, and part (e) is a first type of time-spectrum feature space map;

[0022] Figure 5 is a schematic diagram of a time-spectrum feature space provided by another embodiment of the present disclosure;

[0023] Figure 6is a schematic diagram of crop distribution in different regions provided by another embodiment of the present disclosure, wherein (A) is the Gannan region, (B) is the Hailun region, (C) is the Deyang region, and (D) is the Zhongjiang region;

[0024] Figure 7 (a) is a schematic diagram of classification results of the present disclosure and the existing method in a region of Gannan provided by another embodiment of the present disclosure;

[0025] Figure 7 (b) is a schematic diagram of classification results of the present disclosure and the existing method in another area of ​​Gannan provided by another embodiment of the present disclosure;

[0026] Figure 8 (a) is a schematic diagram of classification results of the present disclosure and the existing method in an area of ​​Helun provided by another embodiment of the present disclosure;

[0027] Figure 8 (b) is a schematic diagram of classification results of the present disclosure and the existing method in another area of ​​Helun provided by another embodiment of the present disclosure;

[0028] Fig. 9 (a) is a schematic diagram of classification results of the present disclosure and the existing method in a region of Deyang provided by another embodiment of the present disclosure;

[0029] Fig. 9 (b) is a schematic diagram of classification results of the present disclosure and the existing method in another area of ​​Deyang provided by another embodiment of the present disclosure;

[0030] Fig.10 It is a schematic diagram of the classification results of the present disclosure and the existing method in a certain area of ​​Zhongjiang provided by another embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The specific embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although the specific embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0032] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present disclosure, but the description is based on the general principles of the specification and is not used to limit the scope of the present disclosure. The scope of protection of the present disclosure shall be determined by the attached claims.

[0033] To facilitate understanding of the embodiments of the present disclosure, further explanation will be given below using specific embodiments as examples in conjunction with the accompanying drawings, and each of the accompanying drawings does not constitute a limitation on the embodiments of the present disclosure.

[0034] Figure 1 A crop classification method based on time-series remote sensing images is provided by an exemplary embodiment of the present disclosure. Figure 1 As shown, the method comprises the following steps:

[0035] S100: Acquire original remote sensing time series images of crops to be classified;

[0036] S200: preprocessing the original remote sensing time series image to obtain a preprocessed original remote sensing time series image;

[0037] S300: normalizing the preprocessed original remote sensing time series image to obtain a normalized remote sensing time series image;

[0038] S400: performing time-spectrum feature space conversion on the normalized remote sensing time series image to obtain a time-spectrum feature space graph;

[0039] S500: Build a crop classification model and perform training;

[0040] S600: Inputting the time-spectrum feature space graph into the trained crop classification model to classify the crop to be classified.

[0041] In another embodiment, in step S200, preprocessing the original remote sensing time series image includes the following steps:

[0042] S201: Yes Figure 3The original remote sensing time series images shown in part (a) (the original remote sensing time series images refer to a series of images taken continuously over the same area at different time points by remote sensing platforms such as satellites or aircraft. These images record information on changes in surface cover (such as crops) over time, including but not limited to vegetation growth conditions, soil moisture, and changes in surface features caused by seasonal climate change, etc.) are cloud masked;

[0043] In this step, because clouds and fog will block the true situation of the surface, the quality of the acquired remote sensing images will be reduced, which will in turn affect the temporal feature analysis of the surface type. Therefore, it is necessary to identify and process the pixels affected by clouds and fog in the original remote sensing time series images through cloud masks. Remote sensing images usually contain quality bands specifically used to indicate cloud coverage. These quality bands can effectively distinguish between clouds and the ground, thereby identifying pixels blocked by clouds or fog and marking these pixels as null values. Through cloud mask processing, the influence of clouds and fog in remote sensing images can be effectively removed, ensuring that the images used in subsequent analysis are closer to the actual surface conditions and providing a clear surface view for subsequent processing.

[0044] S202: Yes Figure 3 The original remote sensing time series image after the cloud mask shown in part (b) is subjected to median synthesis;

[0045] In this step, in order to further reduce the omission of clouds and fog by the remote sensing image quality band and improve the image quality, it is necessary to convert the original remote sensing time series image after cloud mask into a time series remote sensing image with equal time intervals through median synthesis. Specifically, for each pixel in the original remote sensing time series image after cloud mask, all valid pixel values ​​(non-null values) are selected within a fixed time interval, and the median of these valid values ​​is calculated as the final pixel value of the pixel at that time point. Through median synthesis, the omission of clouds and fog by the remote sensing image quality band can be further reduced to ensure that clouds and fog have been fully identified and processed. In addition, using the median as the final pixel value can effectively exclude accidental cloud shadows or outliers, maintain the true situation of surface reflection, and thus improve the overall quality of the image.

[0046] S203: Yes Figure 3 The original remote sensing time series image after median synthesis shown in part (c) is linearly interpolated.

[0047] In this step, after the median synthesis process, there are still some pixels that do not have valid values ​​at certain time points (that is, the pixels are affected by clouds and fog in the corresponding time period), and subsequent feature analysis cannot be performed. Therefore, in order to make all pixels in the remote sensing image after the median synthesis process have a complete valid time series, it is necessary to fill the invalid values ​​in the pixel time series through linear interpolation, that is, for each pixel that does not have a valid value at a certain time point, use the valid values ​​of its adjacent time points for linear interpolation to estimate the pixel value at that time point. In this way, the missing values ​​in the time series can be filled so that each pixel has a value at each time point, and finally the following can be obtained. Figure 3 The remote sensing time series image shown in part (d) is more continuous in time, which helps to better perform dynamic monitoring and change detection and other analyses.

[0048] In another exemplary embodiment, in step S300, the normalizing of the preprocessed original remote sensing time series image includes: normalizing the preprocessed original remote sensing time series image (such as Figure 4 As shown in part (a) above, spatial normalization (SN) and temporal normalization (TN) are performed to obtain spatially normalized remote sensing time series images and temporally normalized remote sensing time series images, respectively.

[0049] In this embodiment, the time normalization refers to normalizing the time sequence of a single pixel in a certain band (or image channel) on a time series remote sensing image by the maximum and minimum values ​​of the time sequence. For a certain pixel time sequence value p, its time normalization value at time t is:

[0050]

[0051] Among them, p(t) represents the original time series value of a single pixel in a certain band at time point t, max(p) represents the maximum value of a single pixel in the band during the entire observation period, min(p) represents the minimum value of a single pixel in the band during the entire observation period, and p normalized (t) represents the normalized time series value.

[0052] Since crops are affected by conditions such as water, heat, and light, different growth conditions will occur even in the same type of crops, which will cause differences in the absolute values ​​of the crop's spectral time series. Temporal normalization scales the maximum and minimum values ​​of the crop's spectral time series and maps these time series values ​​to a unified scale range. In this way, even if the spectral values ​​of crops under different conditions are different, their relative change trends and shapes can still be retained. In addition, through temporal normalization, the spectral characteristics of crops become more consistent, which helps to simplify the subsequent feature extraction process, so that the crop classification model does not need to spend too many resources to learn the spectral changes caused by environmental factors, but instead focuses more on the inherent spectral characteristics of crops.

[0053] The spatial normalization means that for a single-band image (or single-channel image) of a time series remote sensing image at a certain time point, the single-band image is normalized by the maximum and minimum values ​​of the entire image. For a certain pixel's time series value p, its spatial normalization value at time t is:

[0054]

[0055] Among them, p normalized (x, y, t) represents the semantic reflectivity value, p(x, y, t) represents the original reflectivity value of a single pixel at time point t and position (x, y), min t (p) represents the minimum reflectance value of the band in the entire image at time point t, max t (p) represents the maximum reflectance value of this band in the entire image at time point t.

[0056] Although different types of crops belong to the same category of green vegetation, they still have subtle differences in spectral characteristics. Through spatial normalization processing, these subtle differences can be highlighted, thereby enhancing the spectral differences between different types of crops, that is, increasing the differences between classes. At the same time, factors such as atmospheric conditions, sun position and time will have a significant impact on remote sensing images, which may cause large differences between images of the same crop taken at different times. Spatial normalization can reduce the impact of these external factors and ensure that images taken at different times and conditions in the same area are consistent. Through spatial normalization, the spectral differences between different types of crops can be enhanced, thereby improving the accuracy and discrimination of the crop classification model for crop classification.

[0057] In another exemplary embodiment, in step S400, performing time-spectrum feature space conversion on the normalized remote sensing time series image includes the following steps:

[0058] S401: constructing time-spectrum feature space;

[0059] In this step, for a normalized remote sensing time series image with m bands and T time series length, the time-spectral feature space map corresponding to each pixel is generated by combining any two bands. In the time-spectrum feature space generated by any two bands b1 and b2, the coordinates of the pixel at time t are: in, and Respectively represent the normalized reflectance values ​​of bands b1 and b2 at time t. Through this transformation, the original spectral time series is converted into a set of spatial scattered points in the time-spectral feature space.

[0060] S402: gridding the time-spectrum feature space to generate a grid time-spectrum feature space.

[0061] In this step, the constructed time-spectrum feature space is divided into r×r grids, where each grid can be regarded as a pixel in the image, and the pixel value represents the number of points distributed in this grid. This gridded time-spectrum feature space is called a gridded time-spectrum feature space, where each grid represents a small area, which contains the number of all time-spectrum feature points in the area. Such a rasterization operation helps to discretize continuous spatial data, thereby better utilizing computing resources in subsequent processing and capturing subtle differences between different crop types.

[0062] S403: stacking all possible grid time-spectrum feature spaces to obtain a time-spectrum feature space map corresponding to each pixel.

[0063] This step allows the crop classification model to understand the characteristics of crops from multiple perspectives, including the variations of the same crop under different environments and the inherent differences between different crops, thereby enabling a more comprehensive identification of crops.

[0064] In summary, by implementing steps S401 to S403, a remote sensing time series image normalized with time can be obtained for each pixel (such as Figure 4 The first type of time-spectrum feature space diagram (as shown in part (c) of Figure 4 (e)) and the original remote sensing time series image after spatial normalization (as shown in Figure 4 The second type of time-spectrum feature space diagram (as shown in part (b) of Figure 4The two types of time-spectral feature space maps are derived from two types of normalized remote sensing time series images, and feature enhancement is performed from the perspectives of intra-class and inter-class differences. Through these two normalization methods, the crop classification model can understand the characteristics of crops from the following two perspectives: on the one hand, understanding the changes of the same crop under different environments, and on the other hand, distinguishing the inherent differences of different crops. This complementary information enables the crop classification model to more comprehensively identify crops, thereby better classifying crops.

[0065] In another exemplary embodiment, in step S500, Figure 2 As shown, the crop classification model includes:

[0066] A feature extraction module, used for extracting features from the time-spectrum feature space graph;

[0067] A category prediction module is used to predict the crop category based on the features extracted from the time-spectral feature space graph by the feature extraction module.

[0068] In this embodiment, the feature extraction module includes a first branch and a second branch, and the first branch and the second branch have the same structure, wherein the first branch is used to extract features from a first type of time-spectrum feature space graph; and the second branch is used to extract features from a second type of time-spectrum feature space graph. Exemplarily, taking the first branch as an example, the first branch includes a first convolution block CB1 and a second convolution block CB2, and a permutation attention module SA is provided between the first convolution block CB1 and the second convolution block CB2. Specifically, the first convolution block CB1 is used to extract primary semantic features from the first type of time-spectrum feature space graph, and the primary semantic features can be processed by the permutation attention module SA to obtain enhanced primary semantic features, and the second convolution block CB2 uses the result of multiplying the enhanced primary semantic features with the primary semantic features as input, and further extracts high-level semantic features.

[0069] It should be noted that the first convolution block CB1 consists of two convolutional layers with 64 and 128 convolution kernels respectively, and the kernel size is set to 7. Such a setting allows the feature extraction module to capture the overall shape and spatial distribution characteristics of the crop spectrum from a larger receptive field, wherein the large-size convolution kernel helps to identify a wider range of patterns in the image and helps to understand the changes within the crop growth cycle.

[0070] The second convolution block CB2 consists of 3 convolutional layers, with 64, 32 and 16 convolution kernels respectively, and the size is set to 3. Among them, the use of smaller convolution kernels is to extract features more finely and gradually increase the level of abstraction. As the depth of the network increases, the number of convolution kernels gradually decreases, which helps to extract more advanced semantic features in the later stages of the network. The second convolution block CB2 takes the result of multiplying the primary semantic features enhanced by the SA module with the original primary semantic features as input, which not only retains the details of the original features, but also combines the emphasized important features, so as to further extract richer high-level semantic features.

[0071] The working principle of the replacement attention module SA is described as follows:

[0072] Feature grouping (Group): For the primary semantic features extracted by the first convolutional block CB1, SA is first divided into g groups along the channel dimension. This grouping strategy allows the crop classification model to obtain specific semantic information from each group and can reduce the amount of computation. Subsequently, each group is further divided into two subgroups along the channel dimension, one subgroup is used to generate deep features weighted by channel attention, and the other subgroup is used to generate deep features weighted by spatial attention.

[0073] Attention feature enhancement: Specifically, channel attention focuses on determining which channel is more important, and the channel attention weighted features are obtained by embedding the global information of each channel of the feature into the original channel by multiplication through global average pooling (GAP). Spatial attention focuses on the "position" of each channel of the feature containing important information. Spatial attention weighted features are obtained by embedding spatial information into the features through group normalization. Then, the two types of weighted features are connected along the channel dimension to obtain a joint weighted feature with the same number of channels as the input feature.

[0074] Feature aggregation (Agg): After connecting all joint weighted features along the channel dimension to obtain the aggregated features, a channel permutation operation (Shuffle Feature, SF) is applied. This operation ensures that subgroups that originally belonged to the same group are evenly divided into new and different groups, so that information can flow between different groups.

[0075] The category prediction module includes three branches, wherein the first branch includes a first fully connected layer, the second branch includes an overlay layer and a second fully connected layer, and the third branch includes a third fully connected layer. The first fully connected layer corresponds to the first branch of the feature extraction module, and the semantic features extracted by the first branch of the feature extraction module are used as inputs of the first fully connected layer to output a first auxiliary category prediction c sn , the first auxiliary category prediction c snIt can help the training process to update the model parameters more effectively; the third fully connected layer corresponds to the second branch of the feature extraction module, and the semantic features extracted by the second branch of the feature extraction module are used as the input of the third fully connected layer to output the second auxiliary category prediction c tn At the same time, the semantic features extracted by the first branch and the second branch of the feature extraction module are superimposed by the superposition layer as the input of the second fully connected layer to output the final category prediction c final .

[0076] The design idea of ​​the category prediction module is that the inputs of the first branch and the third branch are time-spectrum feature space graphs obtained by two normalization methods, respectively. The high-level semantic features obtained by the feature extraction module from these two time-spectrum feature space graphs have their own emphasis when making category predictions. In the model training stage, the model parameters are iteratively updated through the auxiliary category prediction of the first branch and the third branch, so that the two types of high-level semantic features extracted from the preamble of the model have the best prediction ability. Moreover, by using two different feature extraction paths (i.e., the first branch and the second branch), the model can analyze the input data from multiple perspectives, which helps to capture more comprehensive features to improve the recognition accuracy. Here, it should be emphasized that the auxiliary predictions provided by the first branch and the third branch can not only serve as additional supervisory signals to help adjust the model weights, but also prevent the occurrence of overfitting to a certain extent. When the main prediction has problems, these auxiliary predictions can also be used as a correction mechanism. Subsequently, these two types of high-level semantic features are used as the input of the second branch together. The second branch can make full use of the complementary information between the two sets of features by superimposing the two sets of features and then making classification decisions, thereby enhancing the model's adaptability to complex situations.

[0077] In summary, the category prediction module based on the above structural design can not only improve the accuracy of crop classification tasks, but also enhance the ability of the crop classification model to process complex and diverse remote sensing images, thereby ensuring reliable identification of different crop types throughout the growing season.

[0078] In another exemplary embodiment, in step S500, the crop classification model is trained by the following steps:

[0079] S501: construct a crop remote sensing time series image dataset, and divide it into a training set and a validation set in proportion, for example, the division ratio may be 7:3;

[0080] S502: Setting training parameters, for example, a learning rate of 0.001, a batch size of 32, a maximum number of training times of 100, training the model using the training set, and using a cross entropy loss function to calculate the loss of the current batch until the loss function converges;

[0081] In this step, the loss function is expressed as: l = l sn +l tn +l final =CE(c sn ,c)+CE(c tn ,c)+CE(c final , c)

[0082] Among them, CE represents the cross entropy loss function, l sn , l tn , l final Represent the prediction results of the first branch, the third branch and the second branch respectively, c sn ,c sn and c final They represent the loss values ​​relative to the ground truth c respectively.

[0083] S503: Use the validation set to validate the trained model, and use precision, recall, F1 score and mAP as evaluation indicators to validate the performance of the model. When the evaluation indicators all reach 90%, the model is validated. Otherwise, adjust the training parameters (for example, the learning rate is adjusted to 0.01, the batch size is adjusted to 64, and the number of training times is adjusted to 200) or expand the training samples to retrain the model until the model is validated.

[0084] In a specific embodiment, the present disclosure is based on this model and five classic models such as RF (Random Forest), 1DCNN (One-Dimensional Convolutional Neural Network), Transformer, DCM (Dynamic CotpMapping), Transformer, etc. Figure 6 The classification experiments were conducted on different crop regions in the four regions shown in FIG. (where (A) is the Gannan region of Heilongjiang, (B) is the Hailun region of Heilongjiang, (C) is the Deyang region of Sichuan, and (D) is the Zhongjiang region of Sichuan). The classification results of the four regions are as follows: Figures 7 to 10 Specifically, in Figure 7 In the crop remote sensing image schematic diagram of Gannan shown in FIG. , the present disclosure shows the highest recognition accuracy. Figure 7In (a), compared with the true label GT, the performance of the present disclosure is the best, only a few pixels are misclassified as rice, and the high consistency of pixel identification within the plot is maintained. For other comparison methods, except for 1DCNN, there is a problem of over-identification of rice, among which Transformer's over-estimation is the most significant. In addition, all comparison methods have inconsistencies in pixel identification within corn and soybean plots, among which 1DCNN even misclassifies almost all soybean plots as other categories. In another plot such as Figure 7 In the rice-dominated area shown in (b), except for 1DCNN which misclassified some rice pixels as soybeans, all other methods successfully identified rice pixels. However, for the identification of corn and soybean pixels, all methods except the present disclosure had obvious misclassification and omission problems. Figure 8 The differences between the different methods are even more obvious in the two local crop remote sensing images of Helen shown in the figure. Figure 8 In the highlighted area in (a), the contrasting methods have obvious omission and misclassification problems for soybeans and corn: Transformer and 1DCNN over-identify soybeans and corn; RF encounters challenges in addressing the heterogeneity within the corn plots, misclassifying a large number of corn pixels as other categories. Figure 8 In (b), according to the true label (GT), all the comparison methods over-identified rice in this area, especially DCM. In contrast, the performance of the present disclosure in these local areas is more robust and accurate. In Deyang area, the main crops grown are wheat and rapeseed. Fig. 9 (a) and Fig. 9 As shown in (b), in the results produced by all the comparison methods, a large number of wheat pixels are incorrectly identified as rapeseed or other categories. This shows that these methods have certain difficulties in handling crop classification in heterogeneous areas. Fig.10 In the Zhongjiang area shown, DCM showed over-identification of wheat pixels, Transformer misclassified large fields in the northwest as other types, and RF significantly underestimated the area of ​​wheat fields in the central hilly area. Specifically, DCM extracted far more wheat pixels than the actual number; Transformer's results not only failed to maintain the recognition consistency within the wheat field, but also misclassified large areas of wheat pixels as rapeseed; RF performed the worst, with low recognition accuracy for both wheat and rapeseed; in contrast, the results of 1DCNN were comparable to those of the present disclosure, but there were still errors in some details, such as misidentifying a pond in the northeast as wheat.

[0085] The above significant advantages of the method described in the present disclosure come from the following three points:

[0086] 1. Explicit representation of the synchronous dynamic relationship of paired bands: Each channel of the time-spectral feature space diagram is obtained by transforming the spectral time series of the paired bands. This transformation combines the temporal information of the crops on the paired bands at each time point and more effectively presents the interaction and correlation between the dynamic changes of the bands. In addition, presenting the spectral time series in one-dimensional form in two-dimensional form can display and analyze the distribution pattern of information as a whole. At the same time, since two bands are taken into account when extracting features, the noise in a single direction can be weakened by the information in the other direction, thereby reducing the instability of the features.

[0087] 2. Significantly enhanced crop separability: The time-spectral feature space graph significantly enhances the separability of crops by removing the order information from the original spectral time series, spatially gridding the spectral time series, and an effective normalization strategy. Although the removal of order information will lead to a certain degree of information loss, it can effectively overcome the intra-class differences of the same crop due to growth differences. On the other hand, the operation of gridding the gridded time-spectral feature space can tolerate feature jitter within the grid range, which corresponds to the numerical differences in crop reflectance due to growth differences and helps to reduce intra-class differences. Finally, two novel normalization strategies are proposed from the perspective of enhancing inter-class differences and reducing intra-class differences, which further enhance the separability between crops.

[0088] 3. CNN-learnable image-based time-spectrum features: By converting the multi-band spectral time series of crops into multi-channel images, the original time-spectrum features are converted into spatial features. This allows CNN, which is good at mining spatial features, to give full play to its advantages. Since the time-spectrum feature space graph already has a good ability to distinguish crops, combined with the powerful feature extraction ability of CNN and the feature enhancement ability of the attention module, it ultimately contributed to the outstanding performance of the public method in various experimental areas.

[0089] Furthermore, the present disclosure evaluates the classification results of each of the above models based on three indicators: Kappa (Cohen's Kappa), OA (Overall Accuracy), and Mean F1 (the harmonic mean of precision and recall). The specific evaluation results are shown in Table 1.

[0090] Table 1

[0091]

[0092] Depend on Figures 7 to 10 As can be seen from Table 1, compared with the existing classical models, the crop classification model proposed in the present disclosure is more effective in crop classification.

[0093] In another exemplary embodiment, the present disclosure also provides a crop classification device based on time-series remote sensing images, including: an acquisition module, used to acquire original remote sensing time-series images of crops to be classified; a preprocessing module, used to preprocess the original remote sensing time-series images to obtain preprocessed original remote sensing time-series images; a normalization module, used to normalize the preprocessed original remote sensing time-series images to obtain normalized remote sensing time-series images; a conversion module, used to perform time-spectrum feature space conversion on the normalized remote sensing time-series images to obtain a time-spectrum feature space map; a model construction and training module, used to construct a crop classification model and perform training; a classification module, used to input the time-spectrum feature space map into the trained crop classification model to classify the crops to be classified.

[0094] In another exemplary embodiment, the present disclosure further provides a storage medium, which includes instructions, and when the instructions are executed on a computer, the computer executes the method as described in any of the preceding items.

[0095] In another exemplary embodiment, the present disclosure further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the program.

[0096] The above embodiments are only for illustrating the technical concept and features of the present application, and their purpose is to enable people familiar with the technology to understand the content of the present application and implement it accordingly, and they cannot be used to limit the protection scope of the present application. Any equivalent changes or modifications made according to the spirit of the present application should be included in the protection scope of the present application.

Claims

1. A crop classification method based on time series remote sensing images, characterized in that: The method comprises the following steps: Obtaining original remote sensing time-series images of crops to be classified; Preprocessing the original remote sensing time series images to obtain preprocessed original remote sensing time series images; Normalizing the preprocessed original remote sensing time series image to obtain a normalized remote sensing time series image; Performing time-spectrum feature space conversion on the normalized remote sensing time series image to obtain a time-spectrum feature space map; Build and train crop classification models; The time-spectral feature space graph is input into the trained crop classification model to classify the crop to be classified.

2. The method according to claim 1, characterized in that Preprocessing the original remote sensing time series image includes the following steps: Performing cloud masking on the original remote sensing time series image; Performing median synthesis on the original remote sensing time series images after cloud masking; Linear interpolation is performed on the original remote sensing time series image after median synthesis.

3. The method according to claim 1, characterized in that Normalizing the preprocessed original remote sensing time series image comprises the following steps: The preprocessed original remote sensing time series images are respectively subjected to spatial normalization and temporal normalization.

4. The method according to claim 1, characterized in that: Performing time-spectrum feature space conversion on the normalized remote sensing time series image to obtain a time-spectrum feature space map includes the following steps: Constructing time-spectrum feature space; rasterizing the time-spectrum feature space to generate a gridded time-spectrum feature space; The grid time-spectrum feature space is stacked to generate a pixel-level time-spectrum feature space map.

5. The method according to claim 1, characterized in that: The crop classification model includes: A feature extraction module, used for extracting features from the time-spectrum feature space graph; A category prediction module is used to predict the crop category based on the features extracted from the time-spectral feature space graph by the feature extraction module.

6. The method according to claim 1, characterized in that The crop classification model is trained by the following steps: Construct a crop remote sensing time series image dataset and divide it into a training set and a validation set in proportion; Setting training parameters, iteratively training the crop classification model using the training set, and using a cross entropy loss function to calculate the loss of the current batch until the loss function converges; The trained model is verified using the validation set, and the performance of the model is verified using precision, recall, F1 score, and mAP as evaluation indicators. When all of the evaluation indicators reach 90%, the model is verified. Otherwise, the training parameters are adjusted or the training samples are expanded to retrain the model until the model is verified.

7. A crop classification device based on time-series remote sensing images, characterized in that: The device comprises: An acquisition module is used to acquire original remote sensing time-series images of crops to be classified; A preprocessing module, used for preprocessing the original remote sensing time series images to obtain preprocessed original remote sensing time series images; A normalization module is used to normalize the preprocessed original remote sensing time series image to obtain a normalized remote sensing time series image; A conversion module, used for performing time-spectrum feature space conversion on the normalized remote sensing time series image to obtain a time-spectrum feature space map; Model building and training module, used to build crop classification models and conduct training; The classification module is used to input the time-spectrum feature space graph into the trained crop classification model to classify the crop to be classified.

8. A storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

Images