A method and device for ground object classification of multi-temporal remote sensing data

Abstract

The application belongs to the technical field of remote sensing data processing, and provides a ground object classification method and device for multi-element time-series remote sensing data. The ground object classification method comprises the following steps: S1: acquiring multi-element time-series remote sensing data; S2: inputting the multi-element time-series remote sensing data into an MSSTAN algorithm to output ground object classification and probability; wherein the MSSTAN algorithm comprises: an AM-GTU module, which is used to extract spectral and time-series information at different scales through a multi-scale 1D convolution layer; an encoding layer, which is used to focus on the relationship between different time steps in the sequence through a multi-head attention mechanism and capture long-term dependencies within the sequence; a Residual-Gate KAN module, which is used to aggregate multi-level context features; and a classifier, which is used to output ground object classification and probability. The MSSTAN algorithm proposed by the ground object classification method realizes the dual extraction of local detail features and pixel global relationships on the basis of maintaining the original advantages, and realizes accurate land cover classification of multi-element time-series remote sensing data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing data processing, and in particular to a ground object classification method and device for multi-element time series remote sensing data. BACKGROUND

[0002] Time series data has been widely used in many fields due to its rich time dimension information. In the field of remote sensing, time series data is a continuous observation sequence composed of ground object spectral information and phase information, which can effectively reflect the spectral variation characteristics of ground objects in the time dimension. In recent years, with the increasing richness of open source remote sensing satellite data, the availability of time series data has significantly improved, which provides an important foundation for the application of time series classification (TSC) in the field of remote sensing data mining.

[0003] To achieve accurate classification of ground objects, researchers have proposed various TSC algorithms based on machine learning. TSC aims to build a machine learning model to predict the class label of a continuous and ordered real-valued observation sequence. Since multi-element time series remote sensing data belongs to high-dimensional data, which contains features of multiple bands, it is difficult to achieve high precision using existing methods for multi-element time series ground object classification. Therefore, applying multi-element time series remote sensing data for ground object classification is a complex process, and there are many reasons that affect the classification accuracy, such as feature selection of multi-element data, selection of different ground object feature classification methods, etc.

[0004] Traditional machine learning methods (such as DTW, RF, SVM) still have limitations in sequence feature discrimination and mining ability, and have limited ability to extract time series and spectral features, making it difficult to capture complex spatio-temporal patterns. Existing deep learning models (such as CNN, Bi-LSTM) have problems such as insufficient local context capture and loss of detailed information when processing high-dimensional time series remote sensing data. Therefore, the existing methods and models all have the problem of inaccuracy in multi-element time series ground object classification, especially for land cover classification tasks with unbalanced time series data samples in remote sensing, the above problems are more prominent, and it is difficult to meet the actual demand. SUMMARY

[0005] Therefore, the embodiments of the present application provide a ground object classification method and device for multi-element time series remote sensing data, which is based on MSSTAN to improve the model's attention to details and global dependencies, solve the inaccuracy problem of existing models in multi-element time series ground object classification, and realize accurate land cover classification of multi-element time series remote sensing data.

[0006] In a first aspect, the embodiments of the present application provide a ground object classification method for multi-element time series remote sensing data, including the following steps:

[0007] S1: acquire multi-element time sequence remote sensing data, the multi-element time sequence remote sensing data include multi-band spectral characteristics and time sequence;

[0008] S2: input the multi-element time sequence remote sensing data into MSSTAN algorithm (multi-scale spectral time attention network), output ground object target classification and probability;

[0009] Wherein, the MSSTAN algorithm includes AM-GTU module, coding layer, Residual-Gate KAN module and classifier;The AM-GTU module includes multi-scale 1D convolution layer and gate unit layer, the multi-scale 1D convolution layer is used to extract spectral and time sequence information at different scales, and the information import proportion is controlled through the gate unit layer;The coding layer pays attention to the relationship between different time steps in the sequence through multi-head attention mechanism, and captures long-term dependence within the sequence;The Residual-Gate KAN module includes KAN module and linear network layer, KAN module is used to extract local features, the spline in KAN module is accurate for low-dimensional function and local adjustment, and the linear network layer can well reserve and transmit global feature information to extract global features, and the global features reserved by the linear network layer are input to the gate unit together with the local adjustment results obtained by the KAN module through residual connection, so that the gate unit adjusts the extraction proportion of local and global features;The classifier is used to output ground object target classification and probability.

[0010] The above embodiment has the following technical effects: the existing Transformer model directly inputs data to the encoder, which leads to sparse expression of local details of spectrum and time sequence. The ground object classification method fuses important spectral and time sequence information at different scales through the improved multi-scale gate Tanh unit (AM-GTU) to extract the input data, pays more attention to the local context information of spectrum and time sequence in the sample after fusion, and can effectively improve the classification accuracy. In view of the problem that the existing Transformer linear layer is limited in learning subtle pattern ability from high-dimensional data set, the ground object classification method proposes a Residual-Gate KAN module combining KAN and residual gate network, replaces the linear mapping process with a learnable nonlinear mapping process, and effectively reduces the high-dimensional information loss problem. In summary, the MSSTAN method proposed by the ground object classification method realizes the dual extraction of local detail features and global relationships of pixels on the basis of maintaining the original advantages of the Transformer, and realizes the accurate classification of land cover of multi-element time sequence remote sensing data.

[0011] According to a specific implementation mode of the embodiment of the application, the multi-scale 1D convolution layer includes 1D causal convolution kernels of 5 scales, which are 1x3, 1x5, 1x7, 1x9 and 1x12 respectively.

[0012] According to a specific implementation of an embodiment of the present application, the gating unit layer of the AM-GTU module controls the information transmission ratio using the activation functions Sigmoid and Tanh, and then performs residual connection with the original input. The Sigmoid and Tanh gating units are used to screen important local features, and the residual connection is used to alleviate the feature loss problem of the deep network.

[0013] According to a specific implementation of an embodiment of the present application, the encoding layer includes an input embedding layer, a position encoding layer, a multi-head attention layer, a cross-layer and normalization layer, a feedforward network layer, and a cross-layer and normalization layer. The input embedding layer and the position encoding layer perform encoding operations on the input features, and then train the input features through the multi-head attention mechanism in the Transformer, constantly optimize the matching degree of the classification result and the true value, and thus output a more accurate classification result map.

[0014] According to a specific implementation of an embodiment of the present application, the linear network layer includes a linear layer, a ReLU activation function, and a BN normalization layer.

[0015] According to a specific implementation of an embodiment of the present application, the gating unit in the Residual-Gate KAN module adopts a sigmoid function. The fusion ratio of local and global features is dynamically adjusted.

[0016] According to a specific implementation of an embodiment of the present application, the classifier is a linear layer, which maps the feature dimension from d_model / / 2 to the number of categories.

[0017] In a second aspect, an embodiment of the present application provides a ground object classification device for multi-element time sequence remote sensing data, which comprises:

[0018] The acquisition module is configured to acquire multi-element time sequence remote sensing data, wherein the multi-element time sequence remote sensing data comprises multi-band spectral features and time sequences.

[0019] The detection module is configured to input the multi-element time sequence remote sensing data into the MSSTAN algorithm, and output ground object classification and probability.

[0020] The MSSTAN algorithm comprises an AM-GTU module, an encoding layer, a Residual-Gate KAN module and a classifier; the AM-GTU module comprises a multi-scale 1D convolution layer and a gate unit layer, the multi-scale 1D convolution layer is used to extract spectral and time sequence information at different scales, and the gate unit layer is used to control the incoming proportion of information; the encoding layer is used to focus on the relationship between different time steps in the sequence through a multi-head attention mechanism, and capture long-term dependencies within the sequence; the Residual-Gate KAN module is used to aggregate multi-level context features, and comprises a KAN module and a linear network layer, the KAN module is used to extract local features, and the linear network layer is used to extract global features, and the extraction proportions of the local and global features are adjusted through a gate unit; and the classifier is used to output the classification and probability of the ground object target. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual scale.

[0022] Figure 1 A flowchart of a ground object classification method for multi-element time sequence remote sensing data provided by an embodiment of the present application is shown;

[0023] Figure 2 A general framework diagram of the MSSTAN algorithm in an embodiment of the present application is shown;

[0024] Figure 3 A structural schematic diagram of the AM-GTU module in an embodiment of the present application is shown;

[0025] Figure 4 A structural schematic diagram of the Residual-Gate KAN module in an embodiment of the present application is shown;

[0026] Figure 5 A local visual classification result schematic diagram of different methods in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] The embodiments of the technical solutions of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, and cannot limit the protection scope of the present application.

[0028] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in the present application should be understood as the general meanings understood by the skilled person in the field to which the present application belongs.

[0029] Figure 1 A step flow chart of a method for ground object classification of multi-temporal remote sensing data according to an embodiment of the present application is shown in FIG. 1. The method comprises the following steps: Figure 1

[0030] S1: Obtain multi-temporal remote sensing data, which contains multi-band spectral characteristics and time series;

[0031] In this embodiment, the land use public dataset in the TiSeLaC challenge is used for instance description. The TiSeLaC dataset is collected from the 2866*2633 pixel size 2A level Landsat8 satellite image of Reunion Island taken at a spatial resolution of 30m in 2014, 23 scenes in a year, a total of 10 bands, each pixel is composed of 10 channel features: the first 7 bands containing original data (Band1-Band7 in Landsat8), representing the measurement value of each independent multi-spectral band (OLI): ultra blue, blue, green, red, NIR, SWIR1 and SWIR2; also including three complementary radiation indexes (normalized vegetation index, normalized water index and brightness index). The specific Landsat 8 band information and the three constructed vegetation indexes are shown in Table 1 and Table 2.

[0032]

[0033]

[0034] The organizers of TiSeLaC obtained 99687 pixels by sampling the public dataset, and referred to the Corine land cover map in 2012 and the local farmer's land cover registration results in 2014, and divided the land cover types of the study area of Reunion Island into 9 categories, including 81714 pixels of training set and 17973 pixels of test set, the ratio of each class of the training set and the test set is about 4:1, as shown in Table 3 below.

[0035]

[0036] S2: Input the multi-temporal remote sensing data into the MSSTAN algorithm, and output the ground object classification and probability;

[0037] ​The MSSTAN algorithm is a multi-scale spectral temporal attention network based on Transformer. In the multi-temporal classification task in the field of land use, only the Encoder encoding layer of MSSTAN participates in classification, and the Decoder decoding layer is discarded. This is because the Encoder can focus on the relationship between different time steps in the sequence through the self-attention mechanism, and capture the long-term dependencies within the sequence. Since the goal of the classification task is usually to model or feature extract the entire time series, rather than to predict the next time step or generate a new sequence, the Decoder part does not need to play a role in the classification task.

[0038] As shown in Figure 2 , the MSSTAN algorithm includes an AM-GTU module, an encoding layer, a Residual-Gate KAN module, and a classifier.

[0039] In the multi-temporal classification task, the input multi-temporal remote sensing data is subjected to feature extraction by the AM-GTU module (Advanced Multi-Scale Gated Tanh Unit). On the basis of preserving important local information, the enhanced module enhances the ability to extract global features. The improved AM-GTU module structure diagram is shown in Figure 3 . By stacking multiple GTUs, the time dimension can be expanded to improve the model's ability to extract long-term temporal correlations in the data. In this embodiment, the dimension of the input data is (10, 1, 23), representing 10 band features and 23 time dimensions for each sample. Each sample is input to the next gate unit layer through 1x3, 1x5, …, 1x12 such as five layers of multi-scale 1D convolutional layers. The gate unit layer uses the activation functions Sigmoid and Tanh unit to control the proportion of information passing through the multi-scale data, and according to the cross-entropy loss function, the feature part that is more beneficial to the classification task is selected and the original input is connected in residual, and finally the output obtained after the AM-GTU module is consistent with the dimension of the input, which is still (10, 1, 23).

[0040] The features extracted by the AM-GTU module are input into the encoding layer, which sequentially passes through the input embedding layer (Input Embedding), the position encoding layer (Positional Encoding), the multi-head attention layer (Multi-Head Attention), the cross-layer and normalization layer (Add&Norm), the feed-forward network layer (Feed Forward), and the cross-layer and normalization layer (Add&Norm), and repeats the process of multi-head attention layer to the last cross-layer and normalization layer for N rounds. The input embedding layer and the position encoding layer perform encoding operations on the input features, and then train the input features through the multi-head attention mechanism in the Transformer, constantly optimize the matching degree of the classification results and the true values, and output more accurate classification result map.

[0041] After the features pass through the encoding layer, the Residual-Gate KAN module is used to extract local and global features, and the results obtained by residual connection and KAN are finally input into the sigmoid gate unit. It is worth noting that the input of the Residual-Gate KAN module is the result obtained by the Encoder encoding layer. The structure diagram of the Residual-Gate KAN module is as shown in Figure 4 The input is not only transmitted to the KAN module, but also to the linear layer (Linear), the activation function (ReLU), the linear layer (Linear), and the normalization layer (BN, BatchNorm). The final activation function Sigmoid acts as a gate unit to dynamically adjust the importance of information of the two lines to adjust the extraction ratio of local and global feature key information.

[0042] The classifier is a linear layer (Linear) that maps the feature dimension of the fused features of the Residual-Gate KAN module to the number of categories, and outputs the classification and probability of the ground object target in combination with the Softmax function.

[0043] It needs to be further explained that:

[0044] 1. The MSSTAN algorithm improves the local relevance of the context:

[0045] In the AM-GTU module, 5 different scale 1D causal convolutions are used for feature extraction of the multi-element time series input X. Since the importance of the feature interval of each scale in the wave band or time step is different, the multi-scale features extracted by the 5 1D causal convolutions cannot be directly spliced, and the proportion of information flow to the next module needs to be controlled by the gate unit to retain important local information.

[0046] 2. The MSSTAN algorithm dynamically filters important local and global information:

[0047] Residual-Gate KAN can effectively aggregate multi-level context features. Replacing the single linear layer in the traditional Transformer network with Residual-Gate KAN, on the one hand, KAN module is different from the classic MLP network with fixed activation function on the neuron node, and KAN has a learnable weight activation function on the edge. On the other hand, Residual-Gate KAN adds a new linear network layer to the KAN module, which includes a linear layer, a ReLU activation function and a BatchNorm normalization layer. The linear network layer can well preserve and pass on the global feature information. Finally, the result obtained by the residual connection and KAN is finally input to the sigmoid gating unit to adjust the extraction ratio of local and global feature important information.

[0048] The MSSTAN algorithm model training steps are as follows:

[0049] 1. Set the random number seed, set the same seed so that the pre-training weight does not change every time the training is performed, and the training result can be reproduced every time. In addition, the hyperparameter settings required to complete the MSSTAN model are set.

[0050] 2. According to the data form of the public data set, construct your own training data loader and test data loader. Facilitate subsequent access to relevant information about the data set from the DataLoader.

[0051] 3. Create MSSTAN model. The original training data in DataLoader has shape (B, T, C), where B is batch size, C = 10 is the number of bands, and T = 23 is the time step. The input data X is reshaped to (B, C, 1, T) after adjustment and then passed through the improved AM-GTU module. This module uses 5 different scales of 1D causal convolution for feature extraction on the multivariate time series input X, controls the proportion of information flow to the next module through the gating unit to retain important local information. Then the information of multiple scales is spliced in the time dimension, and a linear layer is used to restore and input the same shape. In order to alleviate the problem of important shallow feature loss caused by the deep structure of the AM-GTU module, the last residual connection and ReLU activation function are used as the output of the module. After the multi-head attention mechanism of the Transformer, the obtained feature input is constructed in the Residual-Gate KAN. Here, the Residual-Gate KAN is a new linear network layer added in the KAN module, which contains a linear layer, a ReLU activation function, and a BatchNorm normalization layer. The linear network layer can well preserve and transmit global feature information. The Residual-Gate KAN here also serves as a dimension reduction means to integrate more important features for classification results to improve the final classification accuracy.

[0052] Here the dataset is the public TiSeLaC multivariate time series remote sensing dataset, and the above shows the core part of the MSSTAN algorithm applied to the TiSeLaC feature classification task and capable of improving the classification accuracy. In a wider feature classification scenario, only the corresponding dataset under the scenario needs to be constructed, specifically:

[0053] 1. First, download the corresponding LandSat or Sentinel-2 satellite images for the corresponding year through the Google Earth Engine platform as the time series image dataset for subsequent extraction of category data features.

[0054] 2. Next, select the label value of N classes of land by visual interpretation or according to the public ground truth dataset (such as the Dynamic World land use public dataset), and generate shp files for each class. Note that fuse into one shp face vector, otherwise create the same number of random points for each face vector when creating random points. Then calculate the proportion of each class and create a reasonable number of random points.

[0055] 3. Finally, implement multi-value extraction to points through Python scripts to extract time series features of random points of different categories from the time series image dataset. In this way, a complete land cover dataset is constructed.

[0056] Thus, a universal batch generation method of land cover data set is realized according to different research areas, and then the generated data set is input into the MSSTAN algorithm for classification, and the flow classification target is realized. Experiments prove that this universal method can be applied to land classification tasks in different scenarios, such as land classification, tree classification, etc., and good results have been achieved.

[0057] The MSSTAN algorithm model training method is as follows:

[0058] 1. Position encoding;

[0059] The improved AM-GTU module is input into the Encoder of the Transformer for training. The features input into the Encoder are first position encoded to let the model understand the position (order) of each word in the sequence to help the model learn this information.

[0060] 2. Multi-head attention mechanism;

[0061] The input features with position order are processed through the multi-head attention mechanism. Here, the size of q, k and v is set to be the same as the number of heads, which is set to 8. And 8 layers of such Encoders are stacked, each layer having its own parameters and weights, so that the model can capture features of different depths through stacking. The feature shape after multi-head attention mechanism processing changes from the initial input x(B, C, T) to x(B, d_model*C), where d_model is set to 512. Here, d_model is used as the linear mapping parameter of embedding, which maps the time step T in the input to a high d_model in the Encoder stage.

[0062] 3. Residual-Gate KAN;

[0063] The feature x(B, d_model*C) obtained after the multi-head attention mechanism in the Encoder is then processed through the Residual-Gate KAN module constructed by the application, and the shape changes to output(B, d_model / / 2). On the one hand, the KAN module replaces the linear mapping process with a learnable nonlinear mapping process, reducing the problem of high-dimensional information loss, and on the other hand, the network design of residual and sigmoid gate unit can adjust the extraction ratio of local and global feature important information.

[0064] 4. Classifier;

[0065] At the end of the model, the high-dimensional d_model / / 2 in the output is mapped to the final 9 classes through a linear layer. The detailed settings of each module of the model are shown in Table 4 below.

[0066]

[0067] 5. Calculate loss;

[0068] The loss of each round of training is calculated by using the Cross-Entropy Loss function, the formula of which is shown in the following formula (1). Where C is the total number of categories, y i,c is the real label (one-hot encoding) of sample i in category c. p i ,c is the probability of sample i being predicted as category c (the result of the model output after softmax).

[0069]

[0070] 6. Back propagation and optimization;

[0071] According to the loss value of each round, back propagation is performed to automatically adjust the training weights to find a more suitable classification task.

[0072] 7. Accuracy indicators:

[0073] The overall accuracy (OA), mean intersection over union (mIoU), recall and F1-score are used to evaluate the performance of the model. OA refers to the overall accuracy, which is the proportion of samples correctly predicted by the model to the total samples, and can directly reflect the overall performance of the model, but cannot fully reflect the prediction accuracy of the model for imbalanced data sets. Therefore, mIoU is introduced to measure the accuracy of each class and take the average. Similarly, Recall and F1 Score indicators are introduced to handle imbalanced data sets and better evaluate whether the model can identify samples of minority classes. The formulas of various accuracy indicators are shown in Table 5 below, where C is the total number of categories, TP is the number of correctly identified positive samples, FP is the number of false negative samples, TN is the number of correctly identified negative samples, and FN is the number of missed positive samples. Finally, according to the MSSTAN method proposed in the present application and several common time series classification algorithms, the local classification result graphs of different algorithms on the TiSeLaC land cover public data set are as shown in Figure 5 .

[0074]

[0075] The embodiments of the present application have the following technical effects:

[0076] The local object classification method is a multivariate time series classification method based on a transformer. The MSSTAN algorithm includes an AM-GTU module, which can effectively solve the problem of insufficient local correlation capture. In addition, the proposed Residual-Gate KAN can effectively aggregate multi-level context features. The combination of the improvements of the above two parts can realize high-precision time series classification. The local object classification method verifies the high-precision classification of multivariate time series through a public data set, and also provides an important reference for subsequent expansion applications in land use, electronic health record analysis, human activity recognition, and the like.

[0077] An embodiment of the application provides a structural block diagram of a multivariate time series remote sensing data ground object classification device, which comprises:

[0078] An acquisition module is configured to acquire multivariate time series remote sensing data, wherein the multivariate time series remote sensing data comprises multi-band spectral features and time series.

[0079] A detection module is configured to input the multivariate time series remote sensing data into an MSSTAN algorithm model and output ground object target classification and probability.

[0080] The MSSTAN algorithm comprises an AM-GTU module, an encoding layer, a Residual-Gate KAN module and a classifier. The AM-GTU module comprises a multi-scale 1D convolution layer and a gate unit layer. The multi-scale 1D convolution layer is configured to extract spectral and time series information at different scales and control the incoming proportion of information through the gate unit layer. The encoding layer is configured to focus on the relationship between different time steps in the sequence through a multi-head attention mechanism and capture long-term dependencies within the sequence. The Residual-Gate KAN module is configured to aggregate multi-level context features and comprises a KAN module and a linear network layer. The KAN module is configured to extract local features, and the linear network layer is configured to extract global features. The extraction proportions of the local and global features are adjusted through a gate unit. The classifier is configured to output ground object target classification and probability.

[0081] The functions of the modules in the device embodiment correspond to the contents in the corresponding method embodiment, and thus will not be described here.

[0082] It should be noted that the arrangement of the modules in a streaming layout is only one embodiment of the application, and other arrangements can also be used, and the application does not limit this.

[0083] The above merely illustrates the specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for ground object classification of multi-temporal remote sensing data, characterized in that, The method comprises the following steps: S1: acquiring multi-element time sequence remote sensing data containing multi-band spectral characteristics and time series; S2: inputting the multi-element time sequence remote sensing data into an MSSTAN algorithm to output ground object classification and probability; The MSSTAN algorithm comprises an AM-GTU module, an encoding layer, a Residual-Gate KAN module and a classifier; the AM-GTU module comprises a multi-scale 1D convolution layer and a gate unit layer, the multi-scale 1D convolution layer is used to extract spectral and time sequence information at different scales, and the gate unit layer is used to control the incoming proportion of information; the encoding layer comprises an input embedding layer, a position encoding layer, a multi-head attention layer, a cross-layer and normalization layer, a feedforward network layer and a cross-layer and normalization layer, the encoding layer focuses on the relationship between different time steps in the sequence through the multi-head attention mechanism, and captures long-term dependencies within the sequence; the Residual-Gate KAN module is used to aggregate multi-level context features, and comprises a KAN module and a linear network layer, the KAN module is used to extract local features, the linear network layer is used to extract global features, and the extraction proportions of the local and global features are adjusted through a gate unit; the classifier is used to output ground object classification and probability. The multi-element time sequence remote sensing data is subjected to feature extraction through the AM-GTU module, the extracted features are subjected to the encoding layer, and then the local and global features are extracted through the Residual-Gate KAN module, the feature dimensions of the extracted local and global features are mapped to the number of categories through the classifier, and ground object classification and probability are output.

2. The ground object classification method according to claim 1, characterized by: The multi-scale 1D convolution layer comprises five scales of 1D causal convolution kernels, namely 1x3, 1x5, 1x7, 1x9 and 1x12.

3. The ground object classification method according to claim 1, characterized by: The gate unit layer of the AM-GTU module utilizes activation functions Sigmoid and Tanh to control the information incoming proportion, and then performs residual connection with the original input.

4. The ground object classification method according to claim 1, characterized by: The linear network layer of the Residual-Gate KAN module comprises a linear layer, a ReLU activation function and a BN normalization layer.

5. The method of claim 4, wherein: The gate unit in the Residual-Gate KAN module adopts a sigmoid function to adjust the fusion proportion of local and global features.

6. The ground object classification method according to claim 1, characterized by: The classifier is a linear layer, which maps the feature dimension from d_model / / 2 to the number of categories.

7. A device for classifying ground objects from multi-temporal remote sensing data, characterized in that The method comprises the following steps: An acquisition module is used to acquire multi-element time sequence remote sensing data containing multi-band spectral characteristics and time series; A detection module is used to input the multi-element time sequence remote sensing data into an MSSTAN algorithm to output ground object classification and probability. The MSSTAN algorithm comprises an AM-GTU module, an encoding layer, a Residual-Gate KAN module and a classifier; the AM-GTU module comprises a multi-scale 1D convolution layer and a gating unit layer, the multi-scale 1D convolution layer is used to extract spectral and time sequence information at different scales, and the gating unit layer is used to control the incoming proportion of information; the encoding layer comprises an input embedding layer, a position encoding layer, a multi-head attention layer, a cross-layer and normalization layer, a feedforward network layer and a cross-layer and normalization layer, the encoding layer pays attention to the relationship between different time steps in the sequence through the multi-head attention mechanism, and captures the long-term dependence in the sequence; the Residual-Gate KAN module is used to aggregate multi-level context features, and comprises a KAN module and a linear network layer, the KAN module is used to extract local features, the linear network layer is used to extract global features, and the extraction proportions of the local and global features are adjusted through a gating unit; the classifier is used to output the classification and probability of the ground object target. The multi-element time sequence remote sensing data are subjected to feature extraction through the AM-GTU module, the extracted features are subjected to the encoding layer, and then the local and global features are extracted through the Residual-Gate KAN module, the feature dimensions of the extracted local and global features are mapped to the number of categories through the classifier, and the classification and probability of the ground object target are output.

Images