Rice planting area extraction method based on deep learning integrating spatiotemporal and spectral information of remote sensing images

Abstract

The present invention discloses a deep learning rice-growing area extraction method that integrates the spatiotemporal spectral information of remote sensing images, comprising the following steps: Step 1: Constructing a time series of Sentinel-2 image spectral features and texture features; Step 2: Establishing an image set that integrates spectral-temporal-spatial features; Step 3: Constructing a CTH-Net hybrid deep learning model; The CTH-Net hybrid deep learning model includes: a CNN branch, a Transformer branch, a TSSF module, a residual module, Dropout regularization, and a fully connected layer; By optimizing the spectral-temporal-spatial multidimensional data fusion method, efficient integration of remote sensing image spectral, temporal, and spatial features is achieved. CTH-Net introduces a spatiotemporal spectral fusion (TSSF) module and a residual module, significantly improving the model's ability to extract rice in complex landscapes. Results show that CTH-Net achieves 99.69% accuracy, precision, recall, and F1score in rice extraction, and performs stably on different categories such as single-season rice, double-season rice, and abandoned land.

Description

Technical Field

[0001] The present invention belongs to the fields of remote sensing technology, image processing, and big data technology, and in particular relates to a deep learning rice extraction method that integrates spatiotemporal spectral information of remote sensing images. Background Art

[0002] As one of the world's major food crops, accurate monitoring of rice-growing areas is crucial for ensuring food security, optimizing water resource management, and addressing climate change. However, traditional ground-based survey methods are inefficient and have limited coverage, making them inadequate for dynamic monitoring of large-scale rice-growing areas. With the advancement of remote sensing technology, rice distribution mapping based on satellite remote sensing imagery has become an efficient and reliable method. Sentinel-2 and Landsat satellite imagery, as open-access data resources, provide rich spatiotemporal information that can effectively support rice extraction in complex landscapes, providing data support for precision agriculture and agricultural policy decisions.

[0003] Remote sensing crop mapping methods can be divided into two categories based on the temporal dimension: single-temporal and multi-temporal methods. Single-temporal methods rely on spectral signatures at specific time points, offering advantages such as computational simplicity and fast processing speed. However, they struggle to capture dynamic changes in rice growth, resulting in unstable and inaccurate classification results. While multi-temporal methods can better leverage spectral response and growth cycle characteristics by analyzing imagery throughout the entire growth cycle, they still struggle in integrating temporal, spatial, and spectral information. Classification accuracy and robustness need to be improved, particularly in complex landscapes and diverse cropping patterns. In recent years, deep learning techniques have been widely used in remote sensing image classification. Convolutional neural networks (CNNs) are one of the most commonly used architectures. In crop mapping, CNNs can process high-dimensional remote sensing data and extract deep features related to crop type, geographic location, and spectral information, thereby improving classification accuracy. However, CNNs have limitations in capturing global temporal dependencies in time series data. In contrast, recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) have demonstrated excellent performance in processing time series data. LSTM overcomes the vanishing gradient problem of standard RNNs by introducing a gating mechanism, making it more stable when modeling long-term dependencies. In recent years, the Transformer model has achieved breakthroughs in natural language processing and has gradually been applied to remote sensing image analysis. Compared with RNNs and LSTMs, the Transformer's self-attention mechanism is more effective in capturing long-term temporal dependencies. Furthermore, the labor-intensive task of generating training samples makes large-scale crop mapping challenging. Although some researchers have addressed this issue by developing automatic sample generation methods, these methods are not directly applicable to the extraction of more refined single-season and double-season rice in complex landscapes. Therefore, existing technologies have limitations and need improvement. Summary of the Invention

[0004] The purpose of the present invention is to provide a deep learning rice planting area extraction method that integrates the spatiotemporal spectral information of remote sensing images to improve the accuracy of rice extraction in complex landscapes in response to the shortcomings of the existing technology.

[0005] The technical solutions of the present invention are as follows:

[0006] A deep learning rice planting area extraction method integrating spatiotemporal and spectral information of remote sensing images includes the following steps:

[0007] Step 1: Construct a time series of spectral and texture features of Sentinel-2 images;

[0008] Step 2: Create an image set that integrates spectral, temporal, and spatial features;

[0009] Step 3: Build a CTH-Net hybrid deep learning model; the CTH-Net hybrid deep learning model includes: CNN branch, Transformer branch, TSSF module, residual module, Dropout regularization and fully connected layer;

[0010] In CTH-Net, image samples are input to the CNN branch and the Transformer branch; the CNN branch extracts the combination of local spatial and spectral features at each time point; the Transformer branch is used to extract global spatial-temporal-spectral features from the input data;

[0011] CTH-Net introduces a spatiotemporal fusion (TSSF) module with a multi-level attention mechanism to integrate the local features extracted by the CNN branch and the global features extracted by the Transformer branch. The TSSF module embeds the spatial attention (SA) mechanism, which calculates the average pooling features and the maximum pooling features, and learns the weights of important regions through a 3×3 convolutional layer. Finally, the sigmoid activation function is used for normalization.

[0012] The residual module effectively alleviates the common vanishing gradient problem in deep networks by promoting the learning of complex feature representations;

[0013] The Dropout regularization module further enhances the generalization ability of the model and reduces the risk of overfitting;

[0014] The extracted features are mapped to the category space through the fully connected layer, and the output is converted into a probability distribution using the softmax function to effectively handle the diversity of land cover types.

[0015] In the described method, the CNN branch consists of three 3×3 convolutional layers, and each convolutional layer is equipped with batch normalization and ReLU activation function to ensure the effectiveness and stability of feature extraction.

[0016] In the described method, the Transformer branch first performs channel expansion on the input data through a 1×1 convolutional layer; then, position encoding is added to the input data, enabling the Transformer to capture the potential relationships between different time points.

[0017] In the described method, the Transformer branch consists of a three-layer Transformer encoder, each of which contains self-attention, a multi-layer perceptron (MLP), layer normalization, and a residual connection; the Transformer encoder uses the self-attention mechanism and MLP to capture the correlation between different time points, thereby extracting the global features of rice.

[0018] The SA mechanism integrates different local information to enhance detail extraction, thereby reducing land cover misclassification caused by abandoned land and fragmented rice distribution. Its mathematical expression is as follows:

[0019]

[0020] in, represents the global maximum pooling feature, represents the global average pooling feature, σ is the Sigmoid function, C 3×3 Represents a convolution operation with a convolution kernel size of 3×3.

[0021] The method described introduces a channel attention CA module in the Transformer branch to adaptively adjust the weight of each channel and enhance feature representation; applies adaptive maximum pooling and adaptive average pooling to the input features, and then performs two-stage channel feature conversion through one-dimensional convolution and ReLU activation to simplify the model while retaining cross-channel feature extraction capabilities; the CA module learns the importance of each channel and selectively enhances or suppresses specific features.

[0022] As described, the CA module enables the model to focus on the most relevant channels, thereby improving classification accuracy. Its mathematical expression is as follows:

[0023]

[0024] in, represents the adaptive maximum pooling feature, represents the adaptive average pooling feature, C 1×1represents a convolution operation with a convolution kernel size of 1×1, and σ is the Sigmoid function.

[0025] The method described introduces a multi-head attention mechanism (MHSA) to fuse the spatial, temporal and spectral features extracted by CNN and Transformer branches; MHSA enhances the feature expression ability of the model by parallel computing multiple independent attention heads, each of which independently focuses on different aspects of the input data, thereby capturing richer information.

[0026] In the described method, in MHSA, the input features are first mapped into query (Query, Q), key (Key, K), and value (Value, V) matrices respectively through a learnable parameter matrix. Then, for each attention head, its output is calculated using the following formula:

[0027]

[0028] MultiHead(Q,K,V)=Concat(head1,head1,...,head h )W 0

[0029] Among them, d k is the feature dimension of each attention head, Used to scale the dot product results to prevent gradient instability; the attention score is normalized by the softmax function to highlight key features and suppress noise; then, the output of each attention head is spliced ​​and passed through the learnable parameter matrix W 0 A linear transformation is performed to generate the final feature representation. The output of the multi-head attention mechanism is further passed through the residual module to strengthen the feature flow and improve the stability of the model.

[0030] In the described method, the residual module consists of two 3×3 convolutional layers, each of which is followed by BatchNormalization to stabilize the training process; in addition, a 1×1 convolutional shortcut connection directly connects the input and output of the module, enabling the model to learn residual mapping rather than a complete transformation.

[0031] The beneficial effect of the present invention is that by optimizing the spectral-temporal-spatial multidimensional data fusion method, the efficient integration of spectral, temporal and spatial features of remote sensing images is achieved. CTH-Net introduces the spatiotemporal spectral fusion (TSSF) module and the residual module to significantly improve the model's ability to extract rice in complex landscapes. The results show that CTH-Net achieved 99.69% in accuracy, precision, recall and F1score of rice extraction, and performed stably (>96%) in different categories such as single-season rice, double-season rice and abandoned land. Compared with other models such as CNN, Transformer, LSTM and SVM, CTH-Net performs well in dealing with fragmented rice distribution and mixed land types, and improves the accuracy of rice extraction. In addition, CTH-Net exhibits good regional portability and can adapt to different agricultural environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 Flowchart of the rice planting area extraction method based on deep learning by integrating spatiotemporal and spectral information of remote sensing images;

[0033] Figure 2 CTH-Net model structure diagram;

[0034] Figure 3 Evaluation of regional migration capabilities of CTH-Net. DETAILED DESCRIPTION

[0035] The present invention is described in detail below with reference to specific embodiments.

[0036] Step 1: Construct a time series of spectral and texture features of Sentinel-2 images

[0037] Sentinel-2 Level-2A (L2A) satellite remote sensing imagery was obtained from the European Space Agency's Copernicus Open Access Center. The Sentinel-2 satellite's Multispectral Imager (MSI) includes 13 bands, covering the spectral range from visible light to shortwave infrared, with spatial resolutions of 10m, 20m, and 60m. Bands with 20m and 60m spatial resolution were resampled to 10m to ensure consistent spatial resolution across all bands. Cloud and shadow masking was performed using the quality_scene_classification quality flag, resulting in 19 clear-sky images from March to October 2020.

[0038] Ten spectral bands (B2, B3, B4, B5, B6, B7, B8, B8A, B11, and B12) were selected to capture rice growth characteristics. To better reflect vegetation vitality, moisture content, and soil conditions, and enhance rice extraction, five spectral indices were calculated using these spectral bands: the Normalized Difference Vegetation Index (NDVI) and the Enhanced Vegetation Index (EVI), which are effective in detecting vegetation changes and widely used in crop mapping; the Normalized Difference Water Index (NDWI), which identifies changes in soil and vegetation moisture content and is crucial for identifying rice fields that require high amounts of water during their growth phase; and the Modified Soil Adjusted Vegetation Index (MSAVI) and the Brightness Index (BI2), which account for soil background effects and brightness differences between different land cover types, are crucial for accurately extracting rice in areas with complex soil conditions or low vegetation cover. These ten spectral bands and five spectral indices together comprise 15 spectral features.

[0039] Image texture features can describe local variations and patterns in pixel intensity, effectively distinguishing different land cover types and identifying structural details within images. The gray-level co-occurrence matrix (GLCM) can capture grayscale distribution patterns across different directions, distances, and intensities. The GLCM calculates the frequency of pixel value combinations and describes 10 texture features of an image: homogeneity, angular second moment (ASM), contrast, energy, maximum probability, dissimilarity, variance, entropy, mean, and correlation. To optimize extraction, principal component analysis (PCA) was used to extract the first principal component. The image was quantized into 32 grayscale levels, with pixel shifts set to 1 and orientations set to 0°, 45°, 90°, and 135°. The texture feature values ​​for these orientations were averaged to obtain a comprehensive set of texture features for analysis.

[0040] Taking into account the dimensional differences between spectral and textural features, a data mapping strategy is used to normalize these feature values ​​into a unified feature space. This not only ensures feature comparability but also enhances consistency in data processing, thereby improving the reliability of the analysis. The data mapping strategy, based on confidence interval and percentile truncation and stretching techniques, determines a confidence interval that encompasses the data distribution, capturing the main distribution while excluding extreme values. By selecting an interval between 1% and 99%, the grayscale values ​​corresponding to the 1st and 99th percentiles are calculated. These values ​​serve as reference points for linear stretching, mapping the original grayscale values ​​to a unified interval. This process enhances pixel contrast and allows different feature values ​​to be represented on the same scale.

[0041] By arranging 25 feature images (including 15 spectral features and 10 texture features) at each Sentinel-2 image time point, a time series of Sentinel-2 image spectral features and texture features was constructed.

[0042] Step 2: Create an image set that integrates spectral, temporal, and spatial features

[0043] An active learning strategy was used to select high-quality single-season and double-season rice samples. First, an NDVI time series curve was generated for each rice sample, and manual interpretation was used to distinguish single-season and double-season rice. The NDVI curve for double-season rice showed two distinct peaks, corresponding to the growing seasons of early rice (March to July) and late rice (July to October), respectively; single-season rice showed only one peak, with a growing cycle from late May to October. These manually labeled samples were used to train a support vector machine (SVM) classifier to predict the rice type of a wider range of samples. Samples with a prediction probability exceeding 85% were classified as high-confidence samples and directly included in the sample dataset; samples with lower confidence levels underwent further manual inspection to ensure accuracy. The manually labeled samples and high-confidence samples were combined to iteratively train the SVM model, resulting in a rice sample dataset suitable for complex landscapes.

[0044] Based on a high-quality sample dataset, an image set that integrates spectral, temporal, and spatial features is constructed to comprehensively capture the synergistic effects between these features. First, starting from the starting time point of the time series, the values ​​of each pixel across all bands are extracted and combined into a one-dimensional feature vector. This process is repeated at each subsequent time point as the time series progresses, ensuring that the temporal information of each pixel is fully recorded. Finally, the one-dimensional feature vectors obtained from 19 time points are integrated into a two-dimensional matrix and converted into a 19×25 grayscale image, where each pixel value represents the spectral or spatial feature at a specific time point. For any remote sensing image of size W×L, this method is applied to each pixel, generating W×L grayscale images, ultimately forming a multidimensional feature dataset. This approach establishes detailed spectral and spatial response maps for each pixel at different time points, comprehensively integrating temporal, spatial, and spectral features.

[0045] Step 3: Build the CTH-Net hybrid deep learning model

[0046] CTH-Net is a hybrid architecture combining a convolutional neural network (CNN) and a Transformer mechanism. It aims to enhance the extraction of spectral, spatial, and temporal features of rice, thereby achieving accurate rice identification. Rice not only exhibits seasonal spectral dynamics but also possesses unique spatial characteristics. Therefore, efficiently representing these features is crucial for accurate rice identification. To enhance the extraction of high-level, abstract features of the target rice, CTH-Net leverages CNNs and Transformers to mine deep, representative features. In CTH-Net, image samples are input to the CNN branch to extract local spatial and spectral feature combinations at each time point. The CNN branch consists of three 3×3 convolutional layers, each followed by batch normalization and ReLU activation to ensure efficient and stable feature extraction. Simultaneously, the Transformer branch extracts global spatial, temporal, and spectral features from the input data. This branch first performs channel expansion on the input data through a 1×1 convolutional layer to improve feature representation. Subsequently, in order to preserve time series information, positional encoding was added to the input data, enabling the Transformer to capture the potential relationships between different time points. The Transformer branch consists of three layers of Transformer encoders, each of which contains self-attention, multi-layer perceptron (MLP), layer normalization, and residual connection. The Transformer encoder uses the self-attention mechanism and MLP to capture the correlation between different time points, thereby extracting the global features of rice. Compared with traditional convolutional networks, the Transformer has significant advantages in processing long time series and can more effectively identify the temporal change characteristics between single-season rice and double-season rice.

[0047] To effectively integrate the local features extracted by the CNN branch and the global features extracted by the Transformer branch, CTH-Net introduces a spatiotemporal spectral fusion (TSSF) module with a multi-level attention mechanism. This module embeds a spatial attention (SA) mechanism to emphasize the responses of key regions in the feature map extracted by the convolution operation. This mechanism calculates the average pooling features and the maximum pooling features, learns the weights of important regions through a 3×3 convolutional layer, and finally normalizes them using a sigmoid activation function. The SA mechanism integrates different local information and enhances the ability to extract details, thereby reducing land cover misclassification caused by abandoned land and fragmented rice distribution. Its mathematical expression is as follows:

[0048]

[0049] in, represents the global maximum pooling feature, represents the global average pooling feature, σ is the Sigmoid function, C 3×3 Represents a convolution operation with a convolution kernel size of 3×3. The channel attention (CA) mechanism is introduced in the Transformer branch to adaptively adjust the weight of each channel and enhance feature representation. Adaptive maximum pooling and adaptive average pooling are applied to the input features, followed by a two-stage channel feature transformation through one-dimensional convolution and ReLU activation, simplifying the model while retaining cross-channel feature extraction capabilities. Unlike traditional convolution, the CA module learns the importance of each channel and selectively enhances or suppresses specific features. This mechanism enables the model to focus on the most relevant channels, thereby improving classification accuracy. Its mathematical expression is as follows:

[0050]

[0051] in, represents the adaptive maximum pooling feature, represents the adaptive average pooling feature, C 1×1 Denotes a convolution operation with a kernel size of 1×1, and σ is the sigmoid function. Unlike traditional convolution, the CA module learns the importance of each channel and selectively enhances or suppresses specific features. This mechanism enables the model to focus on the most relevant channels, thereby improving classification accuracy.

[0052] In addition, a multi-head attention mechanism (MHSA) is introduced to fuse the spatial, temporal, and spectral features extracted by the CNN and Transformer branches. Unlike the traditional attention mechanism (which uses a single weight matrix), MHSA enhances the model's feature expression capabilities by computing multiple independent attention heads in parallel. Each attention head independently focuses on different aspects of the input data, thereby capturing richer information. In MHSA, the input features are first mapped to query (Query, Q), key (Key, K), and value (Value, V) matrices through a learnable parameter matrix. Subsequently, for each attention head, its output is calculated using the following formula:

[0053]

[0054] MultiHead(Q,K,V)=Concat(head1,head1,...,head h )W 0

[0055] Among them, d k is the feature dimension of each attention head, Used to scale the dot product results to prevent gradient instability. The attention score is normalized by the softmax function to highlight key features and suppress noise. Then, the outputs of each attention head are concatenated and passed through the learnable parameter matrix W 0 A linear transformation is performed to generate the final feature representation. The output of the multi-head attention mechanism is further passed through the residual module to strengthen the feature flow and improve the stability of the model.

[0056] The residual module effectively alleviates the vanishing gradient problem common in deep networks by facilitating the learning of complex feature representations. The module consists of two 3×3 convolutional layers, each followed by batch normalization to stabilize the training process. Furthermore, a 1×1 convolutional shortcut connection directly connects the module's input and output, enabling the model to learn a residual mapping rather than a complete transformation. This design not only helps preserve temporal, spatial, and spectral features during feature fusion but also enables the model to capture complex relationships in the input data by allowing efficient gradient flow, thereby improving the accuracy of rice extraction under complex surface conditions. Dropout regularization is introduced to further enhance the model's generalization ability and reduce the risk of overfitting. Finally, a fully connected layer maps the extracted features to a category space, and a softmax function is used to convert the output into a probability distribution to effectively handle the diversity of land cover types.

[0057] Step 4: Evaluate Rice Extraction Accuracy

[0058] To evaluate the accuracy of rice extraction results, a confusion matrix was introduced to quantify the model's correct and incorrect predictions. As shown in Table 1, multiple accuracy evaluation metrics were further calculated based on the confusion matrix, including accuracy, precision, recall, and F1 score. Accuracy, ranging from 0 to 1, measures the model's overall performance in classifying six land cover types (single-season rice, double-season rice, water bodies, forests, abandoned land, and other). A high value indicates good classification performance. Precision, also ranging from 0 to 1, reflects the model's ability to accurately identify rice-growing areas while minimizing misclassification of other land cover types as rice. High precision values ​​for single-season and double-season rice extraction indicate a low model error rate. Recall, also ranging from 0 to 1, measures the model's comprehensiveness in identifying single-season and double-season rice. A high recall value indicates that the model has good coverage of actual rice-growing areas, minimizing omissions. The F1 score is a balance between precision and recall, ranging from 0 to 1. A score close to 1 indicates that the model strikes a good balance between accurate recognition and comprehensive coverage, showing strong overall performance.

[0059] Table 1. Performance evaluation of different models

[0060]

[0061]

[0062] like Figure 3 As shown, the model's transferability assessment across six land types demonstrates excellent performance across four indicators. Each indicator for single-season rice is approximately 0.97, while each indicator for double-season rice approaches 0.96. The overall classification accuracy for rice reaches 0.9960, demonstrating that the model can effectively identify rice-growing areas. The precision and recall for abandoned land are 0.9814 and 0.9875, respectively. Indicators for forests, water bodies, and other land types all exceed 0.99, confirming the model's excellent regional adaptability.

[0063] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A deep learning rice planting area extraction method integrating spatiotemporal and spectral information of remote sensing images, characterized by: The following steps are involved: Step 1: Construct a time series of spectral and texture features of Sentinel-2 images; Step 2: Create an image set that integrates spectral, temporal, and spatial features. First, starting from the starting time point of the time series, extract the values ​​of each pixel in all bands and combine these band values ​​into a one-dimensional feature vector. As the time series progresses, this process is repeated at each subsequent time point to ensure that the temporal information of each pixel is fully recorded. Finally, the one-dimensional feature vectors obtained from the 19 time points are integrated into a two-dimensional matrix and converted into a 19×25 grayscale image, where each pixel value represents the spectral or spatial feature at a specific time point. Finally, a multidimensional feature data set is formed; Step 3: Build a CTH-Net hybrid deep learning model; the CTH-Net hybrid deep learning model includes: CNN branch, Transformer branch, TSSF module, residual module, Dropout regularization and fully connected layer; In CTH-Net, image samples are input to the CNN branch and the Transformer branch; the CNN branch extracts the combination of local spatial and spectral features at each time point; the Transformer branch is used to extract global spatial-temporal-spectral features from the input data; CTH-Net introduces a spatiotemporal fusion (TSSF) module with a multi-level attention mechanism to integrate the local features extracted by the CNN branch and the global features extracted by the Transformer branch. The TSSF module embeds the spatial attention (SA) mechanism, which calculates the average pooling features and the maximum pooling features, and learns the weights of important regions through a 3×3 convolutional layer. Finally, the sigmoid activation function is used for normalization. The residual module effectively alleviates the common vanishing gradient problem in deep networks by promoting the learning of complex feature representations; Dropout regularization further enhances the generalization ability of the model and reduces the risk of overfitting; The extracted features are mapped to the category space through the fully connected layer, and the output is converted into a probability distribution using the softmax function to effectively handle the diversity of land cover types.

2. The method according to claim 1, characterized in that The CNN branch consists of three 3×3 convolutional layers, each of which is followed by batch normalization and ReLU activation function to ensure the effectiveness and stability of feature extraction.

3. The method according to claim 1, characterized in that The Transformer branch first performs channel expansion on the input data through a 1×1 convolutional layer; then, positional encoding is added to the input data, enabling the Transformer to capture the potential relationships between different time points.

4. The method according to claim 1, wherein The Transformer branch consists of a three-layer Transformer encoder, each of which contains self-attention, a multi-layer perceptron (MLP), layer normalization, and residual connections; the Transformer encoder uses the self-attention mechanism and MLP to capture the correlation between different time points, thereby extracting the global features of rice.

5. The method according to claim 1, wherein The SA mechanism integrates different local information and enhances detail extraction capabilities, thereby reducing land cover misclassification caused by abandoned land and fragmented rice distribution. Its mathematical expression is as follows: in, represents the global maximum pooling feature, represents the global average pooling feature, σ is the Sigmoid function, C 3×3 Represents a convolution operation with a convolution kernel size of 3×3.

6. The method according to claim 4, characterized in that A channel attention (CA) module is introduced in the Transformer branch to adaptively adjust the weight of each channel and enhance feature representation. Adaptive maximum pooling and adaptive average pooling are applied to the input features, followed by a two-stage channel feature transformation through one-dimensional convolution and ReLU activation, simplifying the model while retaining cross-channel feature extraction capabilities. The CA module learns the importance of each channel and selectively enhances or suppresses specific features.

7. The method according to claim 6, characterized in that The CA module enables the model to focus on the most relevant channels, thereby improving classification accuracy. Its mathematical expression is as follows: in, represents the adaptive maximum pooling feature, represents the adaptive average pooling feature, C 1×1 represents a convolution operation with a convolution kernel size of 1×1, and σ is the Sigmoid function.

8. The method according to claim 6, characterized in that The multi-head attention mechanism (MHSA) is introduced to fuse the spatial, temporal and spectral features extracted by CNN and Transformer branches; MHSA enhances the feature expression ability of the model by parallel computing multiple independent attention heads. Each attention head independently focuses on different aspects of the input data, thereby capturing richer information.

9. The method according to claim 1, characterized in that The residual module consists of two 3×3 convolutional layers, each of which is followed by Batch Normalization to stabilize the training process. In addition, a 1×1 convolutional shortcut connection directly connects the input and output of the module, enabling the model to learn a residual mapping instead of a complete transformation.

Images