Wetland vegetation extraction method and system based on spatial-temporal feature fusion

Abstract

The invention relates to a wetland vegetation extraction method and system based on spatial-temporal feature fusion, and belongs to the field of wetland remote sensing intelligent interpretation. The problems that an existing wetland vegetation classification method does not make full use of remote sensing data time sequence features, and the wetland vegetation extraction precision is insufficient are solved. According to the technical scheme, a single optical multi-temporal remote sensing image is used as input, frequency domain decomposition and convolution long and short time memory network modeling long-term phenology and short-term fluctuation are carried out through a time-frequency enhancement time sequence branch, and spectral grouping of a spectral space branch and cavity convolution multi-scale feature extraction are combined; and a cross-domain cross attention fusion module is utilized to realize time sequence spectral space three-dimensional feature deep fusion, and finally a wetland vegetation community classification map is output. On the premise of not depending on multi-source data, the time-frequency enhanced double-branch space-time model is innovatively provided, the multi-temporal remote sensing data time sequence feature self-adaptive extraction capacity can be effectively improved, the classification stability and robustness are improved, and finally the remote sensing monitoring precision of a wetland ecosystem is improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent interpretation of wetland remote sensing, and relates to a method and system for extracting wetland vegetation based on spatiotemporal feature fusion. Background Technology

[0002] Wetlands, along with forests and cultivated land, constitute the three major ecosystems of Earth, covering approximately six percent of the Earth's total surface area. As open, complex, and highly interactive ecological complexes, wetlands play vital ecological roles, including water purification, flood regulation, carbon sequestration, and habitat provision, making them an indispensable component of terrestrial ecosystems. Wetland vegetation, as a key component of wetland ecosystems, significantly impacts wetland ecological functions through its type, distribution, and dynamic changes. Therefore, the identification and classification of wetland vegetation is a crucial foundation for the monitoring, protection, and management of wetland ecosystems.

[0003] The distribution and dynamic changes of wetland vegetation are key aspects of ecological environment monitoring. Existing wetland vegetation monitoring methods mainly include field surveys and remote sensing. Field surveys can provide high-precision information on vegetation types, but are limited by time, space, and manpower factors, making them difficult to implement on large scales and over long periods. Remote sensing technology, utilizing sensors mounted on satellites or aircraft, can acquire continuous observation data with high spatiotemporal resolution, offering significant advantages in complex terrain or inaccessible areas. Therefore, remote sensing technology plays a crucial role in wetland vegetation classification and identification.

[0004] Existing wetland cover classification methods based on remote sensing data can generally be divided into two categories: unsupervised classification and supervised classification. Unsupervised classification often employs exponential clustering methods, which, while simple to implement, struggle to accurately distinguish specific land cover types. Supervised classification methods are trained using labeled samples. With the development of machine learning and deep learning technologies, supervised algorithms such as Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), Convolutional Neural Network (CNN), Generative Adversarial Network (GAN), and Vision Transformer have been widely applied in wetland classification. Existing research has shown that deep learning models have significant advantages in extracting high-dimensional spectral and temporal features. Researchers have used Sentinel satellite time-series data to find that the classification accuracy of deep neural networks is significantly higher than that of traditional machine learning methods; other researchers have successfully used convolutional neural networks to achieve high-precision classification of wetland land cover types; and some scholars have implemented land cover identification based on multi-temporal satellite data using multi-layer neural networks.

[0005] However, existing methods for classifying wetland vegetation using remote sensing imagery have certain limitations. For example, most existing methods are based on spectral information from a single or limited time phase to construct classification models. While these methods perform well in some scenarios, their ability to distinguish wetland vegetation with similar spectral characteristics and overlapping ecological niches is limited. On the other hand, although high temporal resolution remote sensing data contains rich phenological information and can reflect changes in vegetation growth rhythms, wetland vegetation is significantly affected by hydrological conditions such as floods and dry seasons, resulting in unstable phenological trajectories over time. When modeling temporal features, existing methods often struggle to adaptively capture the coupling relationship between high-frequency phenological changes and low-frequency hydrological trends, leading to fluctuations in classification results across different periods and insufficient stability.

[0006] Therefore, there is an urgent need for a technical solution that can fully utilize multi-temporal remote sensing image information and combine it with wetland vegetation phenological characteristics to achieve fine classification and dynamic identification of wetland vegetation. This invention, through the design of a time-frequency enhanced dual-branch spatiotemporal remote sensing network, constructs a fusion architecture including a time-frequency enhanced temporal branch and a spectral spatial branch. It aims to solve the problems of insufficient utilization of temporal features and weak adaptability to hydrological changes in existing technologies, ultimately improving the accuracy and timeliness of wetland ecosystem monitoring. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a wetland vegetation extraction method and system based on spatiotemporal feature fusion. By effectively modeling multi-temporal remote sensing images, higher accuracy wetland vegetation classification is achieved. Through a time-frequency enhanced bi-branch structure, the model can adaptively select the importance of long-term change trends and short-term emergencies, ultimately improving the accuracy of wetland vegetation community classification.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] A wetland vegetation extraction method based on spatiotemporal feature fusion includes:

[0010] Input multi-temporal remote sensing image data;

[0011] The multi-temporal remote sensing image data is processed by a time-frequency enhancement time-series branch to obtain time-frequency enhancement time-series features;

[0012] The multi-temporal remote sensing image data is processed by spectral spatial branching to obtain spectral spatial features;

[0013] The time-frequency enhanced temporal features and the spectral spatial features are fused using a Bidirectional Cross-Attention Fusion (BCAF) module to obtain fused features.

[0014] And the fused features are classified using a classification head to output a wetland vegetation classification map.

[0015] Furthermore, the time-frequency enhancement temporal branch includes a frequency domain decomposition module, a Convolutional Long Short-Term Memory (ConvLSTM) module, and a Time-Frequency Attention Fusion (TFAF) module.

[0016] The frequency domain decomposition module is used to decompose the multi-temporal remote sensing image data into low-frequency and high-frequency components using Fast Fourier Transform (FFT).

[0017] The ConvLSTM module is used to perform time-series modeling on the low-frequency components and the high-frequency components respectively to obtain low-frequency time-series features and high-frequency time-series features;

[0018] The TFAF module is used to adaptively weight and fuse the low-frequency time series features and the high-frequency time series features to generate the time-frequency enhanced time series features.

[0019] Furthermore, when the frequency domain decomposition module performs a fast Fourier transform, the multi-temporal remote sensing image data is represented in tensor form X∈R. B×T×C×H×W Where B is the batch size, T is the number of time steps, C is the number of spectral channels, H is the image height, and W is the image width;

[0020] The Fast Fourier Transform is performed along the time dimension T to obtain the frequency domain representation X. freq =FFT(X), and through a low-frequency mask m low and high frequency mask m high Extracting low-frequency components and high frequency components The low-frequency component in the time domain is then obtained through inverse fast Fourier transform. and time-domain high-frequency components

[0021] Furthermore, the ConvLSTM module includes a first convolutional long short-term memory network for processing the time-series low-frequency components and a second convolutional long short-term memory network for processing the time-domain high-frequency components;

[0022] The first convolutional long short-term memory network and the second convolutional long short-term memory network have the same network structure, respectively targeting the low-frequency component X in the time domain. low and time-domain high-frequency components X high Perform independent time series modeling;

[0023] The update of any convolutional long short-term memory network at time t is represented as (H t C t =ConvLSTM(X) t H t-1 C t-1 ), where H t In the hidden state, C t The memory unit represents the state and includes gated convolution operations; the low-frequency and high-frequency branches obtain their corresponding temporal features at the end of the sequence. and

[0024] The H low and H high Spatial refinement is performed through convolution, normalization, and activation units to obtain a dimension of B×C. h ×H×W low-frequency timing characteristics and the high-frequency timing characteristics.

[0025] Furthermore, the TFAF module uses the low-frequency timing feature H low and the high-frequency timing feature H high F is obtained by splicing along the channel dimension. cat =Concat(H low H high Then, a pixel-wise time-frequency attention weight map g∈[0,1] is generated through a gated subnetwork. B×1×H×W And according to g on H low and H high Weighted fusion is performed to obtain time-frequency enhanced temporal features F t =g⊙H low +(1-g)⊙H high .

[0026] Furthermore, the spectral spatial branch includes a spectral grouping and dilated convolution module and a spectral-scale attention fusion module;

[0027] The spectral grouping and dilated convolution module is used to divide the multi-temporal remote sensing image data into G spectral subgroups according to the band range, and apply dilated convolution with different dilation rates to each spectral subgroup to obtain multi-scale features.

[0028] The spectral-scale attention fusion module is used to weight and fuse the multi-scale features through attention weights to generate the spectral spatial features.

[0029] Furthermore, the spectral grouping and dilated convolution module takes the input U of the g-th spectral subgroup as an argument. g Applying a void ratio of δ g The 3×3 dilated convolution is obtained Where g = 1,...,G, and a 1×1 convolution is applied to the original multispectral features to obtain V0 = Conv 1×1 (U), thus obtaining G+1 scale features {V0,V1,...,V G The spectral-scale attention fusion module generates attention weights a through 1×1 convolution and Softmax normalization. i =Softmax(Conv 1×1 (V0,...,V G ), i = 0,...,G, and perform a weighted summation on the features. The spectral spatial characteristics are thus obtained.

[0030] Furthermore, the cross-domain cross-attention fusion module BCAF achieves fusion through bidirectional cross-attention:

[0031] The attention mechanism that moves from temporal features to spectral features uses temporal features as the query and spectral features as the key and value to calculate... and Z t→i =A t→i ·V i ;

[0032] The attention mechanism from spectral features to temporal features uses spectral features as queries and temporal features as keys and values ​​to compute... and Z i→t =A i→t ·V t Where Q, K, and V represent query, key, and value, respectively, and d is the embedding dimension scaling factor; the final fused features are obtained through Z... t→i and Z i→t After summation, the result is obtained by convolution-normalization-activation CBR processing, yielding F = CBR(Z). t→i +Z i→t ).

[0033] A wetland vegetation extraction system based on spatiotemporal feature fusion includes:

[0034] The input module is used to input multi-temporal remote sensing image data;

[0035] The time-frequency enhancement timing branch module has its input end connected to the input module and is used to process the multi-temporal remote sensing image data to obtain time-frequency enhancement timing features;

[0036] A spectral spatial branching module, whose input end is connected to the input module, is used to process the multi-temporal remote sensing image data to obtain spectral spatial features;

[0037] The cross-domain attention fusion module BCAF has its input connected to the output of the time-frequency enhancement temporal branch module and the spectral spatial branch module, and is used to fuse the time-frequency enhancement temporal features and the spectral spatial features to obtain fused features;

[0038] And a classification head module, whose input is connected to the output of the cross-domain cross-attention fusion module, is used to classify the fused features to output a wetland vegetation classification map.

[0039] Furthermore, the time-frequency enhancement timing branch module includes a frequency domain decomposition submodule, a ConvLSTM submodule, and a TFAF submodule; the spectral spatial branch module includes a spectral grouping and dilated convolution submodule and a spectral-scale attention fusion submodule; the cross-domain cross attention fusion module BCAF includes a query, key and value generation submodule and a bidirectional attention calculation submodule.

[0040] Furthermore, the classification head module includes a 1×1 convolutional submodule, used to map the fused features to a category space and output a pixel-level wetland vegetation classification map. N cls This represents the number of land feature categories.

[0041] The beneficial effects of this invention are as follows:

[0042] (1) This invention is based on a wetland vegetation extraction method and system using spatiotemporal feature fusion. It constructs a time-frequency enhanced dual-branch spatiotemporal feature fusion network, which improves the overall accuracy and stability of wetland vegetation extraction. Through the overall architecture design of "time-frequency enhanced temporal branch + spectral spatial branch + cross-domain cross-attention fusion", this invention achieves joint modeling of temporal, spectral and spatial three-dimensional features under the condition of single optical multi-temporal remote sensing data. It effectively overcomes the limitations of traditional methods that rely only on a small number of temporal phases and have insufficient feature expression capabilities, resulting in poor adaptability to complex wetland environments. It exhibits higher accuracy and better robustness in wetland vegetation monitoring tasks.

[0043] (2) This invention achieves a fine distinction between long-term phenological trends and short-term disturbance signals of wetland vegetation through frequency domain decomposition and time-frequency enhancement mechanisms. By performing low-frequency and high-frequency decomposition on the temporal variation patterns of multi-temporal images and then performing adaptive time-frequency fusion, this invention can automatically assign feature weights to different types of temporal changes, thereby significantly improving the distinction effect among wetland vegetation types with similar ecological niches and seasonal rhythms, and enhancing the stability and robustness of classification.

[0044] (3) This invention achieves deep bidirectional synergy between temporal features and spectral-spatial features through a cross-domain attention fusion mechanism. It establishes a complementary relationship between the temporal and spectral-spatial domains at the pixel level, enabling phenological change patterns and spectral-spatial responses to mutually constrain and align, thus improving the consistency of feature fusion. Compared to traditional simple splicing or unidirectional fusion methods, this invention can more fully utilize the correlation between multidimensional features, thereby obtaining more discriminative fused features and improving the overall accuracy of wetland vegetation extraction.

[0045] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0047] Figure 1 This is a schematic diagram of a time-frequency enhanced dual-branch spatiotemporal remote sensing network structure.

[0048] Figure 2 This is a schematic diagram of the Time-Frequency Attention Fusion (TFAF) module structure.

[0049] Figure 3 A schematic diagram of the spectral grouping and dilated convolution module structure for spectral spatial branching;

[0050] Figure 4 A schematic diagram of the cross-domain attention fusion module BCAF;

[0051] Figure 5 Example image of the original remote sensing image;

[0052] Figure 6 This is a diagram showing the classification results of wetland vegetation. Detailed Implementation

[0053] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0054] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0055] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0056] This invention proposes a time-frequency enhanced dual-branch spatiotemporal remote sensing network. Through a "dual-branch + three-domain fusion" design, it achieves joint modeling of temporal, spectral, and spatial three-dimensional features. Figure 1 As shown, the network consists of two parts: a time-frequency enhancement temporal branch and a spectral spatial branch. At the high level, the two features are deeply fused through a cross-domain attention fusion module, and finally the pixel-level classification result of wetland vegetation is output.

[0057] (1) Input and Data Representation

[0058] The input is a stack of multi-temporal remote sensing image data, denoted as:

[0059] X∈R B×T×C×H×W

[0060] Where B is the batch size; T is the number of time steps, representing the time dimension of the remote sensing image sequence; C is the number of spectral channels per time step; and H and W are the height and width of the image. The goal of this model is to generate a class prediction map for each pixel.

[0061]

[0062] Where, N cls Number of land cover categories

[0063] (2) Time-frequency enhancement timing branch

[0064] This module first uses Fast Fourier Transform to decompose multi-temporal features into low-frequency and high-frequency components along the time dimension, and then models long-term phenological changes and short-term hydrological fluctuations using two ConvLSTMs with shared structures. Based on this, a Time-Frequency Attention Fusion (TFAF) module is constructed. Using the temporal representations of low-frequency and high-frequency features, a pixel-by-pixel time-frequency weight map is generated, and the two frequency domain features are adaptively weighted and fused to obtain a time-frequency enhanced temporal representation that simultaneously considers long-term trends and short-term fluctuations.

[0065] ① Frequency Decomposer module

[0066] To extract long-term trends and short-term variations from time-series data, this module first performs a Fast Fourier Transform (FFT) on the input features in the time dimension, mapping the time-series signal to the frequency domain to obtain its frequency domain representation.

[0067] X freq =FFT(X)

[0068] Where FFT(·) denotes the Fast Fourier Transform along the time dimension T, X freq This represents the input data in the frequency domain. Subsequently, a low-frequency mask m is constructed. low With high frequency mask m high These are used to preserve low-frequency components and high-frequency components, respectively.

[0069]

[0070] By performing a Fourier transform on the above frequency domain components, the low-frequency and high-frequency components in the time domain can be obtained:

[0071]

[0072] Among them, X low The main focus is on depicting the smooth phenological changes of wetland vegetation during the growing season (such as long-term trends like budding, vigorous growth, and withering), while X high This indicates short-term disturbances and abnormal fluctuations caused by changes in meteorological and hydrological conditions.

[0073] ②ConvLSTM (Temporal Modeling)

[0074] In obtaining the low-frequency component X low With high-frequency component X high Then, two identical Convolutional Long Short-Term Memory (ConvLSTM) networks are constructed, and their temporal sequences are modeled independently. Let the input sequence of the low-frequency branch be... The input sequence of the high-frequency branch is The update of each ConvLSTM at time t can be simply expressed as:

[0075] H t C t =ConvLSTM(X) t H t-1 C t-1

[0076] Among them, H t In the hidden state, C t The memory unit represents the state and includes an input gate, a forget gate, an output gate, and gated convolution operations for candidate states, which will not be elaborated here. The low-frequency and high-frequency branches yield their corresponding temporal features at the end of the sequence:

[0077]

[0078] ③TFAF module (Time-Frequency Attention Fusion Module)

[0079] To adaptively fuse low-frequency and high-frequency features, this invention designs a Time-Frequency Attention Fusion (TFAF) module, such as... Figure 2 As shown.

[0080] This module uses H low With H high As input, the data is first concatenated along the channel dimension:

[0081] F cat =Concat(H low H high )

[0082] Subsequently, F cat The input is fed into a gated subnetwork consisting of convolutional layers, normalization layers, and a sigmoid activation function to generate a pixel-wise time-frequency attention weight map g∈[0,1]. B×1×H×W Finally, based on this weighted graph, the low-frequency and high-frequency features are weighted and fused to obtain the time-frequency enhanced temporal representation:

[0083] F t =g⊙H low +(1-g)⊙H high

[0084] Here, g represents the relative confidence level of long-term phenological trends and short-term hydrological fluctuations at different spatial locations. After subsequent processing by convolution-normalization-activation (CBR) layers, the output feature map of the temporal branch is obtained, which is used for cross-domain fusion in the spectral spatial branch.

[0085] (3) Spectral spatial branching

[0086] Spectral spatial branching is used to jointly mine multi-scale discriminative features of wetland vegetation across both the spectral and spatial dimensions. This invention constructs a spectral-spatial multi-scale modeling module with SpectralGroupASPP (Spectral Grouped Spatial Dilated Pyramid Convolution) as its core structure. For example... Figure 3 As shown, this module mainly includes a spectral grouping and dilated convolution submodule and a spectral-scale attention fusion submodule.

[0087] ①SpectralGroupASPP (Spectral Grouping and Dilated Convolution)

[0088] First, the input multispectral features are divided into G spectral subgroups based on their wavelength range or physical meaning, such as the visible light group, near-infrared group, and short-wave infrared group. Let the input of the g-th spectral subgroup be U. g The corresponding void ratio is δ g Then, a 3×3 dilated convolution is applied to each spectral subgroup:

[0089]

[0090] This invention aims to acquire multi-scale receptive fields ranging from small to large within different spectral subspaces. To supplement the overall spectral information and local detail responses, the original multispectral features are further convolved with a 1×1 convolution to construct full-spectrum small-scale feature branches.

[0091] V0 = Conv 1×1 (U)

[0092] This yields spectral-spatial features at G+1 scales {V0, V1, ..., V G}

[0093] ②Spectral-scale attention fusion

[0094] To adaptively select contributions from different spectral subspaces and spatial scales, this invention introduces a spectral-scale attention fusion mechanism. First, the G+1 scale features mentioned above are concatenated in the channel dimension. Then, the attention weight 'a' of each scale at different spatial locations is learned through 1×1 convolution and Softmax normalization. i :

[0095] a i =Softmax(Conv 1×1 (V0,...,V G ), i = 0, ..., G

[0096] Based on this, multi-scale features are weighted and fused:

[0097]

[0098] Thus, the spectral-spatial multi-scale fusion feature F is obtained. i It is used to characterize the spatial texture and spectral differences of communities in complex wetland environments.

[0099] (4) Bidirectional Cross-Attention Fusion Module

[0100] To achieve temporal branching features (F t ) and spectral spatial branching characteristics (F i To address the deep interaction between domains, this invention designs a cross-domain cross-attention fusion module (BCAF), such as... Figure 4 As shown.

[0101] This module first applies a 1×1 convolution to each of the two feature paths to generate the corresponding query (Q), key (K), and value (V) representations:

[0102] For spectral spatial characteristics of F i :

[0103]

[0104] For time series features F t :

[0105]

[0106] Based on this, the present invention constructs cross-domain attention in two directions:

[0107] Attention mechanism from temporal features to spectral features (Temporal→Spectral Attention):

[0108] Using temporal features as queries and spectral features as keys and values, this study focuses on the locations in the spectral-spatial domain most relevant to phenological patterns from a temporal perspective:

[0109]

[0110] Z t→i =A t→i ·V i

[0111] Spectral-Temporal Attention: This approach uses spectral features as the query and temporal features as the key and value, selecting the temporal pattern that best matches the current spectral response from a spectral-spatial perspective.

[0112]

[0113] Z i→t =A i→t·V t

[0114] Where d is the embedding dimension scaling factor. The output Z of the two-way cross-attention is... t→i With Z i→t After concatenation or summation along the channel dimension, the input is fed into a CBR module consisting of convolution, normalization, and activation functions to obtain the final cross-domain fusion feature:

[0115] F = CBR(Z) t→i +Z i→t )

[0116] Through bidirectional interaction between the temporal domain and the spectral spatial domain, this invention achieves joint modeling and deep fusion of temporal-spectral-spatial information of wetland vegetation.

[0117] (5) Classification Header

[0118] Finally, the cross-domain fused feature map F is input into the 1×1 convolution class head and mapped along the channel dimension to the class space to obtain a pixel-level land cover class prediction map:

[0119] Y = Conv 1×1 (F)

[0120] In this context, the c-th channel of Y represents the predicted response of a pixel belonging to the c-th type of land cover.

[0121] Based on Sentinel-2 multi-temporal remote sensing images with cloud cover below 20% in 2024, preprocessing operations such as atmospheric correction, resampling, band synthesis, and regional cropping were performed on the raw data, and a time-series image dataset of the same region was constructed. The multi-temporal images were input into the time-frequency enhancement branch of this invention, and the multispectral image of any temporal phase was input into the spectral spatial branch. After network processing, wetland vegetation classification results were obtained, such as... Figure 5 and Figure 6 As shown.

[0122] Verification experiment:

[0123] To verify the effectiveness of the method of this invention, its performance was compared with two typical existing wetland vegetation classification methods. Experimental results show that the method of this invention achieves advantages in both overall classification accuracy and F1 score, with an overall accuracy exceeding 90% and high identification accuracy for each vegetation type, thus meeting the application requirements of wetland ecological monitoring.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for extracting wetland vegetation based on spatiotemporal feature fusion, characterized in that: include: Input multi-temporal remote sensing image data; The multi-temporal remote sensing image data is processed by a time-frequency enhancement time-series branch to obtain time-frequency enhancement time-series features; The multi-temporal remote sensing image data is processed by spectral spatial branching to obtain spectral spatial features; The time-frequency enhanced temporal features and the spectral spatial features are fused using the cross-domain cross-attention fusion module BCAF to obtain fused features; And the fused features are classified using a classification head to output a wetland vegetation classification map.

2. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 1, characterized in that: The time-frequency enhancement timing branch includes a frequency domain decomposition module, a Convolutional Long Short-Term Memory (ConvLSTM) module, and a time-frequency attention fusion module (TFAF). The frequency domain decomposition module is used to decompose the multi-temporal remote sensing image data into low-frequency and high-frequency components using Fast Fourier Transform (FFT). The ConvLSTM module is used to perform time-series modeling on the low-frequency components and the high-frequency components respectively to obtain low-frequency time-series features and high-frequency time-series features; The TFAF module is used to adaptively weight and fuse the low-frequency time series features and the high-frequency time series features to generate the time-frequency enhanced time series features.

3. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 2, characterized in that: When the frequency domain decomposition module performs a fast Fourier transform, the multi-temporal remote sensing image data is represented in tensor form X∈R. B ×T×C×H×W Where B is the batch size, T is the number of time steps, C is the number of spectral channels, H is the image height, and W is the image width; The Fast Fourier Transform is performed along the time dimension T to obtain the frequency domain representation X. freq =FFT(X), and through a low-frequency mask m low and high frequency mask m high Extracting low-frequency components and high frequency components The low-frequency component in the time domain is then obtained through inverse fast Fourier transform. and time-domain high-frequency components 4. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 2, characterized in that: The ConvLSTM module includes a first convolutional long short-term memory network for processing the time-series low-frequency components and a second convolutional long short-term memory network for processing the time-domain high-frequency components. The first convolutional long short-term memory network and the second convolutional long short-term memory network have the same network structure, respectively targeting the low-frequency component X in the time domain. low and time-domain high-frequency components X high Perform independent time series modeling; The update of any convolutional long short-term memory network at time t is represented as (H t C t =ConvLSTM(X) t H t-1 C t-1 ), where H t In the hidden state, C t The memory unit represents the state and includes gated convolution operations; the low-frequency and high-frequency branches obtain their corresponding temporal features at the end of the sequence. and The H low and H high Spatial refinement is performed through convolution, normalization, and activation units to obtain a dimension of B×C. h ×H×W low-frequency timing characteristics and the high-frequency timing characteristics.

5. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 2, characterized in that: The TFAF module uses the low-frequency timing feature H low and the high-frequency timing feature H high F is obtained by splicing along the channel dimension. cat =Concat(H low H high Then, a pixel-wise time-frequency attention weight map g∈[0,1] is generated through a gated subnetwork. B×1×H×W And according to g on H low and H high Weighted fusion is performed to obtain time-frequency enhanced temporal features F t =g⊙H low +(1-g)⊙H high .

6. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 1, characterized in that: The spectral spatial branch includes a spectral grouping and dilated convolution module and a spectral-scale attention fusion module; The spectral grouping and dilated convolution module is used to divide the multi-temporal remote sensing image data into G spectral subgroups according to the band range, and apply dilated convolution with different dilation rates to each spectral subgroup to obtain multi-scale features. The spectral-scale attention fusion module is used to weight and fuse the multi-scale features through attention weights to generate the spectral spatial features.

7. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 6, characterized in that: The spectral grouping and dilated convolution module inputs U to the g-th spectral subgroup g Applying a void ratio of δ g The 3×3 dilated convolution is obtained Where g = 1,...,G, and a 1×1 convolution is applied to the original multispectral features to obtain V0 = Conv 1×1 (U), thus obtaining G+1 scale features {V0,V1,...,V G The spectral-scale attention fusion module generates attention weights a through 1×1 convolution and Softmax normalization. i =Softmax(Conv 1×1 (V0,...,V G ), i = 0,...,G, and perform a weighted summation on the features. The spectral spatial characteristics are thus obtained.

8. The wetland vegetation extraction method based on spatiotemporal feature fusion according to claim 1, characterized in that: The cross-domain cross-attention fusion module BCAF achieves fusion through bidirectional cross-attention: The attention mechanism that moves from temporal features to spectral features uses temporal features as the query and spectral features as the key and value to calculate... and Z t→i =A t→i ·V i ; The attention mechanism from spectral features to temporal features uses spectral features as queries and temporal features as keys and values ​​to compute... and Z i→t =A i→t ·V t Where Q, K, and V represent query, key, and value, respectively, and d is the embedding dimension scaling factor; the final fused features are obtained through Z... t→i and Z i→t After summation, the result is obtained by convolution-normalization-activation CBR processing, yielding F = CBR(Z). t→i +Z i→t ).

9. A wetland vegetation extraction system based on spatiotemporal feature fusion, characterized in that: include: The input module is used to input multi-temporal remote sensing image data; The time-frequency enhancement timing branch module has its input end connected to the input module and is used to process the multi-temporal remote sensing image data to obtain time-frequency enhancement timing features; A spectral spatial branching module, whose input end is connected to the input module, is used to process the multi-temporal remote sensing image data to obtain spectral spatial features; The cross-domain attention fusion module BCAF has its input connected to the output of the time-frequency enhancement temporal branch module and the spectral spatial branch module, and is used to fuse the time-frequency enhancement temporal features and the spectral spatial features to obtain fused features; And a classification head module, whose input is connected to the output of the cross-domain cross-attention fusion module, is used to classify the fused features to output a wetland vegetation classification map.

10. The wetland vegetation extraction system based on spatiotemporal feature fusion according to claim 9, characterized in that: The time-frequency enhancement timing branch module includes a frequency domain decomposition submodule, a ConvLSTM submodule, and a TFAF submodule; the spectral spatial branch module includes a spectral grouping and dilated convolution submodule and a spectral-scale attention fusion submodule; the cross-domain cross attention fusion module BCAF includes a query, key and value generation submodule and a bidirectional attention calculation submodule. The classification head module includes a 1×1 convolutional submodule, used to map the fused features to a category space and output a pixel-level wetland vegetation classification map. N cls This represents the number of land cover categories.

Images