Remote sensing image classification method and system

Abstract

The invention discloses a remote sensing image classification method and system, and relates to the field of remote sensing image classification, and the method comprises the steps: obtaining a remote sensing image, and dividing the remote sensing image into a set number of patch blocks; according to a set number of patch blocks, adopting the trained remote sensing image classification model to obtain a ground feature classification result; the trained remote sensing image classification model sequentially comprises a linear processing layer, a plurality of stacked TMAN coding modules, a regularization layer and a multi-layer perceptron; the TMBA coding module comprises a spectrum TMBA encoder and a spatial graph convolutional neural network; the spectrum Mangbar encoder is used for extracting spectrum information through a global spectrum attention mechanism and self-adaptive residual error connection; the spatial graph convolutional neural network is used for capturing spatial context information by a self-adaptive filtering matrix; the information utilization rate and the classification precision of the remote sensing image can be improved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing image classification, and in particular to a remote sensing image classification method and system. Background Art

[0002] In order to improve the defects of spectral information in supervised classification, researchers have proposed many methods for supervised classification of remote sensing images in the past decade or so. These methods can be roughly summarized into two categories: traditional methods and deep learning methods. Traditional methods focus on discriminative feature extraction and compact expression of spectral data. Representative methods include: random forest, K-nearest neighbor method (KNN) and support vector machine (SVM). However, most of these methods identify pixels in remote sensing images as independent spectral curves, ignoring spatial structural information, resulting in low classification accuracy. At the same time, they are not robust in the presence of noise, missing information, etc.

[0003] In recent years, with the rapid development of deep learning, its powerful nonlinear expression ability has attracted widespread attention. Compared with traditional methods, deep learning methods can automatically extract high-level semantic information, thus avoiding tedious feature extraction engineering. Recurrent neural network (RNN) is a network suitable for processing sequence data, which can well capture the discriminative features in the sequence. Taking hyperspectral remote sensing images as an example, considering that the spectrum is a sequence, RNN can make full use of the high spectral resolution of hyperspectral images (HSI) to capture the long-range dependencies of features. For example, Zhou et al. used cascaded RNN to eliminate redundant information between spectral bands and mine complementary information in spectral sequences. Zhang et al. used a local spatial sequence method to extract low-level structural information and then passed it to RNN to generate high-level semantic information. Compared with RNN, Transformer can better capture global dependencies in sequences and has high flexibility. For example, Mei et al. introduced a group embedding module in Transformer to realize local-global spatial spectral context feature extraction. In order to make up for the lack of Transformer's spatial feature mining ability, Yang et al. proposed a convolutional neural network (CNN) and Transformer joint network to capture subtle spectral differences. In order to jointly extract spectral, spatial and semantic information in HSI, Xie et al. proposed a new semantic and spatial spectral feature fusion transformer (S3FFT) network to achieve the fusion of shallow spatial spectral features and high-level semantic information.

[0004] In addition, convolutional neural networks (CNNs) are also widely used in HSI classification due to their good spatial perception capabilities. In order to cope with the challenges of spectral information redundancy and insufficient spatial resolution, Chang et al. designed a comprehensive CNN, in which a two-dimensional convolutional neural network (2-D CNN) is used to extract spatial semantic information, and a three-dimensional convolutional neural network (3-D CNN) is responsible for extracting spatial spectral discriminative features to reduce information redundancy. Ye et al. creatively used the particle swarm algorithm to optimize the parameters of CNN, thereby improving the generalization performance and rapid learning ability of the model. In addition, autoencoders (AE), generative adversarial networks (GANs) and capsule networks (CapsNets) are also widely used in HSI classification.

[0005] Recently, a framework based on state space model (SSM) Mamba has been proposed. In language audio tasks, Mamba has been shown to outperform Transformer of the same size. Its selective processing of information, simpler design architecture and parallel scanning algorithm make it more efficient and have broad application prospects. The Mamba model is an advanced state space model (SSM) designed to efficiently process complex data-intensive sequences. The model achieves fast and efficient processing of long sequence data by combining linear layers, gating mechanisms and selective structured state space models. The core of the Mamba model lies in its selective mechanism, which can intelligently compress and filter contextual information and retain only the features that are critical to the modeling task, thereby significantly improving computational efficiency and model performance. This innovative design step overcomes the computational bottleneck of the traditional Transformer model when processing long sequences. The rapid development of Mamba has shown us its great potential and high adaptability in sequence data mining. However, high-dimensional spectral data and rich spatial information have brought challenges to the development and application of Mamba in remote sensing image classification.

[0006] However, existing methods have the following disadvantages: 1. The computational complexity of RNN is high during the training and inference stages, which limits the scalability of RNN when processing long sequences and large-scale data. 2. As the sequence length increases, the amount of computation of Transformer increases quadratically, becoming a bottleneck restricting its efficiency. 3. The weights of CNN convolution kernels are fixed and cannot adaptively capture changes in spatial structure. 4. Mamba is designed for sequence data mining. Although it performs well in mining spectral information, it tends to ignore spatial information.

[0007] In summary, the above methods often over-focus on spectral features and ignore spatial features. This series of problems makes it more challenging to efficiently process massive amounts of information and achieve accurate classification by combining spatial and spectral information.

[0008] Therefore, there is an urgent need to provide a remote sensing image classification method that can improve the information utilization and classification accuracy of remote sensing images. Summary of the invention

[0009] The purpose of this application is to provide a remote sensing image classification method and system, which can improve the information utilization rate and classification accuracy of remote sensing images.

[0010] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a remote sensing image classification method, the remote sensing image classification method comprising: Acquire a remote sensing image and divide the remote sensing image into a set number of patches; According to the set number of patch blocks, a trained remote sensing image classification model is used to obtain the object classification result; the trained remote sensing image classification model includes: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron; the Tumanba encoding module includes: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: The linear processing layer is used to flatten and linearly process a set number of patches to obtain the output features of the linear processing layer; The global spectral attention mechanism and adaptive residual connection in the spectral mamba encoder are used to extract the spectral information of the output features to obtain the spectral features; The adaptive filter matrix in the spatial graph convolutional neural network is used to capture the spatial context information of the spectral features and obtain the spatial-spectral joint features. According to the joint features of space and spectrum, regularization layer and multi-layer perceptron are used to obtain the classification results of ground objects.

[0011] Optionally, the spectral mamba encoder specifically includes: a global mask, a first branch, a second branch and a third branch; The first branch includes in sequence: a first linear code, a convolutional neural network, a first SiLU activation function, a state space model, a nonlinear code, and a second linear code; The second branch includes in sequence: a third linear code and a second SiLU activation function; The third branch includes: automatic residual connection; The input end of the global mask is connected to the linear processing layer; the output end of the global mask is connected to the first linear code and the third linear code; the second SiLU activation function is connected to the nonlinear code; the input end of the global mask is connected to the output end of the second linear code through a third branch.

[0012] Optionally, the calculation process of the global mask includes: ; in, is the output feature of the linear processing layer, , , To learn the weight parameters, represents 2-D convolution, represents the features after linear processing, is the output feature after global mask processing.

[0013] Optionally, the calculation process of automatic residual connection is: ; in, is a learnable parameter, is the output feature of the automatic residual connection, For the The layer encodes input features, For the -1 layer encodes input features.

[0014] Optionally, the computation process of the spatial graph convolutional neural network is: ; in, is the empty spectrum joint feature, is the adaptive filter matrix, is the total number of p-hop neighbor nodes of the node, n represents the p-hop neighbor nodes of the node, is the p-hop aggregation feature of the node, and when n=0, Spectral characteristics , , and Represents the adjacency matrix of n-hop connection The estimated value and degree matrix of The estimated value of represents the activation function, is a learnable shared parameter vector.

[0015] Optionally, the adaptive filter matrix Medium Element The calculation process is: ; in, is the i-th node in the remote sensing image, is the jth node in the remote sensing image, =0.2 is the empirical value, is the adjacency matrix exist The corresponding value at, exp() is a mathematical function.

[0016] In a second aspect, the present application provides a remote sensing image classification system, the remote sensing image classification system comprising: A remote sensing image preprocessing module is used to obtain remote sensing images and divide the remote sensing images into a set number of patches; The module for determining the result of ground object classification is used to obtain the ground object classification result according to the set number of patches by using the trained remote sensing image classification model; the trained remote sensing image classification model comprises: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron in sequence; the Tumanba encoding module comprises: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: The linear processing layer is used to flatten and linearly process a set number of patches to obtain the output features of the linear processing layer; The global spectral attention mechanism and adaptive residual connection in the spectral mamba encoder are used to extract the spectral information of the output features to obtain the spectral features; The adaptive filter matrix in the spatial graph convolutional neural network is used to capture the spatial context information of the spectral features and obtain the spatial-spectral joint features. According to the joint features of space and spectrum, regularization layer and multi-layer perceptron are used to obtain the classification results of ground objects.

[0017] According to the specific embodiments provided in this application, this application discloses the following technical effects: This application provides a remote sensing image classification method and system, which mines spectral information through the GraphMamba coding module, and aggregates spectral information according to the correlation between bands by introducing a global mask, further reducing information redundancy interference, and reducing computational overhead through parallel scanning. SpatialGCN, a spatial graph convolutional neural network with enhanced context-awareness, uses a multi-order adaptive neighborhood matrix to better mine dynamic spatial structural features. Automatic residual connection (AutoRes) is used to selectively accept original information, adaptively fuse output features and original information, and mitigate feature over-smoothing; this application can achieve deep joint mining of spatial-spectral information of remote sensing images and improve classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A schematic diagram of a remote sensing image classification method in an embodiment of the present application; Figure 2 This is a schematic diagram of the overall architecture of the remote sensing image classification method; Figure 3 Schematic diagram of the spectral Mamba encoder structure; Figure 4 Classification effect diagram of IP dataset using different methods (where (a) is a pseudo-color image, (b) is a training label, (c) is a test label, (d) is SVM-RBF, (e) is 3D-CNN, (f) is DFFN, (g) is RSSAN, (h) is GAHT, (i) is AB-LSTM, (j) is SF, (k) is WFCG, (l) is CACFTNet, (m) is HSDBIN, and (n) is GraphMamba); Figure 5 SA dataset classification effect diagram of different methods (where (a) is a pseudo-color image, (b) is a training label, (c) is a test label, (d) is SVM-RBF, (e) is 3D-CNN, (f) is DFFN, (g) is RSSAN, (h) is GAHT, (i) is AB-LSTM, (j) is SF, (k) is WFCG, (l) is CACFTNet, (m) is HSDBIN, and (n) is GraphMamba); Figure 6Classification effect diagram of UH2018 dataset using different methods (where (a) is a pseudo-color image, (b) is a training label, (c) is a test label, (d) is SVM-RBF, (e) is 3D-CNN, (f) is DFFN, (g) is RSSAN, (h) is GAHT, (i) is AB-LSTM, (j) is SF, (k) is WFCG, (l) is CACFTNet, (m) is HSDBIN, and (n) is GraphMamba). DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0022] In an exemplary embodiment, Figure 1 and Figure 2 As shown, a remote sensing image classification method is provided, which includes the following S101 to S102. Among them: S101, Acquisition of Remote Sensing Images , and divide the remote sensing image h into N pieces of size Patch block ; Where P is the patch block size, D is the patch size The feature dimension of the remote sensing image is a hyperspectral image (HSI) in this embodiment. Three-dimensional remote sensing images Expand into a two-dimensional patch , where H, W, and D represent the length, width, and spectral dimensions of the remote sensing image, respectively, P is the size of the patch, and N represents the number of patches; S102, according to the set number of patch blocks, using the trained remote sensing image classification model to obtain the object classification result; the trained remote sensing image classification model includes: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron; the Tumanba encoding module includes: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: S1, using the linear processing layer to process a set number of patches Flatten and linearize the patch blocks Linearly project from D dimension to C dimension and introduce position vector , get the output features of the linear processing layer ; The classified pixel features in each patch Extracted as classification features; ; in, is the nth patch block, is the learnable parameter vector, is the output feature after linear processing, is the position vector.

[0023] S2, extracting output features using the global spectral attention mechanism and adaptive residual connections in the spectral mamba encoder The spectral information of ; S3, Capturing spectral features using adaptive filter matrices in spatial graph convolutional neural networks The spatial context information of ; S4, based on the spatial spectrum joint features , use the regularization layer and multi-layer perceptron (Multi-LayerPerceptron, MLP) to predict the value y, and calculate the loss function for back propagation. After L rounds of training, the trained network model is obtained. The test sample is predicted according to the trained network to obtain the predicted value of the test sample, that is, the ground object classification result.

[0024] like Figure 3 As shown, the spectral mamba encoder specifically includes: a global mask (global mask), a first branch, a second branch and a third branch; The first branch includes in sequence: a first linear code, a convolutional neural network, a first SiLU activation function, a state space model, a nonlinear code, and a second linear code; The second branch includes in sequence: a third linear code and a second SiLU activation function; The third branch includes: automatic residual connection; The input end of the global mask is connected to the linear processing layer; the output end of the global mask is connected to the first linear code and the third linear code; the second SiLU activation function is connected to the nonlinear code; the input end of the global mask is connected to the output end of the second linear code through a third branch.

[0025] The calculation process of the global mask includes: ; in, is the output feature of the linear processing layer, , , To learn the weight parameters, represents 2-D convolution, Represents the features after linear processing, further through convolution Form a global mask U, is the output feature after global mask processing.

[0026] Residual connections are often used between different encoding blocks to promote information exchange and gradient propagation between networks, improve the model's expressiveness and feature utilization; and in order to reduce the model's sensitivity to noise, automatic residual connections (AutoRes) are used to selectively accept original information, adaptively fuse output features and original information, and slow down feature over-smoothing; the calculation process of automatic residual connections is: ; in, is a learnable parameter, is the output feature of the automatic residual connection, For the The layer encodes input features, For the -1 layer encodes input features.

[0027] In order to adaptively capture spatial context information, a weighted multi-order spatial graph convolutional coding (SpatialGCN) is proposed to further improve the generalization ability of the model. The calculation process of the spatial graph convolutional neural network is as follows: ; in, is the empty spectrum joint feature, is the adaptive filter matrix, is the total number of p-hop neighbor nodes of the node, n represents the p-hop neighbor nodes of the node, is the p-hop aggregation feature of the node, and when n=0, Spectral characteristics , , and Represents the adjacency matrix of n-hop connection The estimated value and degree matrix of The estimated value of represents the activation function, is a learnable shared parameter vector. When p=1, The output of normal GCN with residual connections provides good scalability and adaptability.

[0028] Adaptive filter matrix Medium Element The calculation process is: ; in, is the i-th node in the remote sensing image, is the jth node in the remote sensing image, =0.2 is the empirical value, is the adjacency matrix exist The corresponding value at, exp() is a mathematical function.

[0029] SpatialGCN uses a shared weight approach to effectively reduce the number of parameters and thus relieve computational pressure. The adaptive filter matrix can abstract the correlation between different nodes and better extract spatial structure features.

[0030] Classified pixel features Normalize it and feed it into the multi-layer perceptron to obtain the final prediction vector : ; ; in, Target object pixel Normalized output features. and They represent the regularization layer and the multi-layer perceptron respectively. This application directly uses pixel features for classification, which makes the extracted features more targeted and can further simplify the network structure.

[0031] In order to efficiently build a bridge for information exchange between different bands and pay attention to more discriminative information, this application proposes a global mask to aggregate spectral information according to the correlation between bands and alleviate the problem of information redundancy. Thus, through the spectral domain attention mechanism, the influence of redundant information is reduced and discriminative spectral information is extracted.

[0032] Residual connections are often used between different coding blocks to promote information exchange and gradient propagation between networks, improve the model's expressiveness and feature utilization. For HSI images, in order to reduce the model's sensitivity to noise, automatic residual connections (AutoRes) are used to selectively accept original information. Automatic residual connections adaptively fuse output features and original information, and reduce the phenomenon of feature over-smoothing.

[0033] In order to adaptively capture spatial context information, a weighted multi-order spatial graph convolutional coding (SpatialGCN) is proposed to further improve the generalization ability of the model; HSI images represent the differences in ground objects through the size difference of pixel values ​​(reflecting the spectral information of the ground objects) and spatial changes (reflecting the spatial information of the ground objects). Compared with RNN and Transformer, Mamba uses SSM to dynamically filter and process information based on the input, allowing the model to selectively remember or ignore the input part, so it can process spectral sequences more efficiently.

[0034] The present application provides a general and scalable Mamba-based HSI classification baseline network, which is a new HSI processing paradigm. The method of establishing a space spectrum cube to retain local information while inputting block by block can reduce the input amount, reduce the problems of spectral redundancy and high consumption of computing resources, and reduce the calculation pressure of the model.

[0035] Experimental verification was conducted on the following public hyperspectral remote sensing image datasets, using three real benchmark datasets, Indian Pines (IP), Salina (SA), and University of Houston 2018 (UH2018), to test the algorithm performance from different angles. The IP dataset is used to test the classification ability under the condition of uneven sample distribution, the SA dataset is used to verify the classification ability under the condition of similar spectra of different objects, and the UH2018 dataset is used to verify the classification performance of large-scale datasets with high spatial resolution. Smaller-scale data was used for training, with 2%, 0.2%, and 0.2% of the labeled samples used for training on IP, SA, and UH2018, respectively.

[0036] The overall classification accuracy OA is used to observe the classification accuracy of all samples. The average classification accuracy (AA) is used to characterize the average of the classification accuracy of different categories. The Kappa coefficient is used as an indicator to judge the degree of consistency. In addition, the classification accuracy of each category is specifically displayed. The higher the index, the better the effect.

[0037] In order to fully ensure the reliability of the experimental results, the most advanced baseline experiments are used for detailed comparison, which can be roughly divided into the traditional machine learning method SVM-RBF, CNN-based methods 3D CNN, DFFN and RSSAN, LSTM-based methods AB-LSTM, Transformer-based methods SpectralFormer (SF), GAHT, and GCN and CNN-based fusion methods WFCG. The quantitative comparison results of the CNN and Transformer-based fusion methods CACFTNet and HSDBIN are shown in Tables 1-3, and the qualitative comparison results are shown in Figure 4-Figure 6 shown.

[0038] Table 1 Quantitative classification results on the IP dataset

[0039] Table 2 Quantitative classification results on the SA dataset

[0040] Table 3 Quantitative classification results on the UH2018 dataset

[0041] Table 1 shows the comparison results of different methods with GraphMamba. In the experiment, we selected different baseline methods based on CNN, LSTM, GCN, Transformer and their improved methods or fusion methods for comparison, so as to more comprehensively test the effectiveness of Mamba in HSI classification. From Table 1, it can be seen that the classification accuracy of the LSTM-based network is low, because it only ignores the joint features of the space spectrum from the spectral perspective. The CNN-based and Transformer-based methods have better classification effects and can better capture local spectral information. The fusion methods WFCG and CACFTNet based on GCN-CNN and Transformer-CNN have achieved good classification results due to their strong space-spectrum modeling capabilities.

[0042] Compared with other models, GraphMamba achieved the best classification performance in terms of OA, AA, and Kappa, and its performance was improved by 5.3%, 2.38%, and 6.02% respectively compared with the second best classification effect. This shows the effectiveness of the graph structure learning Mamba proposed in this application.

[0043] For visualization results, traditional methods such as SVM-RBF are prone to salt and pepper noise, while the classification map of the deep learning-based classification method is smoother. In addition, the proposed method intuitively handles edge objects better and greatly reduces edge noise. This is due to the context-aware GCN encoding module used, which can better aggregate spatial information and reduce misclassification. In addition, the ability of HyperMmaba encoder to model sequences well also reduces noise in the classification process.

[0044] Since different objects in the SA dataset have similar features, the difficulty of classification is increased. In this case, GraphMamba still has the strongest classification results. Compared with the second best classification method, its OA, AA and Kappa are improved by 1.14%, 1.62% and 1.27% respectively. In addition, the classification results of this application have the smallest standard deviation, which shows that the algorithm has strong stability. GraphMamba better integrates spatial-spectral features by establishing a multi-scale contextual attention mechanism, and obtains top classification results among all methods.

[0045] Since the distance between objects is relatively close, misclassification and noise are prone to occur at the edges. GraphMamba solves the above problems well, with clear boundaries of the edges of objects and a smoother visual effect.

[0046] For the large-scale dataset UH2018, its larger scale and more dispersed land cover types pose a significant challenge to accurate classification. Due to the limited number of training samples and the higher classification difficulty, the overall accuracy of these classification algorithms on the UH2018 dataset is lower than that of the IP and SA datasets. GraphMmaba used 1010 pixels (0.2% of the total pixels) and achieved an overall accuracy of 86.45%. Compared with the suboptimal classification performance, its OA improved by 3.41%. In addition, the smaller standard deviation indicates the stability of its algorithm. Mamba's natural advantage in processing sequences in HyperMamba, coupled with the proposed global mask module, enhances its ability to capture spectral information. Compared with CNN, the SpatialGCN module can process spatial information more flexibly and is also effective in the environment of large-scale heterogeneous objects.

[0047] from Figure 6 It can be seen that GraphMamba can achieve accurate classification even for smaller objects, while other methods tend to misclassify smaller objects. GraphMamba's texture processing is also quite satisfactory, showing a high degree of distinction between different categories. This shows that integrating spatial and spectral information for feature modeling can better represent object relationships and improve classification performance.

[0048] Based on the same inventive concept, the embodiment of the present application also provides a remote sensing image classification system for implementing the remote sensing image classification method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more remote sensing image classification system embodiments provided below can refer to the limitations on the remote sensing image classification method above, and will not be repeated here.

[0049] In an exemplary embodiment, a remote sensing image classification system is provided, comprising: A remote sensing image preprocessing module is used to obtain remote sensing images and divide the remote sensing images into a set number of patches; The module for determining the result of ground object classification is used to obtain the ground object classification result according to the set number of patches by using the trained remote sensing image classification model; the trained remote sensing image classification model comprises: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron in sequence; the Tumanba encoding module comprises: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: The linear processing layer is used to flatten and linearly process a set number of patches to obtain the output features of the linear processing layer; The global spectral attention mechanism and adaptive residual connection in the spectral mamba encoder are used to extract the spectral information of the output features to obtain the spectral features; The adaptive filter matrix in the spatial graph convolutional neural network is used to capture the spatial context information of the spectral features and obtain the spatial-spectral joint features. According to the joint features of space and spectrum, regularization layer and multi-layer perceptron are used to obtain the classification results of ground objects.

[0050] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0051] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0052] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0053] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0054] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A remote sensing image classification method, characterized in that: The remote sensing image classification method comprises: Acquire a remote sensing image and divide the remote sensing image into a set number of patches; According to the set number of patch blocks, a trained remote sensing image classification model is used to obtain the object classification result; the trained remote sensing image classification model includes: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron; the Tumanba encoding module includes: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: The linear processing layer is used to flatten and linearly process a set number of patches to obtain the output features of the linear processing layer; The global spectral attention mechanism and adaptive residual connection in the spectral mamba encoder are used to extract the spectral information of the output features to obtain the spectral features; The adaptive filter matrix in the spatial graph convolutional neural network is used to capture the spatial context information of the spectral features and obtain the spatial-spectral joint features. According to the joint features of space and spectrum, regularization layer and multi-layer perceptron are used to obtain the classification results of ground objects.

2. The remote sensing image classification method according to claim 1, characterized in that: The spectral mamba encoder specifically includes: a global mask, a first branch, a second branch and a third branch; The first branch includes in sequence: a first linear code, a convolutional neural network, a first SiLU activation function, a state space model, a nonlinear code, and a second linear code; The second branch includes in sequence: a third linear code and a second SiLU activation function; The third branch includes: automatic residual connection; The input end of the global mask is connected to the linear processing layer; the output end of the global mask is connected to the first linear code and the third linear code; the second SiLU activation function is connected to the nonlinear code; the input end of the global mask is connected to the output end of the second linear code through a third branch.

3. The remote sensing image classification method according to claim 2, characterized in that: The calculation process of the global mask includes: ； in, is the output feature of the linear processing layer, , , To learn the weight parameters, represents 2-D convolution, represents the features after linear processing, is the output feature after global mask processing.

4. The remote sensing image classification method according to claim 3, characterized in that: The calculation process of automatic residual connection is: ； in, is a learnable parameter, is the output feature of the automatic residual connection, For the The layer encodes input features, For the -1 layer encodes input features.

5. The remote sensing image classification method according to claim 1, characterized in that: The calculation process of the spatial graph convolutional neural network is: ； in, is the empty spectrum joint feature, is the adaptive filter matrix, is the total number of p-hop neighbor nodes of the node, n represents the p-hop neighbor nodes of the node, is the p-hop aggregation feature of the node, and when n=0, Spectral characteristics , , and Represents the adjacency matrix of n-hop connection The estimated value and degree matrix of The estimated value of represents the activation function, is a learnable shared parameter vector.

6. The remote sensing image classification method according to claim 5, characterized in that: Adaptive filter matrix Medium Element The calculation process is: ； in, is the i-th node in the remote sensing image, is the jth node in the remote sensing image, =0.2 is the empirical value, is the adjacency matrix exist The corresponding value at, exp() is a mathematical function.

7. A remote sensing image classification system, characterized in that: The remote sensing image classification system comprises: A remote sensing image preprocessing module is used to obtain remote sensing images and divide the remote sensing images into a set number of patches; The module for determining the result of ground object classification is used to obtain the ground object classification result according to the set number of patches by using the trained remote sensing image classification model; the trained remote sensing image classification model comprises: a linear processing layer, a plurality of stacked Tumanba encoding modules, a regularization layer and a multi-layer perceptron in sequence; the Tumanba encoding module comprises: a spectral mamba encoder and a spatial graph convolutional neural network; the spectral mamba encoder is used to extract spectral information through a global spectral attention mechanism and an adaptive residual connection; the spatial graph convolutional neural network is used to adaptively filter the matrix to capture spatial context information; The process of determining the ground feature classification results is as follows: The linear processing layer is used to flatten and linearly process a set number of patches to obtain the output features of the linear processing layer; The global spectral attention mechanism and adaptive residual connection in the spectral mamba encoder are used to extract the spectral information of the output features to obtain the spectral features; The adaptive filter matrix in the spatial graph convolutional neural network is used to capture the spatial context information of the spectral features and obtain the spatial-spectral joint features. According to the joint features of space and spectrum, regularization layer and multi-layer perceptron are used to obtain the classification results of ground objects.

Images