Remote sensing image cross-domain classification method and system based on graph embedding feature alignment

Abstract

The application discloses a kind of graph embedding feature alignment remote sensing image cross-domain classification method and system, the method of the application includes extracting source domain feature and target domain feature, respectively generates feature map and distribution map, extracts graph embedding feature, and calculates the graph embedding feature alignment loss of source domain and target domain according to graph embedding feature;Source domain feature and target domain feature are classified using classifier respectively and classification loss is calculated, using the loss function comprising graph embedding feature alignment loss and classification loss, the weight parameters of target domain mapping layer, source domain mapping layer, feature extraction network and classifier are adjusted by back propagation, finally the target domain remote sensing image classification model of target domain mapping layer, feature extraction network and classifier is obtained after training is completed.The application aims to solve the problem of poor generalization ability of existing remote sensing image cross-domain classification method model, improve the generalization ability of remote sensing image classification model and the classification precision on target domain is higher.

Description

Technical Field

[0001] This invention relates to the field of image processing technology for hyperspectral images, and specifically to a method and system for cross-domain classification of remote sensing images based on graph embedding feature alignment. Background Technology

[0002] Hyperspectral images contain spectral information from dozens to hundreds of different bands, allowing for a more refined and comprehensive reflection of the spectral characteristics of ground features, providing richer data support for ground feature classification. Unlike traditional three-channel RGB images, hyperspectral images contain multiple channels, and the visualized result is more physically cubic, hence the term "data cube." Each pixel in a hyperspectral image contains information from several bands, and the spectral information of each pixel is a complete spectral line. The aggregation of all the spectral lines of pixels forms the "data cube," and the spectral information corresponding to each ground feature is different. Because hyperspectral images contain rich spectral and spatial information, they can reflect ground feature characteristics from different perspectives and capture more refined differences in ground features. Therefore, hyperspectral images are widely used in various fields such as urban construction, vegetation monitoring, and smart agriculture. The hyperspectral image classification task aims to assign each pixel to a specific category based on the ground feature information in the hyperspectral image, thereby achieving accurate ground feature classification. With the development and advancement of deep learning, single-domain hyperspectral image classification tasks have made significant progress, and classification accuracy has been continuously improved. However, hyperspectral imagery is complex and easily affected by environmental factors, lighting conditions, and sensor variations. Hyperspectral data of the same ground feature acquired at different times, under different weather conditions, or with different sensors often exhibit significant distributional differences. This distributional shift causes models that perform well in a single domain to fail to achieve the same classification performance on new samples, meaning the model's generalization ability is poor. Furthermore, hyperspectral image labeling primarily relies on manual annotation. Achieving good classification results on new data often requires extensive manual annotation work, which is time-consuming, labor-intensive, and costly. Against this backdrop, the cross-domain classification task for hyperspectral images has been proposed. Its core objective is to train a cross-domain classifier using fully labeled source domain data and unlabeled or partially labeled target domain data, enabling it to have good generalization ability and achieve equally good classification performance in both the source and target domains. Feature alignment is a crucial step in cross-domain classification of hyperspectral images. Its core idea is to use constraint methods such as Maximum Mean Difference (MMD) and Covariance Alignment (CORAL) to make the features of the source and target domains as similar as possible, reducing the distributional differences between the source and target domains, thereby improving the model's generalization ability and classification accuracy in the target domain. Combined with methods like few-shot learning, this can effectively alleviate the problems of high data annotation costs and weak model generalization ability in hyperspectral images. However, under current technological conditions, the classification accuracy of feature-aligned cross-domain hyperspectral image methods still cannot match that of single-domain classification tasks, hindering their widespread application. Further improving the classification accuracy of cross-domain feature alignment methods requires researchers to propose more effective data augmentation strategies, feature extraction methods, and feature alignment schemes.In-depth research on feature alignment-based cross-domain classification methods not only has significant theoretical research value, but can also greatly promote their application in practical scenarios such as urban construction, vegetation monitoring, and smart agriculture, providing more efficient and intelligent solutions for hyperspectral image analysis. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a method and system for cross-domain classification of remote sensing images based on graph embedding feature alignment, which addresses the above-mentioned problems in the prior art. This invention aims to solve the problem of poor generalization ability of existing cross-domain classification methods for remote sensing images, and improve the generalization ability of remote sensing image classification models and achieve higher classification accuracy in the target domain.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A cross-domain classification method for remote sensing images with graph embedding feature alignment includes the following steps: S101, divide the target domain remote sensing image and the source domain remote sensing image into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the target domain; the spatial patches of the source domain are processed by the source domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the source domain. S103, generate feature maps and distribution maps for source domain features and target domain features respectively; S104: Extract graph embedding features from the feature map and distribution map of the source domain, extract graph embedding features from the feature map and distribution map of the target domain, and calculate the graph embedding feature alignment loss of the source domain and the target domain based on the graph embedding features of the source domain and the target domain. S105: The source domain features and target domain features are classified using a classifier and the classification loss is calculated. Using a loss function that includes graph embedding feature alignment loss and classification loss, the weight parameters of the target domain mapping layer, source domain mapping layer, feature extraction network and classifier are adjusted through backpropagation. Finally, after training, a target domain remote sensing image classification model consisting of the target domain mapping layer, feature extraction network and classifier is obtained.

[0005] Optionally, in step S103, when generating feature maps and distribution maps for the source domain features and target domain features respectively, the process includes generating feature maps and distribution maps for the source domain features using the graph generation module GGM, and generating feature maps and distribution maps for the target domain features using the graph generation module GGM. Both the feature maps and distribution maps include a set of nodes and a set of edges. The source domain features and target domain features are respectively denoted as... Domain features For field type, , and Representing the source domain and the target domain respectively, Domain features are generated through the graph generation module GGM, which generates feature maps and distribution maps of the source domain, including: S201, Initialization Feature map of the domain: ; ; in, express Nodes in the initial feature map of the domain , Indicates the first indivual Domain characteristics; express Domain Metata Task Query Set express The domain's meta-task support set; express Nodes in the initial feature graph of the domain and nodes The edge between; This indicates a two-layer convolution operation. express Nodes in the initial feature map of the domain ; initialization Distribution map of the domain: ; ; in, express Nodes in the initial distribution graph of the domain ; This is a function that determines whether two inputs are the same; it takes the value 1 if the two inputs are the same, and 0 otherwise. and They represent the first The and the first indivual Category labels for domain features express The number of sample categories in the meta-task of the domain. Indicates the number of samples in each class. express Nodes in the initial distribution graph of the domain and nodes The edge between; express Nodes in the initial distribution graph of the domain ; S202, for The feature map and distribution map of the domain are iterated, and the iteration ends when the number of iterations equals a preset value to obtain the final result. The final feature map and distribution map of the domain; whereby the feature map undergoes the first... The function expression for the next iteration is: ; ; in, express Nodes in the feature map after the (k+1)th iteration of the domain , express Nodes in the feature graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the feature graph after the k-th iteration of the domain ,node For nodes The neighboring nodes, For the summation operation, This indicates a splicing operation. express Nodes in the feature map after the k-th iteration of the domain , express Nodes in the feature map after the (k+1)th iteration of the domain and nodes The edge between; express Nodes in the feature map after the (k+1)th iteration of the domain ; Distribution map is performed for the first time The function expression for the next iteration is: ; ; in, express Nodes in the distribution graph after the (k+1)th iteration of the domain , This represents a multilayer perceptron. express Nodes in the feature maps after the (k+1)th and kth iterations of the domain and nodes The edge between, express Nodes in the distribution graph after the k-th iteration of the domain , express Nodes in the distribution graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the distribution graph after the (k+1)th iteration of the domain .

[0006] Optionally, in step S104, when extracting graph embedding features from the feature map and distribution map of the source domain, and extracting graph embedding features from the feature map and distribution map of the target domain, the source domain and target domain are respectively denoted as... domain, For field type, , and Representing the source and target domains respectively, including the graph embedding feature extraction module GEEM, respectively... Extracting map embedding features from the feature map and distribution map of the domain, and then... The two graph embedding features of the domain are concatenated to obtain Graph embedding features of the domain; GEEM graph embedding feature extraction module for... Any type of graph in the feature map and distribution map of the domain. Extracted graph embedding features include: S301, for the diagram The feature matrix formed by the features of the nodes in the middle map By using the neighbor features of nodes in a random walk aggregate graph, the graph is analyzed. Perform feature enhancement to obtain the enhancement matrix : ; in, Represents the enhancement matrix The i-th row, This indicates the number of random walks. Indicates the length of the random walk. Representation of the characteristic matrix The OK; For graph type, , and These are the feature map and the distribution map, respectively. S302 will enhance the matrix Adjusting the shape yields the enhanced feature matrix. : ; in, The weight matrix is ​​shared by the feature maps and distribution maps of all domains. This indicates a concatenation operation, with the superscript T indicating a transpose operation; the enhanced feature matrix. The dimension is , Representation diagram The number of nodes in Represents the enhanced feature matrix The dimension of the enhanced feature matrix The dimension is equal to the enhanced feature matrix. The number of columns; S303, the enhanced feature matrix Graph extraction using graph attention network The graph embedding features are described in the graph attention network, which includes an attention weight calculation module, a multi-head attention module, and a flattening module. The attention weight calculation module is used to calculate the attention weight for each node. Calculate its relationship with neighboring nodes Attention weights: ; in, Represents a node with neighboring nodes Attention weights Represents the normalized exponential function, Represents a non-linear activation function. It is a learnable parameter matrix; for transpose, and These are the enhanced feature matrices. The row and number OK; The multi-head attention module is used to target the graph respectively. Each node calculates the output features: ; in, The attention module is the node. The calculated output features For an attention module consisting of K attention heads, For nodes The set of neighboring nodes, For nodes with neighboring nodes Attention weights For the enhanced feature matrix The Okay; will the diagram All nodes Output characteristics As a row, the output matrix is ​​formed by the output features of all nodes. ; The flattening module is used to flatten the output matrix. Flattening to obtain the diagram Graph embedding features: ; in, express Graph of the domain Graph embedding features, Indicates the flattening operation; The The two graph embedding features of the domain are concatenated to obtain The functional expression for the graph embedding features of the domain is: ; in, for Graph embedding features of the domain and for The graph embedding features of the feature map and distribution map of the domain correspond to, respectively. China Library The graph embedding features are extracted when the feature map and distribution map are used.

[0007] Optionally, the calculation function expression for the graph embedding feature alignment loss in step S104 is: ; in, For graph embedding feature alignment loss, The number of source domain features, The number of features in the target domain. This represents the kernel mapping function used to map to the regenerated Hilbert space. , For the graph embedding feature extraction module GEEM, and These are the feature map and distribution map of the source domain features, respectively. and These are the feature map and distribution map of the target domain features, respectively. Let denote the 2-norm in the regenerated Hilbert space.

[0008] Optionally, the function expression for the loss function including graph embedding feature alignment loss and classification loss in step S105 is as follows: ; ; ; in, The loss function includes graph embedding feature alignment loss and classification loss. Represents the source domain classification loss. This represents the target domain classification loss. Indicates alignment loss; and Let be the category labels of the i-th source domain feature and the target domain feature, respectively. and Let be the predicted categories of the i-th source domain feature and the target domain feature, respectively. and These are the i-th source domain feature and the target domain feature, respectively.

[0009] Optionally, both the target domain mapping layer and the source domain mapping layer are linear projection layers, the feature extraction network is a ghost convolutional network, which consists of multiple stacked 3D convolutional blocks and linear transformation operations; and the classifier is a KNN classifier.

[0010] Optionally, after step S105, the method further includes classifying the target domain remote sensing image using the target domain remote sensing image classification model: S101, Divide the remote sensing image of the target domain into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the target domain features. S103, classify the target domain features using a classifier to obtain the classification results for each spatial patch.

[0011] The present invention also provides a graph embedding feature aligned remote sensing image cross-domain classification system, including a microprocessor and a memory interconnected thereto, wherein the microprocessor is programmed or configured to execute the graph embedding feature aligned remote sensing image cross-domain classification method.

[0012] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the graph embedding feature aligned remote sensing image cross-domain classification method by a processor.

[0013] The present invention also provides a computer program product, including a computer program or instructions programmed or configured to execute the graph embedding feature aligned remote sensing image cross-domain classification method via a processor.

[0014] Compared with existing technologies, the present invention mainly achieves the following beneficial effects: The graph embedding feature alignment method for cross-domain classification of remote sensing images in the present invention includes extracting source domain features and target domain features, generating feature maps and distribution maps respectively, extracting graph embedding features, and calculating the graph embedding feature alignment loss of the source domain and target domain based on the graph embedding features; classifying the source domain features and target domain features respectively using a classifier and calculating the classification loss; using a loss function that includes graph embedding feature alignment loss and classification loss, adjusting the weight parameters of the target domain mapping layer, source domain mapping layer, feature extraction network, and classifier through backpropagation, and finally obtaining a target domain remote sensing image classification model composed of the target domain mapping layer, feature extraction network, and classifier after training. The graph embedding feature alignment method in the present invention, by using graph convolution to aggregate non-Euclidean information captured by the graph structure, and then aligning the source domain and target domain through metric learning, can solve the problem of poor generalization ability of existing cross-domain classification methods for remote sensing images, improve the generalization ability of remote sensing image classification models, and achieve higher classification accuracy in the target domain. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0016] Figure 2 This is a schematic diagram illustrating the working principle of the method in an embodiment of the present invention.

[0017] Figure 3 The results of the method of this embodiment and various existing cross-domain classification methods on the Salinas dataset as the target domain dataset are compared. (a) is the label map, (b) is the classification result of the SVM method, (c) is the classification result of the 3D-CNN method, (d) is the classification result of the SSRN method, (e) is the classification result of the DAAN method, (f) is the classification result of the DFSL-NN method, (g) is the classification result of the DFSL-SVM method, and (h) is the classification result of the method of this embodiment.

[0018] Figure 4 This document compares the results of the method of this embodiment and various existing cross-domain classification methods on the Indian Pines dataset as the target domain dataset. (a) is the label map, (b) is the classification result of the SVM method, (c) is the classification result of the 3D-CNN method, (d) is the classification result of the SSRN method, (e) is the classification result of the DAAN method, (f) is the classification result of the DFSL-NN method, (g) is the classification result of the DFSL-SVM method, and (h) is the classification result of the method of this embodiment. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0020] like Figure 1 and Figure 2 As shown, the remote sensing image cross-domain classification method based on graph embedding feature alignment in this embodiment includes the following steps: S101, divide the target domain remote sensing image and the source domain remote sensing image into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the target domain; the spatial patches of the source domain are processed by the source domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the source domain. S103, generate feature maps and distribution maps for source domain features and target domain features respectively; S104: Extract graph embedding features from the feature map and distribution map of the source domain, extract graph embedding features from the feature map and distribution map of the target domain, and calculate the graph embedding feature alignment loss of the source domain and the target domain based on the graph embedding features of the source domain and the target domain. S105: The source domain features and target domain features are classified using a classifier and the classification loss is calculated. Using a loss function that includes graph embedding feature alignment loss and classification loss, the weight parameters of the target domain mapping layer, source domain mapping layer, feature extraction network and classifier are adjusted through backpropagation. Finally, after training, a target domain remote sensing image classification model consisting of the target domain mapping layer, feature extraction network and classifier is obtained.

[0021] In step S101, when dividing the target domain remote sensing image and the source domain remote sensing image into spatial patches of a specified size, the size of the extracted spatial patches can be specified as needed, for example, 9×9. The extracted spatial patches can be represented as follows: ; in, Spatial patches representing the source and target domains, respectively. and These represent the number of spectral channels in the target domain remote sensing image and the source domain remote sensing image, respectively. The dimension is , The dimension is .

[0022] In step S102, the spatial patches of the target domain are processed by a target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the target domain, which can be represented as: ; In step S102, the spatial patches of the source domain are processed by a source domain mapping layer to adjust the spectral channels to a specified number, and then the source domain features are extracted by a feature extraction network. These features can be represented as follows: ; in, These represent the source domain mapping layer and the target domain mapping layer, respectively. This represents the spectral dimension after mapping. This represents the feature extraction network. In this embodiment, both the target domain mapping layer and the source domain mapping layer are linear projection layers with the same structure, both consisting of 1×1 convolution operations, but differing in the number of input spectral channels. Different. The feature extraction network can be selected as needed. As an optional implementation, the feature extraction network in this embodiment is a ghost convolutional network, which consists of multiple stacked 3D convolutional blocks and linear transformation operations.

[0023] In step S103 of this embodiment, when generating feature maps and distribution maps for source domain features and target domain features respectively, the process includes generating feature maps and distribution maps for the source domain features using the graph generation module GGM, and generating feature maps and distribution maps for the target domain features using the graph generation module GGM. This can be represented as follows: ; in, Feature maps representing the source / target domains A distribution diagram representing the source / target domain. This represents the support set and query set characteristics in the source / target domain meta-task. The graph generation module is GGM. Both the feature graph and the distribution graph include a set of nodes and a set of edges, denoted as source domain features and target domain features, respectively. Domain features For field type, , and If the source domain and the target domain are respectively, then: ; in, for Domain feature map, for Domain distribution map and For a set of nodes, and Let it be the set of edges.

[0024] In this embodiment, a graph generation module (GGM) is used to construct feature maps and distribution maps of the source and target domain data. A graph embedding feature extraction module aligns the data distributions of the source and target domains, enabling the model to extract non-Euclidean information from the data and align the source and target domains structurally. Random walks are introduced into the graph embedding feature extraction module to enhance features, thereby improving the model's information mining capabilities and achieving better classification performance. Specifically, in this embodiment, Domain features are generated through the graph generation module GGM, which generates feature maps and distribution maps of the source domain, including: S201, Initialization Feature map of the domain: ; ; in, express Nodes in the initial feature map of the domain , Indicates the first indivual Domain characteristics; express Domain Metata Task Query Set express The domain's meta-task support set; express Nodes in the initial feature graph of the domain and nodes The edge between; This indicates a two-layer convolution operation. express Nodes in the initial feature map of the domain ; initialization Distribution map of the domain: ; ; in, express Nodes in the initial distribution graph of the domain ; This is a function that determines whether two inputs are the same; it takes the value 1 if the two inputs are the same, and 0 otherwise. and They represent the first The and the first indivual Category labels for domain features express The number of sample categories in the meta-task of the domain. Indicates the number of samples in each class. express Nodes in the initial distribution graph of the domain and nodes The edge between; express Nodes in the initial distribution graph of the domain ; S202, for The feature map and distribution map of the domain are iterated, and the iteration ends when the number of iterations equals a preset value to obtain the final result. The final feature map and distribution map of the domain; whereby the feature map undergoes the first... The function expression for the next iteration is: ; ; in, express Nodes in the feature map after the (k+1)th iteration of the domain , express Nodes in the feature graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the feature graph after the k-th iteration of the domain ,node For nodes The neighboring nodes, For the summation operation, This indicates a splicing operation. express Nodes in the feature map after the k-th iteration of the domain , express Nodes in the feature map after the (k+1)th iteration of the domain and nodes The edge between; express Nodes in the feature map after the (k+1)th iteration of the domain ; Distribution map is performed for the first time The function expression for the next iteration is: ; ; in, express Nodes in the distribution graph after the (k+1)th iteration of the domain , This represents a multilayer perceptron. express Nodes in the feature maps after the (k+1)th and kth iterations of the domain and nodes The edge between, express Nodes in the distribution graph after the k-th iteration of the domain , express Nodes in the distribution graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the distribution graph after the (k+1)th iteration of the domain .

[0025] In step S104 of this embodiment, when extracting graph embedding features from the feature map and distribution map of the source domain, and extracting graph embedding features from the feature map and distribution map of the target domain, the source domain and target domain are respectively denoted as... domain, For field type, , and Representing the source and target domains respectively, including the graph embedding feature extraction module GEEM, respectively... Extracting map embedding features from the feature map and distribution map of the domain, and then... The two graph embedding features of the domain are concatenated to obtain Graph embedding features of the domain; GEEM graph embedding feature extraction module for... Any type of graph in the feature map and distribution map of the domain. Extracted graph embedding features include: S301, for the diagram The feature matrix formed by the features of the nodes in the middle map By using the neighbor features of nodes in a random walk aggregate graph, the graph is analyzed. Perform feature enhancement to obtain the enhancement matrix : ; in, Represents the enhancement matrix The i-th row, This indicates the number of random walks. Indicates the length of the random walk. Representation of the characteristic matrix The Okay, here we use an average method to aggregate the local information of the nodes; For graph type, , and These are the feature map and the distribution map, respectively; given the feature map... , It is a set of nodes. , It is a set of edges, for any node They all have a feature vector Then the feature matrix of the source domain feature map Perform T unbiased random walks of length L on each node to obtain the node. The augmented vector. The random walk embedding feature enhancement strategy improves model performance by mining graph information through random walks; S302 will enhance the matrix Adjusting the shape yields the enhanced feature matrix. : ; in, The weight matrix is ​​shared by the feature maps and distribution maps of all domains. This indicates a concatenation operation, with the superscript T indicating a transpose operation; the enhanced feature matrix. The dimension is , Representation diagram The number of nodes in Represents the enhanced feature matrix The dimension of the enhanced feature matrix The dimension is equal to the enhanced feature matrix. The number of columns; S303, the enhanced feature matrix Graph extraction using graph attention network The graph embedding features are described in the graph attention network, which includes an attention weight calculation module, a multi-head attention module, and a flattening module. The attention weight calculation module is used to calculate the attention weight for each node. Calculate its relationship with neighboring nodes Attention weights: ; in, Represents a node with neighboring nodes Attention weights Represents the normalized exponential function, Represents a non-linear activation function. It is a learnable parameter matrix; for transpose, and These are the enhanced feature matrices. The row and number OK; The multi-head attention module is used to target the graph respectively. Each node calculates the output features: ; in, The attention module is the node. The calculated output features For an attention module consisting of K attention heads, For nodes The set of neighboring nodes, For nodes with neighboring nodes Attention weights For the enhanced feature matrix The Okay; will the diagram All nodes Output characteristics As a row, the output matrix is ​​formed by the output features of all nodes. ; The flattening module is used to flatten the output matrix. Flattening to obtain the diagram Graph embedding features: ; in, express Graph of the domain Graph embedding features, Indicates the flattening operation; The The two graph embedding features of the domain are concatenated to obtain The functional expression for the graph embedding features of the domain is: ; in, for Graph embedding features of the domain and for The graph embedding features of the feature map and distribution map of the domain correspond to, respectively. China Library The graph embedding features are extracted when the feature map and distribution map are used.

[0026] The source domain meta-task is composed of randomly sampled source domain features and labeled target domain features. , Perform few-shot learning. Each meta-task contains... The support set and query set for each class of samples, with the support set containing the number of samples per class. The number of samples in each class of the query set is Based on this, this embodiment aligns the data features of the source and target domains through a graph embedding feature alignment strategy. Its core is the graph embedding feature alignment loss, which can be expressed as: ; in, Representative graph embedding feature alignment strategy; This represents all features of the support set and query set in the source domain meta-task; This represents all features in the support set and query set within the target domain meta-task. Specifically, the calculation function expression for the graph embedding feature alignment loss in step S104 is: ; in, For graph embedding feature alignment loss, The maximum mean difference is used to align the graph embedding features between the source and target domains; For the graph embedding feature extraction module GEEM, and These are the feature map and distribution map of the source domain features, respectively. and These are the feature map and distribution map of the target domain features, respectively, and we have: ; in, The number of source domain features, This represents the kernel mapping function used to map to the regenerated Hilbert space. , The number of features in the target domain. Let denote the 2-norm in the regenerated Hilbert space.

[0027] like Figure 2 As shown, the function expression for the loss function including graph embedding feature alignment loss and classification loss in step S105 is as follows: ; ; ; in, The loss function includes graph embedding feature alignment loss and classification loss. Represents the source domain classification loss. This represents the target domain classification loss. Indicates alignment loss; and Let be the category labels of the i-th source domain feature and the target domain feature, respectively. and Let be the predicted categories of the i-th source domain feature and the target domain feature, respectively. and These are the i-th source domain feature and the target domain feature, respectively.

[0028] Following step S105 in this embodiment, the method further includes classifying the target domain remote sensing image using the target domain remote sensing image classification model: S101, Divide the remote sensing image of the target domain into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the target domain features. S103, the target domain features are classified using a classifier to obtain the classification results for each spatial patch. The classifier can be any classifier required, for example, in this embodiment, the classifier is a KNN classifier.

[0029] To verify the effectiveness and performance of the graph embedding feature-aligned cross-domain classification method for remote sensing images in this embodiment, the Chikusei dataset was used as the source domain data, and the Salinas dataset and Indian Pines dataset were used as the target domain data for experiments. In the experiments, five samples were selected as labeled samples for each class in the target domain, and the rest were used as unlabeled samples to verify the classification accuracy of the proposed method. In this embodiment, it was compared with six cross-domain classification methods, with a total of three comparison indicators: overall classification accuracy (OA), average classification accuracy (AA), and kappa coefficient. The closer the values ​​of all three are to 100%, the better. The results of this embodiment method and several existing cross-domain classification methods on the Salinas dataset as the target domain dataset are compared as follows. Figure 3 As shown in Table 1, the results of this embodiment's method and various existing cross-domain classification methods on the Indian Pines dataset as the target domain dataset are compared. Figure 4 As shown in Table 2. Figure 3 and Figure 4 In the diagram, (a) is the label map, (b) is the classification result of the SVM method, (c) is the classification result of the 3D-CNN method, (d) is the classification result of the SSRN method, (e) is the classification result of the DAAN method, (f) is the classification result of the DFSL-NN method, (g) is the classification result of the DFSL-SVM method, and (h) is the classification result of the method in this embodiment.

[0030] Table 1: Objective performance metrics of the method in this embodiment and six cross-domain classification methods on the Salinas dataset.

[0031] Table 2: Objective performance metrics of the method in this embodiment and six cross-domain classification methods on the Indian Pines dataset.

[0032] In Tables 1 and 2, “ k" " indicates the Kappa coefficient.

[0033] As shown in Tables 1 and 2, in two scenarios where the Salinas dataset and the Indian Pines dataset are used as target domain data, the method in this embodiment outperforms existing methods such as SVM, 3D-CNN, SSRN, DAAN, DFSL-NN, and DFSL-SVM in three objective evaluation metrics: overall classification accuracy (OA), average classification accuracy (AA), and kappa coefficient. This demonstrates the superiority of the method in this embodiment. It is evident that the graph embedding feature-aligned remote sensing image cross-domain classification method in this embodiment captures non-Euclidean information of the source and target domain data through graph structure, further mines the implicit structural information of the data by combining a random walk strategy, and guides the model to focus on the structural similarity between the source and target domain data through a graph attention network, thereby improving the model's generalization ability. Compared with existing methods, this method achieves higher classification accuracy in the target domain.

[0034] This embodiment also provides a graph embedding feature-aligned remote sensing image cross-domain classification system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the graph embedding feature-aligned remote sensing image cross-domain classification method. This embodiment also provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the graph embedding feature-aligned remote sensing image cross-domain classification method via a processor. This embodiment also provides a computer program product including a computer program or instructions programmed or configured to execute the graph embedding feature-aligned remote sensing image cross-domain classification method via a processor.

[0035] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0036] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A cross-domain classification method for remote sensing images based on graph embedding feature alignment, characterized in that, Includes the following steps: S101, divide the target domain remote sensing image and the source domain remote sensing image into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the target domain; the spatial patches of the source domain are processed by the source domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the features of the source domain. S103, generate feature maps and distribution maps for source domain features and target domain features respectively; S104: Extract graph embedding features from the feature map and distribution map of the source domain, extract graph embedding features from the feature map and distribution map of the target domain, and calculate the graph embedding feature alignment loss of the source domain and the target domain based on the graph embedding features of the source domain and the target domain. S105, classify the source domain features and target domain features using a classifier and calculate the classification loss. Using a loss function that includes graph embedding feature alignment loss and classification loss, adjust the weight parameters of the target domain mapping layer, source domain mapping layer, feature extraction network and classifier through backpropagation. Finally, after training, a target domain remote sensing image classification model consisting of the target domain mapping layer, feature extraction network and classifier is obtained. In step S103, generating feature maps and distribution maps for source domain features and target domain features respectively includes generating feature maps and distribution maps for the source domain features using the graph generation module GGM, and generating feature maps and distribution maps for the target domain features using the graph generation module GGM. Both the feature maps and distribution maps include a set of nodes and a set of edges. The source domain features and target domain features are respectively denoted as... Domain features For field type, , and Representing the source domain and the target domain respectively, Domain features are generated through the graph generation module GGM, which generates feature maps and distribution maps of the source domain, including: S201, Initialization Feature map of the domain: ； ； in, express Nodes in the initial feature map of the domain , Indicates the first indivual Domain characteristics; express Domain Metata Task Query Set express The domain's meta-task support set; express Nodes in the initial feature graph of the domain and nodes The edge between; This indicates a two-layer convolution operation. express Nodes in the initial feature map of the domain ; initialization Distribution map of the domain: ； ； in, express Nodes in the initial distribution graph of the domain ; This is a function that determines whether two inputs are the same; it takes the value 1 if the two inputs are the same, and 0 otherwise. and They represent the first The and the first indivual Category labels for domain features express The number of sample categories in the meta-task of the domain. Indicates the number of samples in each class. express Nodes in the initial distribution graph of the domain and nodes The edge between; express Nodes in the initial distribution graph of the domain ; S202, for The feature map and distribution map of the domain are iterated, and the iteration ends when the number of iterations equals a preset value to obtain the final result. The final feature map and distribution map of the domain; whereby the feature map undergoes the first... The function expression for the next iteration is: ； ； in, express Nodes in the feature map after the (k+1)th iteration of the domain , express Nodes in the feature graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the feature graph after the k-th iteration of the domain ,node For nodes The neighboring nodes, For the summation operation, This indicates a splicing operation. express Nodes in the feature map after the k-th iteration of the domain , express Nodes in the feature map after the (k+1)th iteration of the domain and nodes The edge between; express Nodes in the feature map after the (k+1)th iteration of the domain ; Distribution map is performed for the first time The function expression for the next iteration is: ； ； in, express Nodes in the distribution graph after the (k+1)th iteration of the domain , This represents a multilayer perceptron. express Nodes in the feature maps after the (k+1)th and kth iterations of the domain and nodes The edge between, express Nodes in the distribution graph after the k-th iteration of the domain , express Nodes in the distribution graph after the (k+1)th iteration of the domain and nodes The edge between, express Nodes in the distribution graph after the (k+1)th iteration of the domain .

2. The remote sensing image cross-domain classification method based on graph embedding feature alignment according to claim 1, characterized in that, In step S104, graph embedding features are extracted from the feature map and distribution map of the source domain, and graph embedding features are extracted from the feature map and distribution map of the target domain, respectively, with the source domain and target domain recorded as... domain, For field type, , and Representing the source and target domains respectively, including the graph embedding feature extraction module GEEM, respectively... Extracting map embedding features from the feature map and distribution map of the domain, and then... The two graph embedding features of the domain are concatenated to obtain Graph embedding features of the domain; The graph embedding feature extraction module GEEM... Any type of graph in the feature map and distribution map of the domain. Extracting graph embedding features includes: S301, for the diagram The feature matrix formed by the features of the nodes in the middle map By using the neighbor features of nodes in a random walk aggregate graph, the graph is analyzed. Perform feature enhancement to obtain the enhancement matrix : ； in, Represents the enhancement matrix The i-th row, This indicates the number of random walks. Indicates the length of the random walk. Representation of the characteristic matrix The OK; For graph type, , and These are the feature map and the distribution map, respectively. S302 will enhance the matrix Adjusting the shape yields the enhanced feature matrix. : ； in, The weight matrix is ​​shared by the feature maps and distribution maps of all domains. This indicates a concatenation operation, with the superscript T indicating a transpose operation; the enhanced feature matrix. The dimension is , Representation diagram The number of nodes in Represents the enhanced feature matrix The dimension of the enhanced feature matrix The dimension is equal to the enhanced feature matrix. The number of columns; S303, the enhanced feature matrix Graph extraction using graph attention network The graph embedding features are described in the graph attention network, which includes an attention weight calculation module, a multi-head attention module, and a flattening module. The attention weight calculation module is used to calculate the attention weight for each node. Calculate its relationship with neighboring nodes Attention weights: ； in, Represents a node with neighboring nodes Attention weights Represents the normalized exponential function, Represents a non-linear activation function. It is a learnable parameter matrix; for transpose, and These are the enhanced feature matrices. The row and number OK; The multi-head attention module is used to target the graph respectively. Each node calculates the output features: ； in, The attention module is the node. The calculated output features For an attention module that includes K attention heads, For nodes The set of neighboring nodes, For nodes with neighboring nodes Attention weights For the enhanced feature matrix The Okay; will the diagram All nodes Output characteristics As a row, the output matrix is ​​formed by the output features of all nodes. ; The flattening module is used to flatten the output matrix. Flattening to obtain the diagram Graph embedding features: ； in, express Graph of the domain Graph embedding features, Indicates the flattening operation; The The two graph embedding features of the domain are concatenated to obtain The functional expression for the graph embedding features of the domain is: ； in, for Graph embedding features of the domain and for The graph embedding features of the feature map and distribution map of the domain correspond to, respectively. China Library The graph embedding features are extracted when the feature map and distribution map are used.

3. The remote sensing image cross-domain classification method based on graph embedding feature alignment according to claim 2, characterized in that, The expression for the calculation function of the graph embedding feature alignment loss in step S104 is: ； in, For graph embedding feature alignment loss, The number of source domain features, The number of features in the target domain. This represents the kernel mapping function used to map to the regenerated Hilbert space. , For the graph embedding feature extraction module GEEM, and These are the feature map and distribution map of the source domain features, respectively. and These are the feature map and distribution map of the target domain features, respectively. Let denote the 2-norm in the regenerated Hilbert space.

4. The remote sensing image cross-domain classification method based on graph embedding feature alignment according to claim 1, characterized in that, The functional expression for the loss function including graph embedding feature alignment loss and classification loss in step S105 is as follows: ； ； ； in, The loss function includes graph embedding feature alignment loss and classification loss. Represents the source domain classification loss. This represents the target domain classification loss. Indicates alignment loss; and Let be the category labels of the i-th source domain feature and the target domain feature, respectively. and Let be the predicted categories of the i-th source domain feature and the target domain feature, respectively. and These are the i-th source domain feature and the target domain feature, respectively.

5. The remote sensing image cross-domain classification method based on graph embedding feature alignment according to claim 1, characterized in that, Both the target domain mapping layer and the source domain mapping layer are linear projection layers. The feature extraction network is a ghost convolutional network, which consists of multiple stacked 3D convolutional blocks and linear transformation operations. The classifier is a KNN classifier.

6. The remote sensing image cross-domain classification method based on graph embedding feature alignment according to claim 1, characterized in that, Step S105 is followed by classifying the target domain remote sensing image using the target domain remote sensing image classification model: S101, Divide the remote sensing image of the target domain into spatial patches of a specified size; S102, the spatial patches of the target domain are processed by the target domain mapping layer to adjust the spectral channels to a specified number, and then the feature extraction network is used to extract the target domain features. S103, classify the target domain features using a classifier to obtain the classification results for each spatial patch.

7. A cross-domain classification system for remote sensing images based on graph embedding feature alignment, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the graph embedding feature alignment remote sensing image cross-domain classification method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the remote sensing image cross-domain classification method with graph embedding feature alignment as described in any one of claims 1 to 6.

9. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute, via a processor, the remote sensing image cross-domain classification method with graph embedding feature alignment as described in any one of claims 1 to 6.

Images