A hyperspectral wetland image classification method based on graph capsule neural network

Abstract

The application discloses a hyperspectral wetland image classification method based on a graph capsule neural network, which comprises the following steps: S1, learning feature transformation is performed on an adversarial domain self-adaptive framework, so that source domain samples and target domain samples of a hyperspectral wetland image are matched in features; S2, a graph capsule neural domain self-adaptive network structure is constructed, domain-invariant features and domain-related features are extracted, and transferable features are discovered and shared across domains; and S3, a coupling structure two-classifier is designed, the two-classifier is trained by using the source domain samples, classification differences of the target domain samples are maximized, and precise classification of the hyperspectral wetland image is realized by identifying a classification boundary. Meanwhile, the application discovers transferable knowledge and realizes cross-domain sharing, enhances effective discrimination of a class boundary, and finally realizes precise classification of the hyperspectral wetland image under conditions of unknown regions, complex scenes, and lack, deficiency and imbalance of data types.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a hyperspectral wetland image classification method based on a graph capsule neural network. BACKGROUND

[0002] A hyperspectral remote sensing image is stacked by a hyperspectral imaging spectrometer to contain tens or hundreds of continuous spectral bands containing feature information of ground objects. The hyperspectral remote sensing image not only contains rich spectral information, but also has good spatial characteristics, and has the characteristics of image and spectrum integration, providing a good data basis for ground object classification. The hyperspectral image classification technology refers to determining and labeling the class attributes of the ground object represented by each pixel in the hyperspectral image data according to the different spectral information of different ground objects, and has been applied to the fields of agricultural management, environmental monitoring, geological exploration, military reconnaissance, etc.

[0003] Due to the influence of problems such as being easily blocked by clouds and rain, complexity of species, data type missing / deficiency / imbalance, data / label distribution difference, etc., the existing hyperspectral image classification algorithm cannot meet the application requirements of wetland intelligent monitoring and accurate classification. At the same time, considering the complex environment of the wetland ecosystem, the difficulty of on-site detection, the rich types of vegetation, the difference in distribution and the lack of obvious boundaries, and the influence of climate / season, etc., how to realize fine classification of hyperspectral wetland images under the conditions of limited samples and complex scenes is an urgent problem to be solved. In addition, most of the existing fine classification algorithms for wetlands use traditional machine learning methods, which have poor generalization ability for unknown areas / complex scenes and cannot be applied to actual situations under large scenes and large data. Therefore, exploring a hyperspectral wetland image fine classification framework with high accuracy, strong robustness and low complexity is still a research hotspot in wetland protection. SUMMARY

[0004] In view of the above shortcomings of the prior art, the present application provides a hyperspectral wetland image classification method based on a graph capsule neural network, which is used to solve the problem that the traditional classification method cannot effectively handle the scene complexity and detection difficulty existing in the wetland ecosystem.

[0005] To achieve the above-mentioned application purposes, the technical solutions adopted by the present application are as follows:

[0006] A hyperspectral wetland image classification method based on a graph capsule neural network is provided, which comprises the following steps:

[0007] S1: Perform learning feature transformation on the generated adversarial domain adaptation framework to match the source domain samples and the target domain samples of the hyperspectral wetland image in features;

[0008] S2: Construct a graph capsule neural domain adaptive network structure, extract domain-invariant and domain-related features, discover transferable features and share them across domains;

[0009] S3: Design a coupled binary classifier, train the binary classifier using source domain samples, maximize the classification difference of target domain samples, and achieve accurate classification of hyperspectral wetland images by identifying classification boundaries.

[0010] Further, step S1 includes:

[0011] S11: Select from the source region hyperspectral scene N Images of wetlands in 10 categories, with each category containing 10 images. K A wetland image sample was selected from the target domain scene. H A sample of labeled wetland images;

[0012] S12: From the perspective of adaptive generation domain, combined with the expected error theory of target domain, a generative adversarial adaptive framework is constructed.

[0013] S13: Establish a domain classifier to explore the differences between source domain samples and target domain samples, and perform feature matching between source domain samples and target domain samples.

[0014] S14: Construct two different classifiers in the source domain. Train two different classifiers This enables it to correctly classify source domain samples; the classifier The objective function is:

[0015]

[0016] in, X s For source domain samples, Y s The labels corresponding to the source domain samples. p ( y | x s () is a source domain sample X s The middle category is y The probability of a class; x s For a subsample in the source domain, SSF For generator, (.) represents the classifier. The objective function, K The number of samples selected for the source domain. k Selected for the source domain K One of the samples;

[0017] Further, the objective function of the generative adversarial domain adaptation framework is:

[0018]

[0019] wherein, and are the generalization errors of the source domain and the target domain respectively, represents the difference between the domains, H is the hypothesis space, λ is the optimal joint generalization error, ; h is the feature of the source domain and the target domain sample mapped to the hypothesis space H , S is the source domain sample space, T is the target domain sample space;

[0020] Further, the objective function of the domain classifier is:

[0021]

[0022] wherein, is the indicator function, h ( x ) is the domain classifier output of the source domain, h ( x ) is the domain classifier output of the target domain, E is the expectation operation;

[0023] Further, the step S2 comprises:

[0024] S21: establishing a graph capsule neural domain adaptation network, mapping the source domain sample and the target domain sample to the same feature space to obtain an adjacency matrix and a node feature matrix , and constructing a fully connected graph structure:

[0025]

[0026] wherein, , M is the number of nodes of the fully connected graph structure, M s is the number of source domain sample nodes, M t is the number of target domain sample nodes, is the node feature dimension extracted by the feature extraction network;( X 0 ) T is the transpose matrix of the node feature matrix;

[0027] S22: input the full connection graph structure into the deep hierarchical graph neural network to learn the hierarchy within the same class and between different classes of the wetland image samples;

[0028] S23: jointly learn the hierarchical relationship between the source domain and the target domain wetland image nodes, reduce the difference between the wetland image samples in different domains, extract domain invariant features and domain related features, find transferable features and share across domains.

[0029] Further, step S23 comprises:

[0030] The bottom-up reasoning phase:

[0031] S231: construct a bottom-up reasoning module, sample the source domain sample nodes within the same class and between different classes, retain the target domain sample nodes, and sample the source domain sample nodes hierarchically; the objective function of the bottom-up reasoning module is:

[0032]

[0033]

[0034]

[0035] wherein, is the input feature matrix of the L layer bottom-up reasoning module, is the graph convolution process, and are the classification functions of the source domain sample nodes within the same class and between different classes, respectively, is the number of subgraphs of the layer, is the node feature matrix of the L th subgraph of the i layer before downsampling, is a random node in the i th subgraph, is a neighborhood set, is the distance from to , is the node feature matrix of the L th subgraph of the i layer after downsampling;

[0036] S232: construct a new full connection graph according to the sampled wetland image sample nodes to obtain the adjacency matrix of the current bottom-up reasoning module;

[0037] S233: Repeat steps S231-S232, stacking multiple bottom-up reasoning modules to extract multi-level feature representations and deep map relationships of wetland images in the source and target domains;

[0038] Top-down reasoning stage:

[0039] S234: Combining the location information of wetland image sample nodes in each bottom-up inference module, a top-down inference module is constructed. By sampling new graph nodes, the hierarchical graph structure within the same category and between different categories is explored layer by layer to mine the hierarchical relationships between nodes within the same category and between different categories of wetland image samples in the source or target domain. The objective function of the top-down inference module is:

[0040]

[0041] in, For the first L The input feature matrix of the +1 layer top-down inference module This is the node feature matrix output after upsampling. P L For the first L Position information of graph nodes in the bottom-up reasoning module;

[0042] S235: Utilize the correlation between wetland image samples in the source and target domains to reduce the differences between wetland image samples in different domains, extract domain-invariant and domain-related features, discover transferable features, and share them across domains.

[0043] Further, step S3 includes:

[0044] S31: Fixed Two Classifier The generator SSF is trained to extract features from the source and target domains, ensuring that the differences between the domains are minimized and a binary classifier is used. Minimize the differences among wetland image samples in the classification target domain:

[0045]

[0046] in, and Two classifiers , and Corresponding to p 1 and p 2 pairs k The probability of the class output; X t For the target domain sample, n For the number of source domain categories, m For training batch size, and Respectively, the same batch of data;

[0047] S32: fixed generator SSF, training two classifiers Maximize the classification difference of two classifiers on the target domain wetland image samples:

[0048] ;

[0049] S33: get the optimal classifier and generator, realize the fine classification of hyperspectral wetland images in the target domain through the adversarial training strategy.

[0050] The beneficial effects of the present application are: the present application takes the cross-domain data representation of the graph capsule neural network as the core, aiming at the problems of complex environment, difficult field detection, rich vegetation types, distribution difference and lack of obvious boundary, easy to be affected by climate / season, etc. in the wetland ecosystem, starting from the perspective of generative adversarial domain adaptation, a hyperspectral image space and spectral feature joint learning method is proposed to fully exploit the correlation between source domain and target domain data (same sensor, similar ground material, same scene, etc.), while discovering transferable knowledge and realizing cross-domain sharing, enhancing the effective discrimination of class boundaries, and finally realizing the precise classification of hyperspectral wetland images in unknown areas, complex scenes, and conditions of data type missing, insufficient, and unbalanced. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The flowchart of the hyperspectral wetland image classification method based on the graph capsule neural network. DETAILED DESCRIPTION

[0052] The specific embodiments of the present application are described below to facilitate those skilled in the art to understand the present application, but it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all applications utilizing the concept of the present application are within the scope of protection.

[0053] As Figure 1 shown, the hyperspectral wetland image classification method based on the graph capsule neural network of the present application includes the following steps:

[0054] S1: learning feature transformation by generative adversarial domain adaptation framework, so that the source domain samples and target domain samples of the hyperspectral wetland image are matched in features;

[0055] Step S1 includes:

[0056] S11: selecting N wetland images of KWetland image samples were used as source domain samples; samples were selected from the target domain scene. H A labeled wetland image is used as the target domain sample;

[0057] S12: From the perspective of generative domain adaptation, and combining the target domain expectation error theory, a generative adversarial domain adaptation framework is constructed; the objective function of the generative adversarial domain adaptation framework is:

[0058]

[0059] in, and These are the generalization errors in the source domain and the target domain, respectively. Indicates differences between fields. H For the hypothesis space, λ To achieve the optimal joint generalization error, ; h Mapping samples from the source and target domains to the hypothesis space H Features S For the source domain sample space, T For the target domain sample space;

[0060] S13: Establish a domain classifier to mine the differences between source domain samples and target domain samples, enabling feature matching between source and target domain samples; the objective function of the domain classifier is:

[0061]

[0062] in, For indicator functions, h ( x The output of the domain classifier is the source domain. h '( x The output of the domain classifier is the target domain. E To calculate the expected value;

[0063] S14: Considering that the key factors of generative adversarial networks lie in how to construct the generator and train the discriminator, in order to preserve the discriminative information in the source domain, two different classifiers are constructed in the source domain. And train two different classifiers. This enables it to correctly classify source domain samples; the classifier The objective function is:

[0064]

[0065] in, X s For source domain samples, Y s The labels corresponding to the source domain samples. p (y | x s is a source domain sample X s is a class of y probability of the class; x s is a sub-sample in the source domain sample, SSF is a generator, (.) is a classifier objective function of the target domain, K is the number of samples selected from the source domain, k is one of the K samples selected from the source domain;

[0066] S2: Construct a graph capsule neural domain adaptive network structure, extract domain-invariant features and domain-related features, and discover transferable features and share across domains;

[0067] Step S2 includes:

[0068] S21: Establish a graph capsule neural domain adaptive network, map the source domain sample and the target domain sample to the same feature space to obtain an adjacency matrix and a node feature matrix , and construct a fully connected graph structure:

[0069]

[0070] wherein, , M is the number of nodes of the fully connected graph structure, M s is the number of source domain sample nodes, M t is the number of target domain sample nodes, is the node feature dimension extracted by the feature extraction network;( X 0 ) T is the transpose matrix of the node feature matrix;

[0071] S22: Input the fully connected graph structure into a deep hierarchical graph neural network to learn the hierarchical structure within the same class and between different classes of wetland image samples;

[0072] S23: Jointly learn the hierarchical relationship between the source domain and the target domain wetland image nodes, reduce the difference between the wetland image samples in different domains, extract domain-invariant features and domain-related features, and discover transferable features and share across domains. Step S23 includes:

[0073] Bottom-up inference phase:

[0074] S231: Construct a bottom-up reasoning module, sample the source domain sample nodes within the same category and between different categories, retain the target domain sample nodes, and sample the source domain sample nodes hierarchically; the objective function of the bottom-up reasoning module is:

[0075]

[0076]

[0077]

[0078] wherein, is the input feature matrix of the bottom-up reasoning module of the L layer, is the graph convolution process, and are the classification functions of the source domain sample nodes within the same category and between different categories respectively, is the number of subgraphs of the layer, is the node feature matrix of the L th subgraph of the i layer before downsampling, is a random node in the i th subgraph, is a neighborhood set, is the distance from to , is the node feature matrix of the L th subgraph of the i +1 layer after downsampling;

[0079] S232: Construct a new fully connected graph according to the sampled wetland image sample nodes, and obtain the adjacency matrix of the current bottom-up reasoning module;

[0080] S233: Repeat steps S231-S232, stack multiple bottom-up reasoning modules, and extract multi-level feature representations and deep graph relationships of wetland images in the source domain and the target domain;

[0081] Top-down reasoning phase:

[0082] S234: Construct a top-down reasoning module by combining the position information of the wetland image sample nodes in each bottom-up reasoning module, sample the new graph nodes, and layer by layer the same category within and between different categories, mine the hierarchical relationship of the nodes within the same category and between different categories of the source domain or target domain wetland images, and the objective function of the top-down reasoning module is:

[0083]

[0084] wherein, is the L input feature matrix of the top-down reasoning module of the layer, P is the node feature matrix output after upsampling, L is the L position information of the graph node in the bottom-up reasoning module of the

[0085] S235: By utilizing the correlation between the wetland image samples of the source domain and the target domain, the difference between the wetland image samples of different domains is reduced, the domain-invariant features and the domain-related features are extracted, the transferable features are discovered, and the cross-domain sharing is realized.

[0086] S3: Two classifiers coupled structure is designed, the two classifiers are trained by using the source domain samples, the classification difference of the target domain samples is maximized, and the precise classification of the hyperspectral wetland image is realized by identifying the classification boundary.

[0087] Step S3 includes:

[0088] S31: Fixing the two classifiers , the generator SSF is trained to extract the features of the source domain and the target domain, and the difference between the domains is minimized, the two classifiers classify the wetland image samples of the target domain, and the difference is minimized:

[0089]

[0090] wherein, and are the probabilities of the two classifiers , and corresponding to p 1 and p 2 output for the first k class respectively; X t is the target domain sample, n is the number of source domain categories, m is the training batch size, and are the data of the same batch respectively;

[0091] S32: Fixing the generator SSF, training the two classifiers to maximize the classification difference of the two classifiers on the wetland image samples of the target domain:

[0092] ;

[0093] S33: The optimal classifier and generator are obtained, and by using the adversarial training strategy, the fine classification of the hyperspectral wetland image under the target domain is realized.

[0094] The application takes the cross-domain data representation of the graph capsule neural network as the core, aims at the problems of complex environment, difficult on-site detection, rich vegetation types, distribution difference and lack of obvious boundary of the wetland ecological system, and from the perspective of generating adversarial domain self-adaptation, proposes a joint learning method of hyperspectral image space and spectral features, so as to fully mine the correlation between source domain and target domain data (same sensor, similar ground material, same scene, etc.), discover the transferable knowledge and realize cross-domain sharing, enhance the effective discrimination of class boundary, and finally realize the accurate classification of hyperspectral wetland images in unknown areas, complex scenes and conditions of data type missing, insufficient and imbalance.

Claims

1. A hyperspectral wetland image classification method based on graph capsule neural networks, characterized in that, Includes the following steps: S1: Generative adversarial domain adaptive framework learns feature transformation to enable feature matching between source domain samples and target domain samples of hyperspectral wetland images; S2: Construct a graph capsule neural domain adaptive network structure, extract domain-invariant and domain-related features, discover transferable features and share them across domains; S3: Design a coupled binary classifier, train the binary classifier using source domain samples, maximize the classification difference of target domain samples, and achieve accurate classification of hyperspectral wetland images by identifying classification boundaries; Step S2 includes: S21: Establish a graph capsule neural domain adaptive network to map source domain samples and target domain samples to the same feature space, obtaining the adjacency matrix. and node feature matrix And construct a fully connected graph structure: in, , M The number of nodes in a fully connected graph structure. M s The number of sample nodes in the source domain. M t The number of sample nodes in the target domain. The node feature dimensions extracted by the feature extraction network; X 0 ) T This is the transpose of the node feature matrix; S22: Input the fully connected graph structure into a deep hierarchical graph neural network to learn the hierarchical structure of wetland image samples within the same category and between different categories; S23: Jointly learn the hierarchical relationship between wetland image nodes in the source and target domains to reduce the differences between wetland image samples from different domains, extract domain-invariant and domain-related features, discover transferable features and share them across domains; Step S23 includes: Bottom-up reasoning stage: S231: Construct a bottom-up inference module by sampling source domain sample nodes within the same category and between different categories, retaining target domain sample nodes, and simultaneously sampling source domain sample nodes hierarchically; the objective function of the bottom-up inference module is: in, For the first L The input feature matrix of the bottom-up reasoning module. For the graph convolution process, and These are the classification functions for source domain sample nodes within the same category and between different categories, respectively. For the first Number of subgraphs Before downsampling L +1 floor i The node feature matrix of each subgraph For the first i Random nodes in a subgraph For neighborhood set, for arrive distance, The first after downsampling L +1 floor i The node feature matrix of each subgraph; S232: Construct a new fully connected graph based on the sample nodes of the sampled wetland image to obtain the adjacency matrix of the current bottom-up reasoning module; S233: Repeat steps S231-S232, stacking multiple bottom-up reasoning modules to extract multi-level feature representations and deep map relationships of wetland images in the source and target domains; Top-down reasoning stage: S234: Combining the location information of wetland image sample nodes in each bottom-up inference module, a top-down inference module is constructed. By sampling new graph nodes, the hierarchical graph structure within the same category and between different categories is explored layer by layer to mine the hierarchical relationships between nodes within the same category and between different categories of wetland image samples in the source or target domain. The objective function of the top-down inference module is: in, For the first L The input feature matrix of the +1 layer top-down inference module This is the node feature matrix output after upsampling. P L For the first L Position information of graph nodes in the bottom-up reasoning module; S235: Utilize the correlation between wetland image samples in the source and target domains to reduce the differences between wetland image samples in different domains, extract domain-invariant and domain-related features, discover transferable features, and share them across domains. Step S3 includes: S31: Fixed Two Classifier The generator SSF is trained to extract features from the source and target domains, ensuring that the differences between the domains are minimized and a binary classifier is used. Minimize the differences among wetland image samples in the classification target domain: in, and Two classifiers , and Corresponding to p 1 and p 2 pairs k The probability of the class output; X t For the target domain sample, n For the number of source domain categories, m For training batch size, and Data from the same batch; S32: Fix the generator SSF and train a binary classifier. This maximizes the classification difference between the two classifiers on wetland image samples in the target domain: ； S33: Obtain the optimal classifier and generator, and achieve fine classification of hyperspectral wetland images in the target domain through an adversarial training strategy.

2. The hyperspectral wetland image classification method based on graph capsule neural network according to claim 1, characterized in that, Step S1 includes: S11: Select from the source region hyperspectral scene N Images of wetlands in 10 categories, with each category containing 10 images. K A wetland image sample was selected from the target domain scene. H A sample of labeled wetland images; S12: From the perspective of adaptive generation domain, combined with the expected error theory of target domain, a generative adversarial adaptive framework is constructed. S13: Establish a domain classifier to explore the differences between source domain samples and target domain samples, and perform feature matching between source domain samples and target domain samples. S14: Construct two different classifiers in the source domain. Train two different classifiers This enables it to correctly classify source domain samples; the classifier The objective function is: in, X s For source domain samples, Y s The labels corresponding to the source domain samples. p ( y | x s () is the source domain sample X s The middle category is y The probability of a class; x s For a subsample in the source domain, SSF For generator, (.) represents the classifier. The objective function, K The number of samples selected for the source domain. k Selected for the source domain K One of the samples.

3. The hyperspectral wetland image classification method based on graph capsule neural network according to claim 2, characterized in that, The objective function of the generative adversarial adaptive framework is: in, and These are the generalization errors in the source domain and the target domain, respectively. Indicates differences between fields. H For the hypothesis space, λ To achieve the optimal joint generalization error, ; h Mapping samples from the source and target domains to the hypothesis space H Features S For the source domain sample space, T For the target domain sample space.

4. The hyperspectral wetland image classification method based on graph capsule neural network according to claim 2, characterized in that, The objective function of the domain classifier is: in, For indicator functions, h ( x The output of the domain classifier is the source domain. h '( x The output of the domain classifier is the target domain. E To obtain the expected result.

Images