A hyperspectral image tree species classification method and device based on generative adversarial network

By generating an adversarial network to process hyperspectral image data, extracting and optimizing features, the problems of large sample demand and low accuracy in the hyperspectral image classification method of drone are solved, and efficient and accurate tree species classification is achieved, which is suitable for diverse geographical environments.

CN114882353BActive Publication Date: 2024-11-12NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202210390641.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-14
Publication Date
2024-11-12
Estimated Expiration
2042-04-14

AI Technical Summary

Technical Problem

The existing drone hyperspectral image classification methods have problems such as large demand for sample data, low preprocessing efficiency and low accuracy of unsupervised classification spectral feature extraction, resulting in unsatisfactory tree classification accuracy and insufficient robustness and applicability.

Method used

Using a method based on a generative adversarial network, by obtaining hyperspectral image data, performing image segmentation and image reconstruction, extracting texture features and spectral features, combining attention mechanisms, generating and discriminator for iterative optimization, and generating image features with weights Classification of tree species.

Benefits of technology

It greatly reduces the dependence on the sample demand, improves the accuracy and applicability of tree species classification, and is suitable for different geographical areas and land cover types. The method is highly robust and the prediction results are accurate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114882353B_ABST
    Figure CN114882353B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and device for classifying tree species in hyperspectral images based on a generative adversarial network. The method includes: performing image segmentation on the hyperspectral image and then reconstructing the image; extracting the texture features and spectral features of the hyperspectral image; inputting the texture features and spectral features into a generative adversarial network based on an attention mechanism to obtain image features with attention, and sending them to a discriminator to obtain the output features of the discriminator after maximization and optimization; optimizing the generation distribution of the generator by minimizing the score value obtained by the input features, and sending the final output features of the discriminator to a classifier to obtain the classification results. The distribution and coverage of various tree species in the forest can be obtained, and monocular tree species type classification can be realized, which is convenient for monitoring and supervision. At the same time, the number of samples required for data preprocessing and manual collection is small, which is suitable for most geographical areas and land cover types. The method has high feasibility, robustness and prediction result accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a hyperspectral image tree species classification method and device based on a generative adversarial network. Background Art

[0002] Compared with conventional field surveys, remote sensing technology can monitor forest ecosystems quickly and efficiently. An important direction of forestry remote sensing is tree species classification and identification technology. Some scholars have integrated lidar and hyperspectral data to classify forest tree species and have made positive progress. However, the acquisition cost of lidar data is expensive and is affected by the area of ​​the flight area, so its application prospects are limited. Multispectral satellite remote sensing is affected by spatial resolution and spectral resolution, and it also has limitations for fine classification of ground objects. Hyperspectral data can discover more vegetation information. The current drone hyperspectral data is easy to obtain, low cost, short cycle, and high spatial resolution, which brings a new remote sensing data source for tree species classification.

[0003] The existing methods for classifying UAV hyperspectral images are mainly divided into two categories: supervised classification and unsupervised classification. Supervised classification uses sample data of known ground object categories to determine the category of non-sample data, such as the maximum likelihood method (MLC) and support vector machine (SVM); unsupervised classification only determines the category of data based on the spectral characteristics of remote sensing images, such as the K-means algorithm and ISODATA.

[0004] However, both of the above solutions have drawbacks: on the one hand, supervised classification requires a large amount of sample data and requires a lot of effort to pre-process the sample data, and the efficiency cannot achieve the expected effect; on the other hand, when the spectral features extracted by unsupervised classification are applied to monocular tree species classification, they often encounter problems such as too high crown density and approximate spectral features, resulting in a large error between the model prediction value and the true value, and the classification accuracy is not ideal. Therefore, both of the above methods have their own defects, and are not robust and applicable. Summary of the invention

[0005] Purpose of the invention: The present invention aims to provide a method and device for tree species classification based on hyperspectral images based on generative adversarial networks.

[0006] Technical solution: In an embodiment of the present invention, a method for tree species classification based on hyperspectral images using a generative adversarial network is provided, comprising:

[0007] A hyperspectral image tree species classification method based on a generative adversarial network, characterized by comprising:

[0008] Acquire hyperspectral image data;

[0009] Perform image segmentation on hyperspectral images;

[0010] The segmented hyperspectral image is input into the pre-trained Vgg network model for image reconstruction to obtain the optimized hyperspectral image;

[0011] The texture features of the optimized hyperspectral image are extracted through the gray-level co-occurrence matrix;

[0012] Independent principal component analysis is performed on the optimized hyperspectral image to obtain the spectral features of different bands, and the spectral features of the first m bands with more spectral features are selected as the extracted spectral features;

[0013] The extracted texture features and spectral features are input into the attention network model to obtain the output image features with attention;

[0014] The image features with attention are combined with the random noise z that follows the standard normal distribution as the input of the generator in the generative adversarial network, and the unknown class image features are generated by the generative network;

[0015] The features of the image with attention and the features of the unknown class image are used as the input of the discriminator to obtain the output features of the discriminator after maximization optimization;

[0016] The output features of the discriminator are returned to the input generator to optimize the generation distribution of the generator, and then the output features of the generator are input into the discriminator. This step is repeated N times for iterative optimization until the iteration stop condition is met, and the maximum optimized weighted image features output by the final discriminator are obtained;

[0017] The weighted image features are used as the input of the classifier to obtain the output hyperspectral image tree species classification results.

[0018] In some embodiments, the iteration stopping condition is that the discriminator and the generator reach Nash equilibrium.

[0019] In some embodiments, for normalizing the generative adversarial network, the loss function is calculated using the following formula:

[0020]

[0021] in is the loss function of the generated adversarial network, x~P data and z~P z represents the real input vector and the noise vector; E(·) is the expectation of the model probability distribution; D(x) and G(z) represent the sigmoid output obtained by training the discriminator D on the real sample input vector and the synthetic data generated by random noise by the generator G, respectively; D(G(z)) represents the output result obtained by feeding the data synthesized by G(z) into D.

[0022] In some embodiments, the number of iterations N is obtained by generating adversarial training of a network model, and the adversarial training process further includes: evaluating and judging the classification accuracy of the classification result and the measured data, and in response to the classification result not meeting the classification accuracy condition, returning to iterate N times until the classification result meets the classification accuracy condition;

[0023] The classification accuracy evaluation includes:

[0024]

[0025] Among them, ts and tr are the classification accuracy of known class image features and unknown class image features respectively, and H is the average classification accuracy of image features.

[0026] In some embodiments, the hyperspectral image is acquired by drone.

[0027] In some embodiments, performing image segmentation on a hyperspectral image includes:

[0028] By comparing the segmentation results of different parameters, the optimal parameter combination and the optimal segmentation result are finally selected;

[0029] By judging the change rate of the local change LV of the homogeneity of the segmented object under different segmentation scale parameters; when the change rate of the local change LV is the largest, the scale value corresponding to the point is the optimal segmentation scale of the image;

[0030] There are three parameters that can determine the heterogeneity criteria of the object: smoothness, compactness and color factor. Different experiments are conducted on the segmentation color and scale parameters.

[0031] In some embodiments, the segmented hyperspectral image is input into a pre-trained Vgg network model for image reconstruction, including:

[0032] The Vgg network model includes an average discriminator and a generator; the method for constructing the Vgg network model includes:

[0033] First, the generator is pre-trained using the Vgg network, and the structure of the true and false hyperspectral images of each band of the input is extracted and compared through 8 convolutional layers. Finally, the Leaky ReLU activation function is used to determine the probability of whether the input image is a true hyperspectral image.

[0034] Probability calculation introduces the Relativistic Average Discrimination (RaD), which is defined as:

[0035] D RaD (x r ,x f)=1-σ(C(x r )-E xf [C(x f )])

[0036] D RaD (x f ,x r )=σ(C(x f )-E xr [C(x r )])

[0037] Among them, D RaD (x r ,x f )、D RaD (x f ,x r ) are the probability that the discriminator predicts that the real image features are more realistic than the fake image features and the probability that the discriminator predicts that the fake image features are more realistic than the real image features, respectively. r is the real data, and x f is the "fake" data generated by the generator; σ() refers to the Sigmoid function, C(x r ) refers to the real data before the Leaky ReLU layer conversion, (x f ) refers to the real "fake" data before the Leaky ReLU layer conversion, ) are the average of all selected false data sets before passing through the Leaky ReLU layer and the average of all selected true data sets before passing through the Leaky ReLU layer;

[0038] Discriminator loss function for:

[0039]

[0040] The loss function corresponding to the generator for:

[0041]

[0042] Taking into account the loss of hyperspectral images in spatial and spectral domains, the mean square error function MSE (x f ,x r ),

[0043]

[0044] Among them, m×n is the length and width of the hyperspectral images in the same batch; in addition, the pre-training loss is taken into account, corresponding to the pre-training loss function L vgg for:

[0045]

[0046] where γ i is the coefficient corresponding to the i-th layer in the Vgg-19 network, which is used to adjust the weight of the structural information extracted at different levels; and Vgg19 i (x f -x r ) represents the difference between the data generated by the generator and the real data in the Vgg-19 network;

[0047] Generator overall loss function L G for

[0048]

[0049] in is the loss function of the generator, L vgg is the pre-training loss function, MSE(x f ,x r ) is the mean square error function;

[0050] Corresponding to the overall loss function of the discriminator L D for:

[0051]

[0052] Among them, μ is the coefficient of the degree to which the discriminator is affected by the generator, is the discriminator loss function.

[0053] In some embodiments, the texture features include at least eight texture features: mean, variance, homogeneity, contrast, difference, entropy, second-order moment, and correlation.

[0054] In some embodiments, m=5.

[0055] In some embodiments, the extracted texture features and spectral features are input into an attention network model to obtain output image features with attention, including:

[0056] Input texture features and spectral features into the attention network, and obtain three attention features through three 1*1 convolutional layers with different weights. The first attention feature is transposed and multiplied with the second attention feature, and the result is sent to the Softmax function to obtain the attention map; the obtained attention map is transposed and matrix multiplied with the third attention feature and passed through a 1*1 convolutional layer to obtain the final image features with attention.

[0057] Furthermore, in some embodiments, the attention network model β adopts the following formula:

[0058]

[0059] Where N is the number of image features, and the attention scoring function s i The calculation formula is:

[0060] s i =(W f x) T *(W g x)

[0061] Among them, x is the image attention feature extracted by the convolutional network, W f and W g are two weight matrices, implemented through 1*1 convolution; T is the transposition of the matrix;

[0062] Image features x with attention 0 The calculation formula is:

[0063]

[0064] where h(x i )=W h x i ; W h and W v Are two weight matrices, implemented through 1*1 convolution; x i is the input image feature, and β is the attention network model function.

[0065] In some embodiments, the image features with attention are combined with the set random noise z that obeys the standard normal distribution as the input of the generator in the generative adversarial network, and the unknown class image features are generated by the generative network, including:

[0066] Using random noise z and condition c as the input of the generating network, the optimization objective formula is:

[0067]

[0068] Among them, E(·) is the expectation of the model probability distribution; x′ is the known class feature, c(y) is the semantic information, is the generated unknown class image feature, where The calculation formula is:

[0069]

[0070] Among them, z is random noise and obeys the standard normal distribution, that is, z~N(0,1).

[0071] In a second aspect, the present invention provides a hyperspectral image tree species classification device based on a generative adversarial network, including a processor and a storage medium;

[0072] The storage medium is used to store instructions;

[0073] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.

[0074] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.

[0075] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: it greatly reduces the reliance on samples, uses a generative adversarial network combined with an attention mechanism to optimize all features, can observe the distribution and coverage of various tree species in the forest, and achieve monocular tree species classification, which is convenient for monitoring and supervision. At the same time, the required data preprocessing and the number of samples collected manually are relatively small, and the experience and professional background of the implementers are not high. It is suitable for most geographical areas and land cover types, and the method is theoretically feasible, robust, and the accuracy of the prediction results is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 and Figure 2 4 is a flowchart of a method for tree species classification in hyperspectral images based on a generative adversarial network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0078] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0079] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0080] Example 1

[0081] like Figure 1 , Figure 2 As shown in the figure, a hyperspectral image tree species classification method based on generative adversarial network includes:

[0082] Step S1, performing image segmentation on the hyperspectral image;

[0083] Step S2, inputting the segmented hyperspectral image into the pre-trained Vgg network model for image reconstruction to obtain an optimized hyperspectral image;

[0084] Step S3, extracting texture features of the optimized hyperspectral image through gray-level co-occurrence matrix;

[0085] Step S4, performing independent principal component analysis on the optimized hyperspectral image to obtain spectral features of different bands, and selecting spectral features of the first m (m=5) bands with more spectral features as the extracted spectral features;

[0086] Step S5, inputting the extracted texture features and spectral features into the attention network model to obtain output image features with attention;

[0087] Step S6: The image features with attention are combined with the set random noise z that obeys the standard normal distribution as the input of the generator in the generative adversarial network, and the unknown class image features are generated by the generative network;

[0088] Step S7, using the image features with attention and the image features of unknown class as the input of the discriminator to obtain the output features of the discriminator after maximization optimization;

[0089] Step S8: Return the output features of the discriminator to the input generator to optimize the generation distribution of the generator, then input the output features of the generator into the discriminator, repeat this step N times for iterative optimization until the iteration stop condition is met, and obtain the maximum optimized weighted image features output by the final discriminator;

[0090] Step S9: using the weighted image features as the input of the classifier to obtain the output hyperspectral image tree species classification results.

[0091] In some embodiments, step S1, performing image segmentation on the hyperspectral image, includes:

[0092] By comparing the segmentation results of different parameters, the optimal parameter combination and the optimal segmentation result are finally selected;

[0093] By judging the change rate of the local change LV of the homogeneity of the segmented object under different segmentation scale parameters; when the change rate of the local change LV is the largest, the scale value corresponding to the point is the optimal segmentation scale of the image;

[0094] There are three parameters that can determine the heterogeneity criteria of the object: smoothness, compactness and color factor. Different experiments are conducted on the segmentation color and scale parameters.

[0095] In specific implementation, hyperspectral images can be acquired through drones;

[0096] Image segmentation uses the multi-scale image segmentation in eCognition software; multi-scale segmentation means segmenting images at different segmentation scales to express surface information of objects at different scales.

[0097] In some embodiments, step S2, inputting the segmented hyperspectral image into a pre-trained Vgg network model for image reconstruction, includes:

[0098] The Vgg network model includes an average discriminator and a generator; the method for constructing the Vgg network model includes:

[0099] First, the generator is pre-trained using the Vgg network, and then the structure of the true and false hyperspectral images of each band of the input is extracted and compared through 8 convolutional layers. Finally, the Leaky ReLU activation function is used to determine the probability of whether the input image is a true hyperspectral image.

[0100] In the embodiment of the present invention, probability calculation introduces a Relativistic Average Discrimination (RaD), which is defined as:

[0101] D RaD (x r ,x f )=1-σ(C(x r )-E xf [C(x f )])

[0102] D RaD (x f ,x r )=σ(C(x f )-E xr [C(x r )])

[0103] Where D RaD (x r ,x f )、D RaD (x f ,xr ) are the probability that the discriminator predicts that the real image features are more realistic than the fake image features and the probability that the discriminator predicts that the fake image features are more realistic than the real image features, respectively. r is the real data, and x f is the "fake" data generated by the generator; σ() refers to the Sigmoid function, C(x r ) refers to the real data before the Leaky ReLU layer conversion, (x f ) refers to the real "fake" data before the Leaky ReLU layer conversion, They are the average of all selected false data sets before passing through the Leaky ReLU layer and the average of all selected true data sets before passing through the Leaky ReLU layer;

[0104] The discriminator loss function is obtained as:

[0105]

[0106] The corresponding generator loss function is:

[0107]

[0108] Taking into account the loss of the spatial and spectral domains of the hyperspectral image, the mean square error function is introduced, namely

[0109]

[0110] Among them, m×n is the length and width of the images in the same batch; in addition, taking the pre-training loss into account, the corresponding pre-training loss function is:

[0111]

[0112] where γ i is the coefficient corresponding to the i-th layer in the Vgg-19 network, which is used to adjust the weight of the structural information extracted at different levels; and Vgg19 i (x f -x r ) represents the difference between the data generated by the generator and the real data in the Vgg-19 network;

[0113] On this basis, the overall loss function of the generator L G for

[0114]

[0115] in is the loss function of the generator, L vgg is the pre-training loss function, MSE(x f ,xr ) is the mean square error function;

[0116] Corresponding to the overall loss function of the discriminator L D for:

[0117]

[0118] Among them, μ is the coefficient of the degree to which the discriminator is affected by the generator, is the discriminator loss function.

[0119] In the specific implementation, in order to facilitate the display of hyperspectral image details and improve the problem of inconsistency of existing hyperspectral image raw data, the hyperspectral images of the training set and the validation set will be normalized in the form of a whole block of data. In addition, the normalized hyperspectral image should be divided into grayscale images of the corresponding number of bands before consideration. The number of bands and image size input simultaneously during specific training should be limited to a certain extent and should not exceed the image size of the training set and the validation set.

[0120] In the specific implementation, only the first three convolutional layers of Vgg-19 are needed for pre-training. Deeper pre-training is not very meaningful on the one hand, and on the other hand, it may cause the generator to over-train and start fooling the discriminator, resulting in network training failure.

[0121] In some embodiments, step S3, extracting texture features from the optimized hyperspectral image by using a gray level co-occurrence matrix;

[0122] In the specific implementation, since the texture of vegetation in the image generally has no obvious directionality, the moving direction is taken as the average value of four directions of 0°, 45°, 90°, and 135°, and a gradient window size of 3×3, 5×5, ..., 31×31 is used with a step size of 1 to extract texture features.

[0123] In the embodiment of the present invention, several bands with high band clarity, less interference information, more obvious ground object information, and the ability to fully display the differences in various ground object image features are selected for texture analysis, and eight texture features including mean, variance, homogeneity, contrast, difference, entropy, second-order moment, and correlation are extracted. The appropriate texture window size is selected to maximize the overall classification accuracy of tree species.

[0124] In some embodiments, step S4, performing independent principal component analysis on the optimized hyperspectral image to obtain spectral features of different bands, and selecting spectral features of the first m (m=5) bands with more spectral features as the extracted spectral features.

[0125] Step S5, inputting the texture features and spectral features extracted in step S3 and step S4 into the attention network model to obtain output image features with attention; including:

[0126] Input texture features and spectral features into the attention network, and obtain three attention features through three 1*1 convolutional layers with different weights. The first attention feature is transposed and multiplied with the second attention feature, and the result is sent to the Softmax function to obtain the attention map; the obtained attention map is transposed and matrix multiplied with the third attention feature and passed through a 1*1 convolutional layer to obtain the final image features with attention.

[0127] In some embodiments, the attention network model β adopts the following formula:

[0128]

[0129] Where N is the number of image features, and the attention scoring function s i The calculation formula is:

[0130] s i =(W f x) T *(W g x)

[0131] Among them, x is the image attention feature extracted by the convolutional network, W f and W g are two weight matrices, implemented through 1*1 convolution; T is the transposition of the matrix;

[0132] Image features x with attention 0 The calculation formula is:

[0133]

[0134] where h(x i )=W h x i ; W h and W v Are two weight matrices, implemented through 1*1 convolution; x i is the input image feature, and β is the attention network model function.

[0135] Step S6, combining the image features with attention with the set random noise z that obeys the standard normal distribution as the input of the generator in the generative adversarial network, and generating unknown class image features through the generative network, including:

[0136] In the embodiment of the present invention, random noise z condition c is used as the input of the generated network, and the optimization target formula is:

[0137]

[0138] Among them, E(·) is the expectation of the model probability distribution; x′ is the known class feature, c(y) is the semantic information, is the generated unknown class image feature, where The calculation formula is:

[0139]

[0140] Among them, z is random noise and obeys the standard normal distribution, that is, z~N(0,1).

[0141] Secondly, a convolutional neural network is used to extract the features of the real image, and the features and the generated features are sent to the discriminant network to perform authenticity judgment based on semantic information.

[0142] Step S8: Return the output features of the discriminator to the input generator to optimize the generation distribution of the generator, then input the output features of the generator into the discriminator, repeat this step N times for iterative optimization until the iteration stop condition is met, and obtain the maximum optimized weighted image features output by the final discriminator;

[0143] In some embodiments, the iteration stopping condition is that the discriminator and the generator reach Nash equilibrium.

[0144] For normalizing the generative adversarial network, the loss function is calculated using the following formula:

[0145]

[0146] in is the loss function of the generated adversarial network, x~P data and z~P z represents the real input vector and the noise vector; E(·) is the expectation of the model probability distribution; D(x) and G(z) represent the sigmoid output obtained by training the discriminator D on the real sample input vector and the synthetic data generated by random noise by the generator G, respectively; D(G(z)) represents the output result obtained by feeding the data synthesized by G(z) into D.

[0147] In some embodiments, the number of iterations N is obtained by generating adversarial training of an adversarial network model. The adversarial training process also includes: evaluating the classification accuracy of the classification results and the measured data. In response to the classification result not meeting the classification accuracy condition, returning to iterate N times until the classification result meets the classification accuracy condition.

[0148] In the specific implementation, first, the real features of a visible class are divided into k clusters (k = 3). Then, this paper calculates an intermediate sample for each cluster. Assuming that class a has k clusters, {Xc1, Xc2, ..., Xck}, the intermediate sample is Sc = {sc1, sc2, ..., sck}, the intermediate sample definition of the visible class image feature and the intermediate sample definition of the generated unknown class image feature are as follows:

[0149]

[0150]

[0151] in, is a generated unknown class image feature, and

[0152] In the specific implementation, each generated sample in category a should be as close as possible to at least one intermediate sample sc. Therefore, for the normalization of the generated network, the calculation formula is as follows:

[0153]

[0154] Among them, n1 is the number of samples generated by the generative network, and k is the number of intermediate samples of each class. Since the intermediate samples can be regarded as the average point of a cluster, the generated intermediate samples should be at least close to one real intermediate sample in the same category, expressed as follows:

[0155]

[0156] Where A is the total number of categories of category a. By using L R1 and L R2 These two regularizations can make the generated features more diverse and closer to the real image features. R1 is the regularization that constrains a single sample, and L R2 It is the regularization that constrains the entire intermediate sample cluster.

[0157] In the specific implementation, the attention model (attention mechanism) enhances the quality of unknown image features generated by the network by making the generation process pay more attention to key content and ignoring unnecessary interference information. Compared with the existing technology, it further improves the capabilities of the generative adversarial network generator and judge.

[0158] In step S8, the image features with attention are input into the discriminator to obtain the output features of the discriminator after maximization optimization. Next, the generation distribution of the generator is optimized by minimizing the score value obtained according to the input features. Through continuous adversarial training of the discriminator and the generator, after reaching Nash equilibrium, the final output features are sent to the classifier for classification.

[0159] Step S9: using the weighted image features as the input of the classifier to obtain the output hyperspectral image tree species classification results.

[0160] In the specific implementation, the classifier is used to obtain the classification result of the hyperspectral image, and the effects of ts, tr and H are reported. The accuracy of the classification result can be determined by using the output effects of ts, tr and H as a standard.

[0161] In the embodiment of the present invention, the evaluation and determination of classification accuracy is calculated using the following formula:

[0162]

[0163] Among them, ts and tr are the classification accuracy of known class image features and unknown class image features respectively, and H is the average classification accuracy of image features;

[0164] Among them, ts and tr are calculated using the following formulas:

[0165]

[0166]

[0167] Among them, TP is the number of samples whose actual value is 0 and whose predicted value is also 0; FP is the number of samples whose actual value is 1 and whose predicted value is 0; FN is the number of samples that are judged as negative samples but are actually positive samples.

[0168] Example 2

[0169] In a second aspect, the present embodiment provides a hyperspectral image tree species classification device based on a generative adversarial network, including a processor and a storage medium;

[0170] The storage medium is used to store instructions;

[0171] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0172] Example 3

[0173] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0174] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0175] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A hyperspectral image tree species classification method based on generative adversarial network, characterized in that: include: Acquire hyperspectral image data; Perform image segmentation on hyperspectral images; The segmented hyperspectral image is input into the pre-trained Vgg network model for image reconstruction to obtain the optimized hyperspectral image; The texture features of the optimized hyperspectral image are extracted through the gray-level co-occurrence matrix; Independent principal component analysis is performed on the optimized hyperspectral image to obtain the spectral features of different bands, and the spectral features of the first m bands with more spectral features are selected as the extracted spectral features; The extracted texture features and spectral features are input into the attention network model to obtain the output image features with attention; The image features with attention are combined with the random noise z that follows the standard normal distribution as the input of the generator in the generative adversarial network, and the unknown class image features are generated by the generative network; The features of the image with attention and the features of the unknown class image are used as the input of the discriminator to obtain the output features of the discriminator after maximization optimization; The output features of the discriminator are returned to the input generator to optimize the generation distribution of the generator, and then the output features of the generator are input into the discriminator. This step is repeated N times for iterative optimization until the iteration stop condition is met, and the maximum optimized weighted image features output by the final discriminator are obtained; The weighted image features are used as the input of the classifier to obtain the output hyperspectral image tree species classification results.

2. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 1, characterized in that: The iteration stopping condition is that the discriminator and the generator reach Nash equilibrium.

3. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 1, characterized in that: For normalizing the generative adversarial network, the loss function is calculated using the following formula: in is the loss function of the generated adversarial network, x~P data and z~P z represents the real input vector and the noise vector; E(·) is the expectation of the model probability distribution; D(x) and G(z) represent the sigmoid output obtained by training the discriminator D on the real sample input vector and the synthetic data generated by random noise by the generator G, respectively; D(G(z)) represents the output result obtained by feeding the data synthesized by G(z) into D.

4. The hyperspectral image tree species classification method based on generative adversarial network according to claim 1, 2 or 3, characterized in that: The number of iterations N is obtained by generating adversarial training of the adversarial network model, and the adversarial training process also includes: evaluating and judging the classification accuracy of the classification result and the measured data, and in response to the classification result not meeting the classification accuracy condition, returning to iterate N times until the classification result meets the classification accuracy condition; The classification accuracy evaluation includes: Among them, ts and tr are the classification accuracy of known class image features and unknown class image features respectively, and H is the average classification accuracy of image features.

5. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 1, characterized in that: The segmented hyperspectral image is input into the pre-trained Vgg network model for image reconstruction, including: The Vgg network model includes an average discriminator and a generator; the method for constructing the Vgg network model includes: First, the generator is pre-trained using the Vgg network, and the structure of the true and false hyperspectral images of each band of the input is extracted and compared through 8 convolutional layers. Finally, the Leaky ReLU activation function is used to determine the probability of whether the input image is a true hyperspectral image. Probability calculation introduces the Relativistic Average Discrimination (RaD), which is defined as: Among them, D RaD (x r ,x f )、D RaD (x f ,x r ) are the probability that the discriminator predicts that the real image features are more realistic than the fake image features and the probability that the discriminator predicts that the fake image features are more realistic than the real image features, respectively. r is the real data, and x f is the "fake" data generated by the generator; σ() refers to the Sigmoid function, C(x r ) refers to the real data before the Leaky ReLU layer conversion, (x f ) refers to the real "fake" data before the Leaky ReLU layer conversion, They are the average of all selected false data sets before passing through the LeakyReLU layer and the average of all selected true data sets before passing through the Leaky ReLU layer; Discriminator loss function for: The loss function corresponding to the generator for: Taking into account the loss of hyperspectral images in spatial and spectral domains, the mean square error function MSE (x f ,x r ), Among them, m×n is the length and width of the hyperspectral images in the same batch; in addition, the pre-training loss is taken into account, corresponding to the pre-training loss function L vgg for: where γ i is the coefficient corresponding to the i-th layer in the Vgg-19 network, which is used to adjust the weight of the structural information extracted at different levels; and Vgg19 i (x f -x r ) represents the difference between the data generated by the generator and the real data in the Vgg-19 network; Generator overall loss function L G for in is the loss function of the generator, L vgg is the pre-training loss function, MSE(x f ,x r ) is the mean square error function; Corresponding to the overall loss function of the discriminator L D for: Among them, μ is the coefficient of the degree to which the discriminator is affected by the generator, is the discriminator loss function.

6. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 1, characterized in that: The extracted texture features and spectral features are input into the attention network model to obtain the output image features with attention, including; Input texture features and spectral features into the attention network, and obtain three attention features through three 1*1 convolutional layers with different weights. The first attention feature is transposed and multiplied with the second attention feature, and the result is sent to the Softmax function to obtain the attention map; the obtained attention map is transposed and matrix multiplied with the third attention feature and passed through a 1*1 convolutional layer to obtain the final image features with attention.

7. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 6, characterized in that: The attention network model β adopts the following formula: Where N is the number of image features, and the attention scoring function s i The calculation formula is: s i =(W f x) T *(W g x) Among them, x is the image attention feature extracted by the convolutional network, W f and W g are two weight matrices, implemented through 1*1 convolution; T is the transposition of the matrix; Image features x with attention 0 The calculation formula is: Where h(x i )=W h x i ; W h and W v Are two weight matrices, implemented through 1*1 convolution; x i is the input image feature, and β is the attention network model function.

8. The method for tree species classification based on hyperspectral images using generative adversarial networks according to claim 1, characterized in that: The image features with attention are combined with the random noise z that follows the standard normal distribution as the input of the generator in the generative adversarial network. The unknown class image features are generated by the generative network, including: Using random noise z and condition c as the input of the generating network, the optimization objective formula is: Among them, E(·) is the expectation of the model probability distribution; x′ is the known class feature, c(y) is the semantic information, is the generated unknown class image feature, where The calculation formula is: Among them, z is random noise and obeys the standard normal distribution, that is, z~N(0,1).

9. A hyperspectral image tree species classification device based on generative adversarial network, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 8.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Cited By

  • Forest scene tree species classification method based on improved Deeplabv3 +

    CN116452872A

  • A forest scene tree species classification method based on an improved deeplabv3+

    CN116452872B