Hyperspectral image classification method and device for uncertainty estimation

Abstract

This application proposes a hyperspectral image classification method for uncertainty estimation, which relates to the technical field of hyperspectral image classification. The method includes: obtaining a hyperspectral image; constructing an adaptive dimensionality reduction layer, inputting the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtaining a reduced-dimensional hyperspectral image; constructing a Gabor convolution layer using a Gabor filter, characterizing the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and inputting the reduced-dimensional hyperspectral image into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; constructing a Bayesian fully connected layer, causing the Bayesian fully connected layer to learn the mask distribution of a standard Dropout layer, inputting the spectral-spatial joint features into the Bayesian fully connected layer, and performing classification through the Bayesian fully connected layer to obtain a classification result for the hyperspectral image. This application improves the classification accuracy of the hyperspectral image classification method and provides uncertainty estimation of the hyperspectral classification results, which can reflect the degree of confidence in the classification results.

Description

Technical Field

[0001] The present application relates to the technical field of hyperspectral image classification, and in particular to a hyperspectral image classification method and device for uncertainty estimation. Background Art

[0002] Remote sensing imaging technology is developing rapidly, with hyperspectral imaging demonstrating outstanding performance. Using an imaging spectrometer, hyperspectral imaging records the intensity of electromagnetic radiation from a target area at a specific ratio. Its electromagnetic spectrum spans the ultraviolet, visible, near-infrared, mid-infrared, and even thermal infrared regions, recording dozens or even hundreds of continuous and detailed (typically less than 10 nanometers) high-resolution spectral bands of the target. Hyperspectral imaging integrates both spatial and spectral information of the target area, resulting in a comprehensive remote sensing technology characterized by "map-spectrum integration."

[0003] The emergence of hyperspectral remote sensing technology has advanced conventional remote sensing applications from simple feature identification to a new stage: quantification and analysis based on spectral information. Hyperspectral imaging has garnered significant interest in the remote sensing community, leveraging its advantages to be widely applied in areas such as crop monitoring, mineral identification and mapping, grassland monitoring, oil and gas exploration, and ocean monitoring. In the military, this technology can also be applied to target detection, camouflage and counter-camouflage, and strike effectiveness assessment.

[0004] Classification is a key component of hyperspectral remote sensing data processing and a crucial step in extracting information from data. Hyperspectral image classification has garnered widespread attention in recent years and has become a hot research topic in the field. While deep learning-based methods are widely used in hyperspectral image classification due to their powerful feature extraction capabilities, these methods still have limitations and limited applicability. Summary of the Invention

[0005] The present application aims to solve one of the technical problems in the related art at least to a certain extent.

[0006] To this end, the first purpose of this application is to propose a hyperspectral image classification method for uncertainty estimation. By characterizing the distribution of parameters through a Bayesian neural network, the problem that the existing hyperspectral image classification method based on deep learning has low classification accuracy and cannot provide the credibility of the classification results is solved.

[0007] The second objective of this application is to propose a hyperspectral image classification device for uncertainty estimation.

[0008] The third object of this application is to provide a computer device.

[0009] A fourth object of the present application is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a hyperspectral image classification method for uncertainty estimation, including: acquiring a hyperspectral image; constructing an adaptive dimensionality reduction layer, inputting the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtaining a hyperspectral image after dimensionality reduction; constructing a Gabor convolution layer using a Gabor filter, using Bayesian deep learning to characterize the parameter distribution of the Gabor convolution layer, and inputting the hyperspectral image after dimensionality reduction into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; constructing a Bayesian fully connected layer, making the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, inputting the spectral-spatial joint features into the Bayesian fully connected layer, and performing classification through the Bayesian fully connected layer to obtain the classification result of the hyperspectral image.

[0011] Optionally, in one embodiment of the present application, the adaptive dimensionality reduction layer is expressed as:

[0012] y=xt

[0013] Where y represents the spectral features extracted by the adaptive dimensionality reduction layer, x represents the hyperspectral image, and t represents the parameters of the adaptive dimensionality reduction layer.

[0014] Optionally, in one embodiment of the present application, a Gabor convolution layer is constructed using a Gabor filter, including:

[0015] Encode the direction information of the Gabor filter into the convolution layer to generate a Gabor convolution filter, and use the Gabor convolution filter as the Gabor convolution layer;

[0016] Among them, the Gabor filter is expressed as:

[0017]

[0018]

[0019]

[0020] Where g represents the Gabor filter, λ represents the wavelength of the sinusoidal component, α and φ represent the normal direction and phase offset respectively, δ represents the standard deviation of the Gaussian function, and γ represents the spatial aspect ratio. and They represent the horizontal and vertical coordinates after Gabor filter transformation, and i represents a complex unit.

[0021] Optionally, in one embodiment of the present application, Bayesian deep learning is used to characterize the parameter distribution of the Gabor convolution layer, including:

[0022] The parameter distribution of the Gabor filter satisfies the Gaussian distribution, and the Gaussian distribution is used as the prior distribution of the parameters in the Gabor convolution layer;

[0023] The distribution of Gabor parameters of the optimization model is optimized through Bayesian inference, and the approximate posterior distribution of the Gabor parameter distribution is obtained as the posterior distribution of the parameters in the Gabor convolution layer;

[0024] Fit the prior distribution of the parameters in the Gabor convolution layer to the posterior distribution of the parameters in the Gabor convolution layer;

[0025] The Gabor convolution layer is expressed as:

[0026]

[0027]

[0028] Among them, F represents the feature map output by the convolution operation, F n represents the nth feature map, Represents the convolution operation, C′ represents the filter modulated by the Gabor filter, and C represents the learnable filter in the convolution layer. It is a dot product operation, and g represents a Gabor filter whose prior distribution satisfies the Gaussian distribution.

[0029] Optionally, in one embodiment of the present application, constructing a Bayesian fully connected layer includes:

[0030] Construct a standard Dropout layer and adaptively learn the mask distribution of the Dropout layer. The Dropout layer that satisfies the adaptively learned mask distribution is used as a Bayesian fully connected layer.

[0031] Among them, the mask distribution of the Dropout layer is adaptively learned, including:

[0032] By defining the hidden variable to satisfy a sub-distribution and introducing a mapping function, the distribution of the mask is characterized, where the sub-distribution r satisfies differentiability and is easy to sample, the mapping function satisfies monotony and differentiability, and the output value range is [0, 1];

[0033] The distribution of the mask is expressed as:

[0034]

[0035]

[0036] Among them, m j represents the jth mask, k(·) represents the mapping function, r jrepresents the variable whose j-th mask satisfies sub-distribution r, μ represents the mean of the multivariate Gaussian distribution, and σ represents the variance of the multivariate Gaussian distribution.

[0037] Optionally, in one embodiment of the present application, enabling the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer includes:

[0038] The mask distribution of the Bayesian fully connected layer is adaptively learned to make the distribution of parameters in the classification stage satisfy the variational Bayesian inference of the Gaussian process.

[0039] Optionally, in one embodiment of the present application, after characterizing the parameter distribution of the Gabor convolution layer using Bayesian deep learning, the model uncertainty of the feature extraction stage is estimated;

[0040] After making the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, the model uncertainty of the classification stage is estimated.

[0041] To achieve the above-mentioned purpose, a second embodiment of the present invention provides a hyperspectral image classification device for uncertainty estimation, comprising:

[0042] An acquisition module, used to acquire hyperspectral images;

[0043] The spectral feature extraction module is used to construct an adaptive dimensionality reduction layer, input the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtain a hyperspectral image after dimensionality reduction;

[0044] The spatial feature extraction module is used to construct a Gabor convolution layer using Gabor filters, characterize the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and input the dimensionally reduced hyperspectral image into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image;

[0045] The classification module is used to construct a Bayesian fully connected layer, enable the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer, input the spectral-spatial joint features into the Bayesian fully connected layer, and perform classification through the Bayesian fully connected layer to obtain the classification results of the hyperspectral image.

[0046] To achieve the above objectives, a third embodiment of the present invention proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hyperspectral image classification method for uncertainty estimation described in the above embodiment is implemented.

[0047] To achieve the above objectives, a fourth aspect of the present invention provides a non-transitory computer-readable storage medium. When instructions in the storage medium are executed by a processor, a hyperspectral image classification method for uncertainty estimation can be performed.

[0048] The hyperspectral image classification method, apparatus, computer equipment, and non-temporary computer storage medium for uncertainty estimation of the embodiments of the present application use a Bayesian neural network to characterize the distribution of parameters, thereby solving the problem that the existing hyperspectral image classification methods based on deep learning have low classification accuracy and cannot provide the credibility of the classification results. At the same time, the Gabor convolution layer is used to increase the expression ability of spatial-spectral structure information, improve the classification accuracy, and provide uncertainty estimation of the hyperspectral classification results, thereby increasing the credibility of the classification results.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0051] Figure 1 A schematic diagram of a flow chart of a hyperspectral image classification method for uncertainty estimation provided in Example 1 of the present application;

[0052] Figure 2 This is another flowchart of the hyperspectral image classification method for uncertainty estimation according to an embodiment of the present application;

[0053] Figure 3 This is a schematic diagram of the structure of a hyperspectral image classification device for uncertainty estimation provided in Example 2 of the present application. DETAILED DESCRIPTION

[0054] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0055] At present, although deep learning-based methods are widely used in hyperspectral image classification due to their powerful feature extraction capabilities, such methods still have limitations and are not applicable. This is because the trained model must not only improve the classification accuracy but also the certainty of the classification results.

[0056] The certainty of a classification result refers to the degree to which the model's predicted probability aligns with the true empirical probability. It assigns a specific probability value (between 0 and 1) to each test sample, resulting in a more refined prediction. The size of the certainty value reflects the degree of confidence a classification algorithm has in its classification results for the test sample, and plays an important role in practical applications.

[0057] Currently, deep learning-based hyperspectral image classification methods primarily aim to improve classification accuracy. By designing the neural network structure and finding the network parameters, a hyperspectral image classification model with strong generalization capabilities is obtained. However, due to non-optimal parameters in the training model, the model's determinism is low, which is called model uncertainty.

[0058] To address this issue, this application establishes a highly accurate and deterministic hyperspectral image classification method by characterizing the distribution of parameters through a Bayesian neural network. Furthermore, after using Bayesian deep learning to characterize the parameter distribution of the Gabor convolutional layer, this application can estimate the model uncertainty in the feature extraction stage by learning the probability distribution of the Gabor convolutional layer parameters. Furthermore, after having the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, the model uncertainty in the classification stage is estimated, reflecting the credibility of the classification results.

[0059] The following describes a hyperspectral image classification method and apparatus for uncertainty estimation according to an embodiment of the present application with reference to the accompanying drawings.

[0060] Figure 1 A flowchart of a hyperspectral image classification method for uncertainty estimation provided in Example 1 of the present application.

[0061] like Figure 1 As shown in FIG, the hyperspectral image classification method for uncertainty estimation includes the following steps:

[0062] Step 101, acquiring a hyperspectral image;

[0063] Step 102: construct an adaptive dimensionality reduction layer, input the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtain a hyperspectral image after dimensionality reduction;

[0064] Step 103: constructing a Gabor convolution layer using a Gabor filter, characterizing the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and inputting the hyperspectral image after dimensionality reduction into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image;

[0065] Step 104: construct a Bayesian fully connected layer, make the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, input the spectral-spatial joint features into the Bayesian fully connected layer, perform classification through the Bayesian fully connected layer, and obtain the classification result of the hyperspectral image.

[0066] The hyperspectral image classification method for uncertainty estimation of the embodiment of the present application is as follows: obtaining a hyperspectral image; constructing an adaptive dimensionality reduction layer, inputting the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtaining a hyperspectral image after dimensionality reduction; constructing a Gabor convolution layer using a Gabor filter, using Bayesian deep learning to characterize the parameter distribution of the Gabor convolution layer, inputting the hyperspectral image after dimensionality reduction into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; constructing a Bayesian fully connected layer, making the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, inputting the spectral-spatial joint features into the Bayesian fully connected layer, and performing classification through the Bayesian fully connected layer to obtain the classification result of the hyperspectral image. Thus, by characterizing the distribution of parameters through the Bayesian neural network, the problem of low classification accuracy and inability to provide the credibility of the classification results of the existing hyperspectral image classification method based on deep learning is solved. At the same time, the Gabor convolution layer is used to increase the expression ability of spatial-spectral structure information, improve the classification accuracy, and provide uncertainty estimation of the hyperspectral classification results, thereby increasing the credibility of the classification results.

[0067] Hyperspectral images can be obtained by taking pictures with an airborne infrared imaging spectrometer. In this application, hyperspectral images are obtained by using the public dataset of IEEE GRRASS.

[0068] Since hyperspectral images contain a large number of bands, which will affect the efficiency of subsequent feature extraction, this application proposes an adaptive dimensionality reduction layer, which reduces the dimensionality of hyperspectral images through the adaptive dimensionality reduction layer to obtain a hyperspectral image after dimensionality reduction. In this application, the parameters of the adaptive dimensionality reduction layer are trained through network automatic learning.

[0069] Furthermore, in the embodiment of the present application, the adaptive dimensionality reduction layer is expressed as:

[0070] y=xt

[0071] Where y represents the spectral features extracted by the adaptive dimensionality reduction layer, x represents the hyperspectral image, and t represents the parameters of the adaptive dimensionality reduction layer automatically learned by the network.

[0072] The spatial-spectral feature expression ability of the hyperspectral image classification method needs to be improved. Therefore, this application introduces the Gabor filter, encodes the directional information of the Gabor filter into the convolution layer, and generates a Gabor convolution filter. By learning the features of different directions, the robustness of the spatial-spectral feature is increased, which can improve the classification accuracy.

[0073] In order to reduce the uncertainty of the model caused by non-optimization of parameters, the parameters of the Gabor filter are obtained by sampling from a Gaussian distribution, and each parameter is represented by a learned mean and variance.

[0074] Furthermore, in the embodiment of the present application, a Gabor convolution layer is constructed using a Gabor filter, including:

[0075] Encode the direction information of the Gabor filter into the convolution layer to generate a Gabor convolution filter, and use the Gabor convolution filter as the Gabor convolution layer;

[0076] Among them, the Gabor filter is expressed as:

[0077]

[0078]

[0079]

[0080] Where g represents the Gabor filter, λ represents the wavelength of the sinusoidal component, α and φ represent the normal direction and phase offset respectively, δ represents the standard deviation of the Gaussian function, and γ represents the spatial aspect ratio. and They represent the horizontal and vertical coordinates after Gabor filter transformation, and i represents a complex unit.

[0081] Bayesian deep learning uses a Bayesian neural network to set the network's weights and biases as distributions rather than fixed values. During network initialization, the weights and biases are set to a Gaussian distribution with a mean of 0 and a variance of 1. This known distribution is called the prior distribution. The network learns the features of hyperspectral images and continuously adjusts the distribution of weights and biases. This distribution, learned by the network, is called the posterior distribution.

[0082] In an embodiment of the present application, since the true posterior distribution is difficult to obtain, the Gabor parameters of the optimization model are optimized through Bayesian inference to obtain the approximate posterior distribution of the Gabor parameters as the posterior distribution of the Gabor parameters, wherein Bayesian variational inference uses a known simple distribution to approximate the complex distribution to be inferred, and by limiting the type of approximate distribution, an optimal approximate posterior distribution is obtained.

[0083] Using Bayes' theorem, the prior distribution of Gabor parameters is Gaussian distribution. During the optimization process of the Bayesian neural network, samples of hyperspectral images are continuously input to adjust the form of the posterior distribution. When the loss function of the feature extraction stage in the Bayesian neural network reaches the minimum value, the prior distribution of the parameters fits the posterior distribution of the parameters best.

[0084] Among them, the loss function of the feature extraction stage is expressed as:

[0085]

[0086] Among them, L feature (θ, D) represents the loss function in the feature extraction stage, θ represents the parameters of the Bayesian neural network model in the feature extraction stage, D represents multiple input hyperspectral image samples, KL represents the Kullback-Leibler divergence, q(W|θ) represents the posterior distribution to be estimated, and P(W|D) represents the posterior distribution of Gabor parameters.

[0087] Furthermore, in the embodiment of the present application, Bayesian deep learning is used to characterize the parameter distribution of the Gabor convolution layer, including:

[0088] The parameter distribution of the Gabor filter satisfies the Gaussian distribution, and the Gaussian distribution is used as the prior distribution of the parameters in the Gabor convolution layer;

[0089] The distribution of Gabor parameters of the optimization model is optimized through Bayesian inference, and the approximate posterior distribution of the Gabor parameter distribution is obtained as the posterior distribution of the parameters in the Gabor convolution layer;

[0090] Fit the prior distribution of the parameters in the Gabor convolution layer to the posterior distribution of the parameters in the Gabor convolution layer;

[0091] The Gabor convolution layer is expressed as:

[0092]

[0093]

[0094] Among them, F represents the feature map output by the convolution operation, F n represents the nth feature map, Represents the convolution operation, C′ represents the filter modulated by the Gabor filter, and C represents the learnable filter in the convolution layer. It is a dot product operation, and g represents a Gabor filter whose prior distribution satisfies the Gaussian distribution.

[0095] Since hyperspectral image classification methods based on deep learning use a fully connected layer for classification in the classification stage, there are also a large number of parameters, making it difficult to find the optimal model parameters, resulting in high model uncertainty. Therefore, this application designs a Bayesian fully connected layer to implement a Bayesian approximation method for the distribution of fully connected layer parameters.

[0096] The Bayesian fully connected layer is designed and implemented based on MC-Dropout. Mathematically, MC-Dropout uses a dropout layer during model training, randomly dropping neurons to achieve Bayesian variational approximation of the model parameters using a Gaussian process with a Bernoulli distribution. During testing, enabling the dropout layer yields random sampling of the model parameters approximating the posterior distribution. The variance of the predictions represents the uncertainty of the model.

[0097] However, the ratio of randomly discarded neurons in the MC-Dropout method is a fixed value, and the optimal ratio cannot be determined, which limits its practical application. Therefore, this application designs an adaptive learning fully connected layer in the classification stage, namely the Bayesian fully connected layer, which does not require a specified ratio value. After defining the distribution form of the mask m of the discarded neurons, it is necessary to adaptively adjust the ratio of randomly discarded neurons by optimizing the distribution parameters a priori.

[0098] In the embodiment of the present application, the standard Dropout layer represents:

[0099]

[0100] Among them, z represents the output of the Dropout layer, m represents the mask vector, ζ represents the parameters to be learned by the model, and n represents the input feature information;

[0101] The mask distribution that satisfies adaptive learning is represented by g, and the mask m for the jth discarded neuron is j Obey m j ~g(m j ).

[0102] To characterize the mask m j The distribution of hidden variables r j Satisfy a subdistribution s(·), then, by introducing a monotone and differentiable function k(·) as a mapping function, we can directly obtain a bounded continuous variable m j =k(r j ).

[0103] The subdistribution s(·) and the mapping function k(·) satisfy two boundary conditions:

[0104] First, the parameters of s(·) and its distribution should be differentiable and easy to sample. Second, k(·) should be monotonically differentiable, with its output value range between [0, 1]. Any distribution that meets these two boundary conditions can constitute a subdistribution.

[0105] The relationship between s(·), k(·), and g can be obtained by calculating the formula for the distribution of the random variable function:

[0106]

[0107] Among them, g(m j ) represents the mask distribution that satisfies adaptive learning, s(·) represents the sub-distribution, k -1 (·) represents the inverse of the mapping function, m j represents the j-th mask.

[0108] In order to meet the two boundary conditions, in this application, Gaussian distribution is used to construct the adaptive distribution, k(·) allows m j falls into the bounded interval [0, 1], in which case the variable m j Can be determined by the subvariable r j Explicit definition:

[0109]

[0110]

[0111] Among them, m j represents the jth mask, k(·) represents the mapping function, r j represents the variable whose j-th mask satisfies sub-distribution r, μ represents the mean of the multivariate Gaussian distribution, and σ represents the variance of the multivariate Gaussian distribution.

[0112] In the embodiment of the present application, the Bayesian fully connected layer changes the learning of weights and biases of the fully connected layer commonly used in deep learning from fixed values ​​to learning the distribution of weights and biases. Adaptive learning is automatic learning through the network without the need for manual parameter setting.

[0113] Furthermore, in the embodiment of the present application, a Bayesian fully connected layer is constructed, including:

[0114] Construct a standard Dropout layer and adaptively learn the mask distribution of the Dropout layer. The Dropout layer that satisfies the adaptively learned mask distribution is used as a Bayesian fully connected layer.

[0115] Among them, the mask distribution of the Dropout layer is adaptively learned, including:

[0116] By defining the hidden variable to satisfy a sub-distribution and introducing a mapping function, the distribution of the mask is characterized, where the sub-distribution r satisfies differentiability and is easy to sample, the mapping function satisfies monotony and differentiability, and the output value range is [0, 1];

[0117] The distribution of the mask is expressed as:

[0118]

[0119]

[0120] Among them, mj represents the jth mask, k(·) represents the mapping function, r j represents the variable whose j-th mask satisfies sub-distribution r, μ represents the mean of the multivariate Gaussian distribution, and σ represents the variance of the multivariate Gaussian distribution.

[0121] In an embodiment of the present application, the mask distribution of the adaptive learning of the Bayesian fully connected layer is used as the prior distribution of the Bayesian fully connected layer parameters, and the approximate posterior distribution obtained by variational Bayesian inference is used as the posterior distribution of the Bayesian fully connected layer parameters, and the prior distribution of the Bayesian fully connected layer parameters is fitted to the posterior distribution of the Bayesian fully connected layer parameters.

[0122] Among them, the dimension of the parameter space to be learned in the network is high, which makes the posterior distribution difficult to calculate. Therefore, variational Bayesian inference is adopted. By using a known simple distribution to approximate the complex distribution to be inferred, and by restricting the type of approximate distribution, a posterior distribution of the approximate Gabor parameters that is locally optimal but has a definite solution is obtained.

[0123] During the optimization process of the Bayesian fully connected layer, samples of hyperspectral images are continuously input to adjust the form of the posterior distribution. When the loss function in the classification stage reaches the minimum value, the prior distribution of the Bayesian fully connected layer parameters fits the posterior distribution of the parameters best.

[0124] Among them, the loss function L in the classification stage classification It is defined as follows:

[0125]

[0126] Among them, L classification represents the classification loss function, m represents the number of hyperspectral image samples, y i Represents the label of the i-th hyperspectral image sample, y i ′ is the predicted probability of the i-th hyperspectral image sample.

[0127] Furthermore, in an embodiment of the present application, the Bayesian fully connected layer is made to learn the mask distribution of the standard Dropout layer, including:

[0128] The mask distribution of the Bayesian fully connected layer is adaptively learned to make the distribution of parameters in the classification stage satisfy the variational Bayesian inference of the Gaussian process.

[0129] This application provides an evaluation criterion for measuring the uncertainty of hyperspectral image classification methods - expected calibration error. The expected calibration error measures the degree of match between the accuracy and certainty of the model:

[0130]

[0131] That is, the model certainty value The prediction of the model under the condition that the probability p is equal to The probability of being the true label Y is also p. Therefore, the expected calibration error (ECE) can describe the matching degree of model learning.

[0132] In the embodiment of the present application, the model uncertainty in the feature extraction stage is estimated by learning the probability distribution of the Gabor convolution layer parameters.

[0133] In the embodiment of the present application, the model uncertainty in the classification stage is estimated by calculating the expected calibration error of the Bayesian neural network model.

[0134] where the expected calibration error is expressed as:

[0135]

[0136] where ECE is the expected calibration error, Express expectations, Represents the classification result of the Bayesian neural network model, Y represents the true label, represents the certainty value of the Bayesian neural network model, and p represents the true probability.

[0137] Writing the equation of the perfect calibration model above in the form of the expectation of the difference and further expanding the expectation yields:

[0138]

[0139] in:

[0140]

[0141]

[0142] Among them, B m The ECE is defined as the expected difference between the two. The smaller the ECE, the more certain the classification result.

[0143] Furthermore, in an embodiment of the present application, after characterizing the parameter distribution of the Gabor convolution layer using Bayesian deep learning, the model uncertainty of the feature extraction stage is estimated by learning the probability distribution of the Gabor convolution layer parameters;

[0144] After making the Bayesian fully connected layer learn the mask distribution of the standard Dropout layer, the model uncertainty of the classification stage is estimated.

[0145] Figure 2 This is another flowchart of the hyperspectral image classification method for uncertainty estimation according to an embodiment of the present application.

[0146] like Figure 2 As shown in the figure, a hyperspectral image is input and then reduced to 3 spectral dimensions through an adaptive dimensionality reduction layer. A 27*27 neighborhood centered on the labeled pixel is selected as the input hyperspectral image sample. The hyperspectral image sample is then fed into a Gabor convolutional layer for feature extraction and model parameter estimation. The extracted features are then fed into a ReLU activation function for nonlinear feature extraction. Next, the extracted features are fed into a max-pooling layer to reduce the size of the feature map. Step 2 of feature extraction is identical to step 1. After feature extraction, the features are fed into a Bayesian fully connected layer for classification, where the parameters in this layer are estimated to obtain the final classification result.

[0147] Figure 3 A schematic diagram of a hyperspectral image classification device for uncertainty estimation provided in Example 2 of this application

[0148] like Figure 3 As shown, the hyperspectral image classification device for uncertainty estimation includes:

[0149] An acquisition module 10 is used to acquire a hyperspectral image;

[0150] The spectral feature extraction module 20 is used to construct an adaptive dimensionality reduction layer, input the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtain a hyperspectral image after dimensionality reduction;

[0151] The spatial feature extraction module 30 is used to construct a Gabor convolution layer using a Gabor filter, characterize the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and input the hyperspectral image after dimensionality reduction into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image;

[0152] The classification module 40 is used to construct a Bayesian fully connected layer, so that the Bayesian fully connected layer learns the mask distribution of the standard Dropout layer, inputs the spectral-spatial joint features into the Bayesian fully connected layer, and performs classification through the Bayesian fully connected layer to obtain the classification result of the hyperspectral image.

[0153] The hyperspectral image classification device for uncertainty estimation of the embodiment of the present application includes an acquisition module for acquiring a hyperspectral image; a spectral feature extraction module for constructing an adaptive dimensionality reduction layer, inputting the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtaining a hyperspectral image after dimensionality reduction; a spatial feature extraction module for constructing a Gabor convolution layer using a Gabor filter, using Bayesian deep learning to characterize the parameter distribution of the Gabor convolution layer, and inputting the hyperspectral image after dimensionality reduction into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; and a classification module for constructing a Bayesian fully connected layer, causing the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer, inputting the spectral-spatial joint features into the Bayesian fully connected layer, and performing classification through the Bayesian fully connected layer to obtain a classification result of the hyperspectral image. Therefore, by characterizing the distribution of parameters through Bayesian neural networks, the problem of low classification accuracy and inability to provide the credibility of classification results in existing hyperspectral image classification methods based on deep learning is solved. At the same time, the Gabor convolution layer is used to increase the expression ability of spatial-spectral structure information, improve the classification accuracy, and provide uncertainty estimation of hyperspectral classification results, thereby increasing the credibility of the classification results.

[0154] In order to implement the above embodiments, the present invention further proposes a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the hyperspectral image classification method for uncertainty estimation described in the above embodiments is implemented.

[0155] To implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the hyperspectral image classification method for uncertainty estimation of the above embodiment is implemented.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0157] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0158] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0159] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0160] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0161] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0162] In addition, the functional units in the various embodiments of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0163] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A hyperspectral image classification method for uncertainty estimation, characterized in that: The following steps are involved: Acquire hyperspectral images; Constructing an adaptive dimensionality reduction layer, inputting the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtaining a hyperspectral image after dimensionality reduction; Constructing a Gabor convolution layer using a Gabor filter, characterizing the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and inputting the dimensionally reduced hyperspectral image into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; Constructing a Bayesian fully connected layer, causing the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer, inputting the spectral-spatial joint features into the Bayesian fully connected layer, and performing classification through the Bayesian fully connected layer to obtain a classification result of the hyperspectral image; The method of using Bayesian deep learning to characterize the parameter distribution of the Gabor convolution layer includes: The parameter distribution of the Gabor filter satisfies the Gaussian distribution, and the Gaussian distribution is used as the prior distribution of the parameters in the Gabor convolution layer; Obtaining an approximate posterior distribution of the Gabor parameter distribution as the posterior distribution of the parameters in the Gabor convolution layer by Bayesian inference optimization of the Gabor parameter distribution of the model; Fitting the prior distribution of the parameters in the Gabor convolution layer to the posterior distribution of the parameters in the Gabor convolution layer; The Gabor convolution layer is expressed as: Among them, F represents the feature map output by the convolution operation, F n represents the nth feature map, Represents the convolution operation, C′ represents the filter modulated by the Gabor filter, and C represents the learnable filter in the convolution layer. is a dot product operation, g represents a Gabor filter whose prior distribution satisfies the Gaussian distribution; The construction of the Bayesian fully connected layer includes: Constructing a standard Dropout layer, and adaptively learning the mask distribution of the Dropout layer, and using the Dropout layer that satisfies the adaptively learned mask distribution as the Bayesian fully connected layer; The adaptive learning of the mask distribution of the Dropout layer includes: By defining the hidden variable to satisfy a sub-distribution and introducing a mapping function, the distribution of the mask is characterized, where the sub-distribution r satisfies differentiability and is easy to sample, the mapping function satisfies monotony and differentiability, and the output value range is [0, 1]; The distribution of the mask is expressed as: Among them, m j represents the jth mask, k(·) represents the mapping function, r j represents the variable whose j-th mask satisfies sub-distribution r, μ represents the mean of the multivariate Gaussian distribution, and σ represents the variance of the multivariate Gaussian distribution.

2. The method according to claim 1, wherein The adaptive dimensionality reduction layer is expressed as: y=xt Where y represents the spectral features extracted by the adaptive dimensionality reduction layer, x represents the hyperspectral image, and t represents the parameters of the adaptive dimensionality reduction layer.

3. The method according to claim 1, wherein The Gabor convolution layer is constructed by using the Gabor filter, including: Encoding the direction information of the Gabor filter into a convolution layer to generate a Gabor convolution filter, and using the Gabor convolution filter as the Gabor convolution layer; Wherein, the Gabor filter is expressed as: Where g represents the Gabor filter, λ represents the wavelength of the sinusoidal component, α and φ represent the normal direction and phase offset respectively, δ represents the standard deviation of the Gaussian function, and γ represents the spatial aspect ratio. and They represent the horizontal and vertical coordinates after Gabor filter transformation, and i represents a complex unit.

4. The method according to claim 1, wherein The method of causing the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer includes: The mask distribution of the Bayesian fully connected layer is adaptively learned so that the distribution of parameters in the classification stage satisfies the variational Bayesian inference of the Gaussian process.

5. The method according to claim 1, wherein After characterizing the parameter distribution of the Gabor convolutional layer using Bayesian deep learning, the model uncertainty of the feature extraction stage is estimated; After the Bayesian fully connected layer is made to learn the mask distribution of the standard Dropout layer, the model uncertainty of the classification stage is estimated.

6. A hyperspectral image classification device for uncertainty estimation, characterized in that: include: An acquisition module, used to acquire hyperspectral images; A spectral feature extraction module is used to construct an adaptive dimensionality reduction layer, input the hyperspectral image into the adaptive dimensionality reduction layer to extract spectral features, and obtain a hyperspectral image after dimensionality reduction; A spatial feature extraction module is used to construct a Gabor convolution layer using a Gabor filter, characterize the parameter distribution of the Gabor convolution layer using Bayesian deep learning, and input the reduced-dimensional hyperspectral image into the Gabor convolution layer to extract the spectral-spatial joint features of the hyperspectral image; a classification module, configured to construct a Bayesian fully connected layer, enable the Bayesian fully connected layer to learn the mask distribution of the standard Dropout layer, input the spectral-spatial joint features into the Bayesian fully connected layer, perform classification through the Bayesian fully connected layer, and obtain a classification result of the hyperspectral image; Wherein, the spatial feature extraction module is further used to: The parameter distribution of the Gabor filter satisfies the Gaussian distribution, and the Gaussian distribution is used as the prior distribution of the parameters in the Gabor convolution layer; Obtaining an approximate posterior distribution of the Gabor parameter distribution as the posterior distribution of the parameters in the Gabor convolution layer by Bayesian inference optimization of the Gabor parameter distribution of the model; Fitting the prior distribution of the parameters in the Gabor convolution layer to the posterior distribution of the parameters in the Gabor convolution layer; The Gabor convolution layer is expressed as: Among them, F represents the feature map output by the convolution operation, F n represents the nth feature map, Represents the convolution operation, C′ represents the filter modulated by the Gabor filter, and C represents the learnable filter in the convolution layer. is a dot product operation, g represents a Gabor filter whose prior distribution satisfies the Gaussian distribution; The construction of the Bayesian fully connected layer includes: Constructing a standard Dropout layer, and adaptively learning the mask distribution of the Dropout layer, and using the Dropout layer that satisfies the adaptively learned mask distribution as the Bayesian fully connected layer; The adaptive learning of the mask distribution of the Dropout layer includes: By defining the hidden variable to satisfy a sub-distribution and introducing a mapping function, the distribution of the mask is characterized, where the sub-distribution r satisfies differentiability and is easy to sample, the mapping function satisfies monotony and differentiability, and the output value range is [0, 1]; The distribution of the mask is expressed as: Among them, m j represents the jth mask, k(·) represents the mapping function, r j represents the variable whose j-th mask satisfies sub-distribution r, μ represents the mean of the multivariate Gaussian distribution, and σ represents the variance of the multivariate Gaussian distribution.

7. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images