Hyperspectral image classification method and system based on physical information neural network

Abstract

The application discloses a hyperspectral image classification method and system based on a physical information neural network, first acquires a hyperspectral image; constructs a spectrum-image hybrid encoder and a continuous coordinate mapping network, which are used for extracting a data-driven feature vector and a physical feature vector of the hyperspectral image; and trains and optimizes the spectrum-image hybrid encoder and the continuous coordinate mapping network; finally, inputting the hyperspectral image to be classified into the trained and optimized spectrum-image hybrid encoder and continuous coordinate mapping network, the data-driven feature vector and the physical feature vector are extracted respectively, and after the data-driven feature vector and the physical feature vector are fused, each category probability is output through a linear classifier, and the category with the maximum probability is taken as the final classification result, solving the problems that the existing classification methods ignore spectral physical prior, have poor interpretability, and have insufficient generalization ability when labeled samples are scarce.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image processing technology, and more specifically to a hyperspectral image classification method based on a physical information neural network.

[0002] The present invention also relates to a hyperspectral image classification system based on a physical information neural network. Background Technology

[0003] Hyperspectral remote sensing images contain dozens to hundreds of continuous spectral bands, providing rich spectral information about ground features. They have wide applications in precision agriculture, mineral exploration, environmental monitoring, and urban planning. Hyperspectral image classification aims to assign a ground feature category label to each pixel in the image and is one of the core tasks of hyperspectral remote sensing data analysis.

[0004] In recent years, deep learning-based hyperspectral image classification methods have made significant progress, with discriminative models such as convolutional neural networks (CNNs) and Transformers becoming mainstream solutions. However, these purely data-driven methods have the following shortcomings: First, they ignore the physical laws inherent in hyperspectral data, such as the intrinsic relationship between spectral absorption features and ground material, leading to poor model interpretability. Second, in scenarios with scarce labeled samples, purely data-driven models are prone to overfitting and have limited generalization ability. Third, they fail to utilize the physical constraints of spectral mixing, making it difficult to handle mixed pixel problems. Physical Information Neural Networks (PINNs), as a paradigm that embeds prior physical knowledge into deep learning, have achieved widespread success in scientific computing in recent years. However, existing PINN methods are mainly applied to solving partial differential equations and have not yet effectively introduced the spectral absorption feature equations from derivative spectroscopy into hyperspectral image classification tasks. Furthermore, how to effectively integrate the physical features extracted by PINN with data-driven deep features to achieve synergistic enhancement of physical knowledge and data-driven approaches remains an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems by providing a hyperspectral image classification method and system based on a physical information neural network, thereby solving the problems of existing methods neglecting spectral physical priors, poor interpretability, and insufficient generalization ability when labeled samples are scarce.

[0006] The technical solution adopted in this invention is as follows: A hyperspectral image classification method based on a physical information neural network, the method comprising: Image acquisition, obtaining hyperspectral images; A feature extraction network is constructed, and a spectral spatial hybrid encoder and a PINN coordinate network are constructed. The hyperspectral image is input into the spectral spatial hybrid encoder and the PINN coordinate network respectively to extract data-driven feature vectors and physical feature vectors respectively. Training optimization involves jointly training and optimizing the spectral spatial hybrid encoder and the PINN coordinate network using classification loss, physical constraint loss, spectral reconstruction loss, and endmember diversity loss. The results are output by inputting the hyperspectral image to be classified into the trained and optimized spectral spatial hybrid encoder and PINN coordinate network to extract data-driven feature vectors and physical feature vectors, respectively. The data-driven feature vectors and physical feature vectors are fused and then passed through a linear classifier to output the probability of each category. The category with the highest probability is taken as the final classification result.

[0007] Furthermore, in the image acquisition process, the hyperspectral image is divided into blocks to obtain image blocks centered on each pixel.

[0008] Furthermore, the three-dimensional convolutional layer of the spectral spatial hybrid encoder includes a three-layer cascaded structure, with each convolutional kernel having 7, 5, and 3 convolutional kernels along the spectral dimensions, and the number of channels being 8, 16, and 32 respectively.

[0009] Furthermore, the image patch is input into the spectral spatial hybrid encoder to extract multi-scale spectral features layer by layer. The output of the three-dimensional convolutional layer is re-dimensionalized and then used to extract spatial features through two two-dimensional convolutional layers. Finally, the feature vector is output through global adaptive average pooling and fully connected mapping with layer normalization and random deactivation.

[0010] Furthermore, the backbone of the PINN coordinate network includes a multilayer perceptron structure, with each hidden layer configured with a hyperbolic tangent activation function. The physical feature vector includes reflectivity and absorption coefficient. The PINN coordinate network uses normalized continuous coordinates. For input, where λ is the wavelength coordinate. x and y Given spatial coordinates, a multi-layer fully connected network with the Tanh activation function is used to output the reflectance and absorption coefficient at these coordinates. The reflectance is constrained to the range of (0,1) by the Sigmoid function, and the absorption coefficient is constrained to a positive value by the Softplus function.

[0011] Furthermore, a normalized coordinate point set is generated for each band of the center pixel of the image block. The normalized coordinate point set is input into the PINN coordinate network to extract the absorption coefficient sequence corresponding to each band. The sequence is then transformed by a fully connected transformation, normalized, and activated to form a physical feature vector.

[0012] Furthermore, in the training optimization: The classification loss is the cross-entropy loss between the predicted result of the fused features after passing through the classifier and the true label. The physical constraint loss is a weighted sum of the differential equation constraints and the data fitting; The spectral reconstruction loss is based on an endmember linear mixture model, which predicts the abundance coefficient from the fusion features, multiplies it with the learnable endmember spectral matrix to reconstruct the spectrum, and calculates the mean square error between the reconstructed spectrum and the mean spectrum of the image patch space. The endmember diversity loss is a negative penalty for the cosine distance between different endmember spectra; The total loss is the weighted sum of the classification loss, physical constraint loss, spectral reconstruction loss, and endmember diversity loss.

[0013] Furthermore, the physical constraint loss specifically includes: Based on derivative spectroscopy, a differential equation for spectral absorption is established as a physical canonical term, as follows: (1) In the formula, This is the second derivative of reflectivity with respect to wavelength. The absorption coefficient is... The reflectivity response is determined by using an automatic differentiation technique to continuously differentiate the reflectivity field output by the PINN coordinate network twice. First, the first-order gradient with respect to wavelength is calculated, and then the second-order gradient is calculated. The residual of the spectral absorption differential equation is evaluated at the spatially randomly scattered point configuration location, and the mean square loss is calculated as the equation constraint term. At the actual observation location, the fitting error term between the reflectivity reconstructed by the PINN coordinate network and the original spectrum is calculated. The equation constraint term and the fitting error term are summed to obtain the physical constraint loss.

[0014] Furthermore, the endmember linear mixing model includes K sets of trainable endmember spectral parameters, where K is the total number of categories. Each set of endmember spectra is mapped to the closed interval [0,1] by a Sigmoid function to satisfy the physical boundary of reflectance. The joint representation is processed by an abundance estimation subnetwork and Softmax normalization to output the mixing coefficients of each endmember. The mixing coefficients satisfy the dual constraints of nonnegativity and normalization. The reconstructed spectrum is obtained by linearly weighting and summing the endmember spectra according to the mixing coefficients. The endmember diversity loss is achieved by calculating the global mean of the absolute values ​​of the pairwise cosine similarities of each endmember spectrum after normalization.

[0015] This invention also employs a hyperspectral image classification system based on a physical information neural network and a hyperspectral image classification method based on a physical information neural network. The system includes: The image acquisition module is used to acquire hyperspectral images; The feature extraction network module is used to construct a spectral spatial hybrid encoder and a PINN coordinate network. The hyperspectral image is input into the spectral spatial hybrid encoder and the PINN coordinate network respectively to extract data-driven feature vectors and physical feature vectors. The training module is used to jointly train and optimize the spectral spatial hybrid encoder and the PINN coordinate network using classification loss, physical constraint loss, spectral reconstruction loss and endmember diversity loss. The result output module is used to input the hyperspectral image to be classified into the trained and optimized spectral spatial hybrid encoder and PINN coordinate network to extract data-driven feature vectors and physical feature vectors respectively. After fusing the data-driven feature vectors and physical feature vectors, the linear classifier outputs the probability of each category, and the category with the highest probability is taken as the final classification result.

[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention first employs the PINN coordinate network to embed the spectral absorption characteristic equation from derivative spectroscopy as a physical prior into the neural network. Through automatic differentiation, it accurately calculates the higher-order derivative of spectral reflectance, enabling the network to consciously follow the physical laws of spectral absorption during the learning process. Then, it fuses the absorption coefficient features extracted by the PINN with the data-driven features extracted by the spectral spatial hybrid encoder, achieving synergistic enhancement of physical knowledge and data-driven approaches. During the training phase, the gradient of the classification loss flows simultaneously to both the spectral spatial hybrid encoder and the PINN coordinate network, allowing physical knowledge to directly participate in the optimization of classification decisions. The endmember fusion model further introduces physical constraints for spectral unmixing, enhancing the model's ability to handle mixed pixels.

[0017] 2. This invention, from the perspective of physical information neural networks, organically combines the physical priors of derivative spectroscopy with the data-driven capabilities of deep learning, giving full play to the advantages of PINN in embedding physical constraints and the feature extraction capabilities of the spectral spatial hybrid encoder.

[0018] 3. This invention improves the generalization ability of the model in scenarios with scarce labeled samples by embedding physical priors into the spectral absorption feature equation; the physical features have clear physical meanings, enhancing the interpretability of the model; the fully connected design of the computation graph allows physical knowledge to directly participate in classification optimization, avoiding the separation between physical branches and classification branches; the endmember fusion model introduces spectral unmixing constraints, enhancing the ability to handle fused pixels; classification can be completed with only one forward propagation during the inference phase, resulting in high computational efficiency. Attached Figure Description

[0019] Figure 1 This is a flowchart of the hyperspectral image classification method based on physical information neural networks of the present invention; Figure 2 This is a schematic diagram of the structure of the PINN coordinate network and the ordinary differential equation constraint of spectral absorption characteristics in the method of the present invention; Figure 3This is a schematic diagram illustrating the mechanism of fusion classification of physical feature vectors and data-driven feature vectors in the method of this invention. Detailed Implementation

[0020] The present invention will now be described in detail with reference to the accompanying drawings.

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0022] Example This embodiment provides a hyperspectral image classification method based on a physical information neural network, such as... Figure 1 As shown, based on the spectral absorption characteristic equation in derivative spectroscopy, a spectral field coordinate network, namely the PINN coordinate network, is used to learn the physical characteristics of the continuous spectral field, including reflectance and absorption coefficient. This is combined with a spectral spatial hybrid encoder to extract data-driven features. Hyperspectral image classification is achieved through the fusion of physical and data-driven features. Specifically, for the central pixel of the data cube, a normalized coordinate point set is constructed for all its bands, and then fed into the PINN coordinate network to extract the absorption coefficient sequence corresponding to each band. Where C is the total number of bands, the sequence is transformed by a fully connected layer, normalized, and activated to form a physical driving embedding vector. With data-driven vectors Perform a splicing operation to obtain a joint characterization Finally, a fully connected classification layer is used to predict the class affiliation; specifically including: Image acquisition involves dividing the hyperspectral image to be classified into blocks to obtain image blocks centered on each pixel. This process involves cropping the hyperspectral remote sensing image according to the neighborhood of the pixel center to form a three-dimensional data cube of fixed size.

[0023] A feature extraction network is constructed, in which image patches are input into a spectral spatial hybrid encoder for feature extraction. The spectral spatial hybrid encoder employs a 3D convolutional module to extract inter-band correlations along the spectral dimension, followed by a 2D convolutional module to capture spatial texture information. Finally, the features are mapped through a fully connected layer with global pooling and normalization to generate data-driven embedding vectors. ; The 3D convolutional module of the spectral spatial hybrid encoder adopts a three-level cascaded architecture. The receptive fields of each level of convolutional kernel in the spectral dimension are set to 7, 5, and 3 respectively, and the number of channels is 8, 16, and 32 respectively. Multi-scale spectral features are extracted layer by layer to achieve multi-scale spectral feature extraction from coarse to fine. After the 3D convolutional output is reshaped by tensor, it is input into the second-level 2D convolutional module (with 256 and 128 channels respectively) for spatial feature learning and extraction. Finally, a fixed-dimensional feature vector is generated by global adaptive average pooling and fully connected mapping with normalization and random deactivation.

[0024] In this embodiment, the backbone of the PINN coordinate network adopts a multilayer perceptron structure. All hidden layers are configured with hyperbolic tangent activation functions to ensure that the network output has high-order smoothness with respect to the input coordinates, thereby ensuring that the automatic differentiation algorithm can accurately calculate the second-order partial derivatives. In the dual-branch prediction head at the end of the network, the reflectivity branch compresses the output to the open interval (0,1) through the Sigmoid function to conform to the physical domain of reflectivity, and the absorption coefficient branch maps the output to positive real numbers through the Softplus function to satisfy the non-negative physical characteristics of the absorption coefficient.

[0025] Specifically, this embodiment employs a 3D-2D hybrid convolutional structure as a spectral spatial hybrid encoder to encode hyperspectral image patches into compact feature vectors. The input image patch contains several spectral bands and a fixed spatial size. The encoding process is as follows: First, the input is increased by one channel dimension to form a five-dimensional tensor. Then, it is sequentially passed through three layers of 3D convolution, with kernels along spectral dimensions of 7, 5, and 3, and spatial dimensions of 3×3. The number of channels is 8, 16, and 32 respectively, and the padding methods are (3, 1, 1), (2, 1, 1), and (1, 1, 1) to maintain the feature map size. Each layer is followed by batch normalization and a ReLU activation function to extract multi-scale spectral features layer by layer. After the 3D convolution output... The channel dimension and spectral dimension are merged and recombined into a four-dimensional tensor. Spatial features are then extracted through two layers of two-dimensional convolutions, with a kernel size of 3×3 and padding of 1. No bias term is used, and the number of channels is 256 and 128 respectively. Each layer is followed by batch normalization and ReLU activation function. Finally, the spatial dimension is compressed to 1×1 by global adaptive average pooling. After flattening, a fixed-dimensional data-driven feature vector is output through a linear projection layer (containing layer normalization and Dropout, with a Dropout rate of 0.2). All convolutional layer weights are initialized using Kaiming normalization (fan_out mode), and the bias is initialized to zero. The weights of the batch normalization layer are initialized to 1, and the bias is initialized to zero.

[0026] Constructing the PINN coordinate network The PINN coordinate network generates a normalized set of coordinate points for each band of the center pixel of the image block. This set of coordinates is generated using normalized continuous triplet coordinates. For input, where, Using spectral dimensions (wavelength coordinates) and x and y as spatial coordinates, a multilayer fully connected network with the Tanh activation function is used. After processing by a multilayer perceptron, the network branches into two prediction branches to output the reflectivity field. With absorption coefficient field Obtain the reflectivity r and absorption coefficient at that coordinate. The reflectivity is constrained to the range of (0, 1) by the Sigmoid function, and the absorption coefficient is constrained to a positive value by the Softplus function to ensure strict non-negativity. In this embodiment, the PINN coordinate network uses a three-layer fully connected network as its backbone with a hidden dimension of 64. Each layer is followed by a Tanh activation function to ensure the high-order continuous differentiability of the output with respect to the input coordinates, allowing the first-order partial derivative of reflectivity with respect to wavelength to be accurately calculated using torch.autograd.grad. and second-order partial derivatives ; Specifically, such as Figure 2As shown, the PINN coordinate network constructed in this embodiment is a physical information coordinate network. Based on the principle of derivative spectroscopy, it learns the continuous spectral field and maps any spatial-spectral coordinate to the reflectivity and absorption coefficient at that location. The structure of the PINN coordinate network is as follows: The backbone network consists of three fully connected layers stacked together. The input dimension is 3 (corresponding to wavelength coordinates, spatial horizontal coordinates, and spatial vertical coordinates, all normalized to the range of zero to one), and the hidden dimension is 64. Each layer is followed by a Tanh activation function. The key reason for choosing Tanh instead of ReLU is that the Tanh function is smooth globally and has arbitrary-order continuous differentiability, which is a necessary condition for the subsequent accurate calculation of the second-order partial derivative of reflectivity with respect to wavelength through automatic differentiation. After the backbone network, there are two independent output heads: the reflectivity head passes through a linear layer and is then mapped to the range of zero to one by the Sigmoid function to satisfy the physical constraints of reflectivity; the absorption coefficient head passes through a linear layer and is then... The absorption coefficient is mapped to a strictly positive value by the Softplus function, satisfying the physical constraint that the absorption coefficient is non-negative. The physical feature extraction process is as follows: Normalized wavelength coordinates are generated for each band of the center pixel of the image block. The coordinates are uniformly sampled between zero and one, and the spatial coordinates are fixed at the center position of 0.5. This forms a three-dimensional coordinate point with the same number of bands. The coordinates are expanded into a batch form and gradient tracking is enabled to support subsequent automatic differentiation. The absorption coefficients of each band are obtained by inputting them into the PINN coordinate network and forming an absorption coefficient vector. After the absorption coefficient vector is passed through a linear projection layer, layer normalization, and ReLU activation, a physical feature vector with the same dimension as the output of the spectral spatial hybrid encoder is obtained. The key to this design is that the output of the PINN coordinate network directly participates in the subsequent feature fusion and classification. Therefore, the gradient of the classification loss can be backpropagated to the PINN coordinate network through the computation graph, so that the physical knowledge learned by the PINN coordinate network directly affects the classification decision.

[0027] Training optimization employs a joint training approach using classification loss, physical constraint loss, spectral reconstruction loss, and endmember diversity loss. The classification loss is the cross-entropy loss between the predicted result of the fused features after passing through the classifier and the true label. Its gradient is simultaneously backpropagated to the spectral spatial hybrid encoder and the PINN coordinate network, allowing physical knowledge to directly influence classification. That is, the backward gradient of the classification target will simultaneously update the parameters of the encoding network and the PINN coordinate network, realizing the direct guidance of physical knowledge on classification decisions through absorption coefficient features. The physical constraint loss consists of two parts: first, calculating the mean square value of the residual of the spectral absorption feature ordinary differential equation using torch.autograd.grad at randomly configured points; second, calculating the mean square error between the PINN coordinate network's predicted reflectance and the observed reflectance at real data points. The physical constraint loss is calculated by establishing a spectral absorption differential equation based on derivative spectroscopy as a physical regularization term. Automatic differentiation is used to continuously differentiate the reflectance field output by the PINN coordinate network twice. First, the first-order gradient with respect to wavelength is calculated, and then the second-order gradient is calculated. The residual of the differential equation is evaluated at randomly scattered points in space, and the mean square loss is calculated as a constraint term. Furthermore, at the actual observation location, the fitting error between the reflectance reconstructed by the PINN coordinate network and the actual observed spectrum (i.e., the original spectrum) is calculated. The complete physical constraint loss is obtained by summing the constraint term and the fitting error term. ; The ordinary differential equation for spectral absorption characteristics originates from the principle of derivative spectroscopy, as shown in the following equation: (1) In the formula, This is the second derivative of reflectivity with respect to wavelength. The absorption coefficient is... This refers to the reflectivity response; its physical meaning is: under conditions of a large absorption coefficient, the reflectivity curve is concave downwards (corresponding to a negative second derivative), forming a characteristic absorption valley; the second partial derivative is achieved through nested automatic differentiation: the first round of differentiation obtains the first derivative by taking the partial derivative of the reflectivity r output by the PINN coordinate network with respect to the wavelength dimension of the input coordinates. Eliminating additive illumination shift and highlighting the absorption characteristic slope; the second derivative is obtained by taking the partial derivative of the first derivative with respect to the wavelength dimension in the second round of differentiation. Locate the center and depth of the absorption peak; both rounds of differentiation use the computational graph preservation mode to support end-to-end gradient backpropagation; the absorption coefficient κ is directly related to the material of the ground cover and is the physical basis for classification. Specifically, the calculation of the physical constraint loss is divided into two parts: The first part is the residual of the ordinary differential equation for the placement points. Coordinates of 32 placement points are randomly sampled from a 3D space from zero to one, and gradient tracing is enabled. The placement points are input into the PINN coordinate network to obtain reflectivity and absorption coefficient. The first-order partial derivative of reflectivity with respect to the wavelength component of the input coordinates is calculated using automatic differentiation, maintaining the computational graph to support second-order differentiation. The second-order partial derivative is then obtained by further differentiating the first-order partial derivative, again maintaining the computational graph to ensure gradient backpropagation. The residual of the ordinary differential equation is calculated based on the spectral absorption characteristic equation, and the mean of the squared residuals is taken as the ordinary differential equation loss. The second part is the data fitting loss. Wavelength coordinates are generated for the center pixel of the first sample in the batch, input into the PINN coordinate network to obtain the predicted reflectivity, and the mean square error between this and the observed reflectivity of that pixel is calculated as the data loss. The physical constraint loss is the sum of the above two parts.

[0028] Among them, spectral reconstruction loss Mean square deviation of reconstructed spectrum from image patch spatial mean spectrum in linear mixture model, and endmember diversity loss. The negative penalty for the cosine distance between different endmember spectra is as follows: The spectral reconstruction loss is based on an endmember linear mixture model, which predicts the abundance coefficients from the fusion features. The coefficients are then passed through Softmax to ensure they are non-negative and sum to one. The reconstructed spectrum is multiplied by the learnable endmember spectral matrix, and the mean square error between the reconstructed spectrum and the mean spectrum of the image patch space is calculated. The endmember diversity loss is the mean absolute value of the cosine similarity matrix (excluding the diagonal) of each row of the endmember spectral matrix after normalization, which penalizes overly similar endmembers. The total loss is the weighted sum of the above four losses. The endmember linear mixture model holds K sets of trainable endmember spectral parameters, where K equals the total number of land cover categories. Each set of endmember spectra is mapped to the closed interval [0,1] by the Sigmoid function to satisfy the physical boundary of reflectance. The joint representation outputs the mixing coefficients of each endmember after being processed by the abundance estimation subnetwork and Softmax normalization. These coefficients satisfy the dual constraints of nonnegativity and normalization. The reconstructed spectrum is obtained by linearly weighting and summing the endmember spectra according to the mixing coefficients. The endmember diversity loss is achieved by calculating the global mean of the absolute values ​​of the pairwise cosine similarity of each endmember spectrum after L2 normalization. This loss term suppresses excessive similarity between endmembers to ensure the physical distinguishability of the endmembers.

[0029] Specifically, such as Figure 3 As shown, this embodiment introduces an endmember fusion model, further enhancing the physical interpretability of the model based on linear spectral fusion theory. This theory posits that the spectrum of each pixel can be represented as a linearly weighted combination of several pure endmember spectra. The endmember fusion model contains a learnable endmember spectral matrix, with the number of rows equal to the number of categories and the number of columns equal to the number of bands. Its original parameters are initialized with a normal distribution with a mean of zero and a standard deviation of 0.5, and mapped to the range of zero to one using the Sigmoid function, satisfying the physical constraints of reflectance. The abundance coefficient prediction process is as follows: the fused feature vector is input into the abundance prediction network, i.e., a two-layer fully connected network with a middle dimension of 64, and ReLU activation is used in the middle. The output is normalized by Softmax to obtain an abundance coefficient vector that satisfies the non-negative and summed linear mixture physical constraint. The spectrum is reconstructed by multiplying the abundance coefficients with the endmember spectral matrix. The mean square error between the reconstructed spectrum and the image patch spatial mean spectrum is calculated as the reconstruction loss. The image patch spatial mean spectrum is obtained by taking the mean of the input image patch in the spatial dimension. The endmember diversity loss is calculated as follows: each row of the endmember spectral matrix is ​​L2 normalized, the normalized similarity matrix is ​​calculated, and the mean of the absolute values ​​of the diagonal elements after being set to zero by a mask is used as the diversity loss. This penalizes overly similar endmember spectra and encourages each endmember to learn a discriminative spectral pattern.

[0030] In summary, the total loss function during the training phase of this embodiment is a weighted sum of the four losses mentioned above, where the weight of the physical constraint loss is 0.01, the weight of the reconstruction loss is 0.1, the weight of the endmember diversity loss is 0.05, and the weight of the classification loss is 1. The core design of the training strategy in this embodiment lies in the full connectivity of the computation graph. The gradient of the classification loss flows simultaneously to the spectral spatial hybrid encoder and the PINN coordinate network, enabling data-driven features and physical features to learn collaboratively under the unified supervision of the classification target. The physical constraint loss independently constrains the PINN coordinate network to follow the spectral absorption equation, ensuring the physical rationality of the physical features. The two types of losses each perform their respective functions while cooperating with each other, realizing a deep integration of physical knowledge and data-driven approaches.

[0031] In the inference phase, classification is performed using direct forward propagation. The image cube to be classified, i.e., the hyperspectral image block to be classified, is simultaneously input into the trained spectral spatial hybrid encoder and the PINN coordinate network. The spectral spatial hybrid encoder outputs a data-driven feature vector, and the PINN coordinate network outputs the absorption coefficients of each band, which are then projected to obtain the physical feature vector. With physical eigenvectors The features are concatenated and fused to obtain a fused feature vector. The two are concatenated and input into a linear classifier to obtain the posterior probability distribution of each category, i.e., the predicted score. The category with the highest score is selected as the classification label of the pixel, i.e., the final classification result. The entire inference process only requires one forward propagation each for the spectral spatial hybrid encoder and the PINN coordinate network, which is computationally efficient. The physical features, i.e., the absorption coefficient, provide a discriminative basis with clear physical meaning for classification, which enhances the interpretability of the model.

[0032] This embodiment also provides a hyperspectral image classification system based on a physical information neural network. The system employs a hyperspectral image classification method based on a physical information neural network and includes: The image acquisition module is used to acquire hyperspectral images; The feature extraction network module is used to construct a spectral spatial hybrid encoder and a PINN coordinate network. The hyperspectral image is input into the spectral spatial hybrid encoder and the PINN coordinate network respectively to extract data-driven feature vectors and physical feature vectors. The training module is used to jointly train and optimize the spectral spatial hybrid encoder and the PINN coordinate network using classification loss, physical constraint loss, spectral reconstruction loss and endmember diversity loss. The result output module is used to input the hyperspectral image to be classified into the trained and optimized spectral spatial hybrid encoder and PINN coordinate network to extract data-driven feature vectors and physical feature vectors respectively. After fusing the data-driven feature vectors and physical feature vectors, the linear classifier outputs the probability of each category, and the category with the highest probability is taken as the final classification result.

[0033] Verification Example 1. Experimental setup This experiment uses two standard hyperspectral datasets, Trento and Houston 2013, to verify the classification performance of the PINN method of this invention, and selects SSFTT, CNN, and SpectralFormer as control methods.

[0034] The Trento dataset, collected from southern Trento, contains 63 effective hyperspectral bands, with a spatial resolution of 1m, an image size of 600×166 pixels, and covers 6 types of land cover. The Houston 2013 dataset, collected from a university campus and its surrounding area, contains 144 hyperspectral bands, with a spatial resolution of 2.5m, an image size of 349×1905 pixels, and covers 15 types of complex land cover.

[0035] Dataset partitioning rules: 40 samples are selected for each class in the training set; the validation set accounts for 20% of the total valid samples and is allocated according to the class ratio; the remaining samples are used as the test set; overall accuracy (OA), average accuracy (AA), and Kappa coefficient are used as evaluation indicators.

[0036] 2. Experimental Results As shown in Table 1, on the Trento dataset, the PINN method of this invention outperforms the comparison methods in all metrics. The PINN method achieves an OA of 0.9917, an AA of 0.9830, and a Kappa coefficient of 0.9889, all ranking first. Compared to SSFTT, PINN improves OA by 0.27%, AA by 0.42%, and Kappa coefficient by 0.36%; compared to CNN, the improvement in accuracy is more significant. In terms of category accuracy, PINN classifies various land features accurately, especially for easily confused buildings and roads, demonstrating better classification stability.

[0037] Table 1 Comparison of experimental results for different methods on the Trento dataset.

[0038] As shown in Tables 2 and 3, the PINN method maintains optimal performance on the complex Houston dataset. The proposed method achieves an OA of 0.9741, an AA of 0.9779, and a Kappa coefficient of 0.9719, all outperforming SSFTT, CNN, and SpectralFormer. Compared to the CNN method, PINN improves OA by 1.23% and Kappa coefficient by 1.33%. Compared to SpectralFormer, the accuracy improvement is significant. PINN achieves high-precision classification across most land cover categories, with balanced accuracy across categories, no obvious weaknesses, and is suitable for complex land cover scenarios.

[0039] Table 2 Comparison of experimental results of different methods on the Houston dataset (Part 1)

[0040] Table 3 Comparison of experimental results of different methods on the Houston dataset (Part 2)

[0041] In summary, the experimental results from both datasets demonstrate that the PINN method proposed in this invention outperforms existing mainstream hyperspectral classification methods such as SSFTT, CNN, and SpectralFormer. It achieves high-precision and high-stability land cover classification in both simple and complex multi-class scenarios, effectively improving the recognition accuracy of easily confused land covers. It exhibits strong generalization ability and addresses the problems of low accuracy and poor generalization in traditional methods, demonstrating significant practical value and application prospects.

[0042] This article uses specific embodiments to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A hyperspectral image classification method based on a physical information neural network, characterized in that, The method includes: Image acquisition, obtaining hyperspectral images; A feature extraction network is constructed, comprising a spectral spatial hybrid encoder and a PINN coordinate network. The hyperspectral image is input into the spectral spatial hybrid encoder and the PINN coordinate network, respectively, to extract data-driven feature vectors and physical feature vectors. Specifically, the hyperspectral image is input into the spectral spatial hybrid encoder to extract multi-scale spectral features layer by layer. The output of the 3D convolutional layer is redimensionalized and then used to extract spatial features through two 2D convolutional layers. Finally, the data-driven feature vector is output through global adaptive average pooling and a fully connected mapping with layer normalization and random deactivation. The backbone of the PINN coordinate network includes a multilayer perceptron structure, where the hidden layers are configured with hyperbolic tangent activation functions. The physical feature vector includes reflectance and absorption coefficient. The PINN coordinate network uses normalized continuous coordinates. For input, where λ is the wavelength coordinate. x and y Using spatial coordinates, a multi-layer fully connected network with the Tanh activation function is used to output the reflectance and absorption coefficient at these coordinates. The reflectance is constrained to the range of (0,1) by the Sigmoid function, and the absorption coefficient is constrained to a positive value by the Softplus function. Training optimization involves jointly training and optimizing the spectral spatial hybrid encoder and the PINN coordinate network using classification loss, physical constraint loss, spectral reconstruction loss, and endmember diversity loss. The results are output by inputting the hyperspectral image to be classified into the trained and optimized spectral spatial hybrid encoder and PINN coordinate network to extract data-driven feature vectors and physical feature vectors, respectively. The data-driven feature vectors and physical feature vectors are fused and then passed through a linear classifier to output the probability of each category. The category with the highest probability is taken as the final classification result.

2. The hyperspectral image classification method based on a physical information neural network according to claim 1, characterized in that, In the image acquisition process, the hyperspectral image is divided into blocks to obtain image blocks centered on each pixel.

3. The hyperspectral image classification method based on a physical information neural network according to claim 1, characterized in that, The three-dimensional convolutional layer of the spectral spatial hybrid encoder includes a three-layer cascaded structure, with each level of convolutional kernel having 7, 5, and 3 convolutional kernels along the spectral dimensions, and the number of channels being 8, 16, and 32 respectively.

4. The hyperspectral image classification method based on a physical information neural network according to claim 2, characterized in that, A normalized coordinate point set is generated for each band of the center pixel of the image block. The normalized coordinate point set is input into the PINN coordinate network to extract the absorption coefficient sequence corresponding to each band. The sequence is processed by fully connected transformation, normalization and activation function to form a physical feature vector.

5. The hyperspectral image classification method based on a physical information neural network according to claim 1, characterized in that, In the training optimization: The classification loss is the cross-entropy loss between the predicted result of the fused features after passing through the classifier and the true label. The physical constraint loss is a weighted sum of the differential equation constraints and the data fitting; The spectral reconstruction loss is based on an endmember linear mixture model, which predicts the abundance coefficient from the fusion features, multiplies it with the learnable endmember spectral matrix to reconstruct the spectrum, and calculates the mean square error between the reconstructed spectrum and the mean spectrum of the image patch space. The endmember diversity loss is a negative penalty for the cosine distance between different endmember spectra; The total loss is the weighted sum of the classification loss, physical constraint loss, spectral reconstruction loss, and endmember diversity loss.

6. The hyperspectral image classification method based on a physical information neural network according to claim 5, characterized in that, The physical constraint loss specifically includes: Based on derivative spectroscopy, a differential equation for spectral absorption is established as a physical canonical term, as follows: （1） In the formula, This is the second derivative of reflectivity with respect to wavelength. The absorption coefficient is... The reflectivity response is determined by using an automatic differentiation technique to continuously differentiate the reflectivity field output by the PINN coordinate network twice. First, the first-order gradient with respect to wavelength is calculated, and then the second-order gradient is calculated. The residual of the spectral absorption differential equation is evaluated at the spatially randomly scattered point configuration location, and the mean square loss is calculated as the equation constraint term. At the actual observation location, the fitting error term between the reflectivity reconstructed by the PINN coordinate network and the original spectrum is calculated. The equation constraint term and the fitting error term are summed to obtain the physical constraint loss.

7. The hyperspectral image classification method based on a physical information neural network according to claim 6, characterized in that, The endmember linear mixture model includes K sets of trainable endmember spectral parameters, where K is the total number of categories. Each set of endmember spectra is mapped to the closed interval [0,1] by the Sigmoid function to satisfy the physical boundary of reflectivity. The joint characterization outputs the mixing coefficients of each endmember after being processed by the abundance estimation subnetwork and Softmax normalization. The mixing coefficients satisfy the dual constraints of nonnegativity and normalization. The reconstructed spectrum is obtained by linearly weighting and summing the spectra of each endmember according to the mixing coefficient; the endmember diversity loss is achieved by calculating the global mean of the absolute values ​​of the pairwise cosine similarity of each endmember spectrum after normalization.

8. A hyperspectral image classification system based on a physical information neural network, employing the hyperspectral image classification method based on a physical information neural network as described in any one of claims 1 to 7, characterized in that, The system includes: The image acquisition module is used to acquire hyperspectral images; The feature extraction network module is used to construct a spectral spatial hybrid encoder and a PINN coordinate network. The hyperspectral image is input into the spectral spatial hybrid encoder and the PINN coordinate network respectively to extract data-driven feature vectors and physical feature vectors. Specifically, the hyperspectral image is input into the spectral spatial hybrid encoder to extract multi-scale spectral features layer by layer. The output of the three-dimensional convolutional layer is redimensionalized and then used to extract spatial features through two two-dimensional convolutional layers. Finally, the data-driven feature vector is output through global adaptive average pooling and a fully connected mapping with layer normalization and random deactivation. The backbone of the PINN coordinate network includes a multilayer perceptron structure, where the hidden layers are configured with hyperbolic tangent activation functions. The physical feature vector includes reflectance and absorption coefficient. The PINN coordinate network uses normalized continuous coordinates. For input, where λ is the wavelength coordinate. x and y Using spatial coordinates, a multi-layer fully connected network with the Tanh activation function is used to output the reflectance and absorption coefficient at these coordinates. The reflectance is constrained to the range of (0,1) by the Sigmoid function, and the absorption coefficient is constrained to a positive value by the Softplus function. The feature fusion module is used to fuse the data-driven feature vector with the physical feature vector and then output the prediction result through a linear classifier. The training module is used to jointly train and optimize the spectral spatial hybrid encoder and the PINN coordinate network using classification loss, physical constraint loss, spectral reconstruction loss and endmember diversity loss. The result output module is used to input the hyperspectral image to be classified into the trained and optimized spectral spatial hybrid encoder and PINN coordinate network to extract data-driven feature vectors and physical feature vectors respectively. After fusing the data-driven feature vectors and physical feature vectors, the linear classifier outputs the probability of each category, and the category with the highest probability is taken as the final classification result.

Images