Hyperspectral remote sensing image classification method and system

Abstract

The present invention provides a hyperspectral remote sensing image classification method and system, belonging to the field of image processing technology. The method performs dimensionality reduction processing on hyperspectral remote sensing images; extracts spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; extracts spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map; extracts spectral key information and spatial key information from the spatial-spectral fusion feature map; and uses an optimizer to process the image after the spectral key information and spatial key information are extracted to obtain an image classification result. The method adopts residual connections to effectively reduce the gradient vanishing phenomenon; combines spectral and spatial attention mechanisms to extract more complete spatial and spectral features, effectively suppressing the influence of noise; and uses entropy superpixels to pre-process hyperspectral images. Compared with traditional principal component analysis methods, it pays more attention to the relationship between different bands, improves image classification accuracy, and provides more accurate and clear classification results.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a hyperspectral remote sensing image classification method and system based on a convolutional neural residual network combined with a spectral attention mechanism and a spatial attention mechanism. Background Art

[0002] Hyperspectral remote sensing images, acquired using a hyperspectral imager, contain rich spatial and spectral information. Furthermore, hyperspectral images possess a greater number of bands and extremely high resolution. Therefore, analysis of their spectral and spatial characteristics can reveal detailed features of land features. Currently, hyperspectral imaging technology has been widely applied in fields such as precision agriculture, atmospheric monitoring, and oceanographic surveys. As the application of hyperspectral images expands across a wide range of fields, the rapid and accurate identification of each pixel in a hyperspectral image has become a primary challenge.

[0003] Traditional methods for hyperspectral image classification include random forests, decision trees, and support vector machines. These methods rely on handcrafted features and require extensive prior knowledge of hyperspectral imagery. Both labeling and classification are labor-intensive, slow, and inefficient. Furthermore, these traditional methods ignore rich spatial information, resulting in incomplete feature extraction and ultimately low classification accuracy.

[0004] In recent years, several deep learning models have also been applied to hyperspectral image classification. Convolutional neural networks effectively extract features through local connections and significantly reduce parameters by sharing weights. They are widely used in fields such as object recognition and medical image processing. Currently, convolutional neural networks have three different types of convolution kernels: 1D-CNN, 2D-CNN, and 3D-CNN. All use a backpropagation algorithm to update network parameters.

[0005] Most methods using convolutional neural networks adopt classification methods based on joint spatial-spectral features, which can directly extract spectral and spatial information. However, since hyperspectral images have similar textures in many bands, the computational complexity will increase. In addition, due to the limited training samples of hyperspectral datasets, gradient disappearance may occur as the network depth increases, and traditional dimensionality reduction methods do not consider the homogeneity of adjacent pixels. Summary of the Invention

[0006] The purpose of the present invention is to provide a hyperspectral remote sensing image classification method and system based on a convolutional neural residual network combined with a spectral attention mechanism and a spatial attention mechanism to solve at least one technical problem existing in the above background technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In one aspect, the present invention provides a hyperspectral remote sensing image classification method, comprising:

[0009] Perform dimensionality reduction on hyperspectral remote sensing images;

[0010] Extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0011] Extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0012] The optimizer is used to process the image after extracting the spectral key information and spatial key information to obtain the image classification result.

[0013] Preferably, performing dimensionality reduction processing on the hyperspectral remote sensing image includes: using an entropy rate superpixel algorithm to generate superpixels suitable for the boundaries of the hyperspectral image.

[0014] Preferably, in the entropy rate superpixel algorithm, the image is represented in the form of a vertex-edge network, wherein the vertices are composed of pixels and the weights of the edges connecting the vertices are composed of a pair of similarities given in the form of a similarity matrix.

[0015] Preferably, each pixel belongs to one category, and among all feasible edges, the edge that optimizes the objective function is selected and added to the image.

[0016] Preferably, a hybrid convolutional neural network is used, in which two-dimensional convolution is used to learn spatial features of images without losing spectral information, and three-dimensional convolution is used to extract spatial spectral fusion features.

[0017] Preferably, the step size of each convolutional layer in the hybrid convolutional neural network is set to 1, and ReLU is used as the activation function.

[0018] Preferably, the back propagation algorithm for the optimizer is trained by using a classification loss function, and the parameters are updated by back propagation.

[0019] In a second aspect, the present invention provides a hyperspectral remote sensing image classification system, the system comprising:

[0020] Dimensionality reduction module, used to perform dimensionality reduction processing on hyperspectral remote sensing images;

[0021] The extraction module is used to extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; and to extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0022] Attention mechanism module, used to extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0023] The classification module is used to process the image after extracting key information using the optimizer to obtain image classification results.

[0024] In a third aspect, the present invention provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the hyperspectral remote sensing image classification method described above is implemented.

[0025] In a fourth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory so that the electronic device executes instructions for implementing the hyperspectral remote sensing image classification method as described above.

[0026] The beneficial effects of the present invention are as follows: the use of residual connection can effectively reduce the gradient vanishing phenomenon; the combination of spectral and spatial attention mechanisms can extract more complete spatial and spectral features, effectively suppressing the influence of noise; the use of entropy superpixels to preprocess hyperspectral images pays more attention to the relationship between different bands than the traditional principal component analysis method, improves the image classification accuracy, and makes the classification results more accurate and clear.

[0027] Additional aspects and advantages of the present invention will be set forth in part in the following description, will become apparent from the following description, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a flowchart of a hyperspectral remote sensing image classification method using a convolutional neural residual network with an attention mechanism based on superpixel preprocessing as described in an embodiment of the present invention.

[0030] Figure 2 Schematic diagram of comparison of hyperspectral remote sensing image classification results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0032] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0033] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0034] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0035] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise inconsistent.

[0036] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0037] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0038] Example 1

[0039] This embodiment 1 provides a hyperspectral remote sensing image classification system, which includes:

[0040] Dimensionality reduction module, used to perform dimensionality reduction processing on hyperspectral remote sensing images;

[0041] The extraction module is used to extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; and to extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0042] Attention mechanism module, used to extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0043] The classification module is used to process the image after extracting the spectral key information and spatial key information using the optimizer to obtain the image classification result.

[0044] In this embodiment 1, the hyperspectral remote sensing image classification system described above is used to implement a hyperspectral remote sensing image classification method, which includes:

[0045] The dimensionality reduction module is used to reduce the dimensionality of the hyperspectral remote sensing image; then, the extraction module is used to extract the spatial features of the image after dimensionality reduction to obtain a spatial feature map; the spectral features of the spatial feature map are extracted to obtain a spatial-spectral fusion feature map; the attention mechanism module is used to extract spectral key information and spatial key information from the spatial-spectral fusion feature map; finally, the classification module is used to process the image after key information extraction using the optimizer to obtain the image classification result.

[0046] In this first embodiment, dimensionality reduction processing of a hyperspectral remote sensing image includes employing an entropy-based superpixel algorithm to generate superpixels that fit the boundaries of the hyperspectral image. In the entropy-based superpixel algorithm, the image is represented as a vertex-edge network, where vertices are composed of pixels and the weights of edges connecting vertices are composed of pairs of similarities given in the form of a similarity matrix. Each pixel belongs to a class, and from all feasible edges, the edge that optimizes the objective function is selected and added to the image.

[0047] Specifically, first, the image is represented in the form of P = (V, E), where V represents the vertex, which consists of pixels, and E represents the edge weight, which consists of a pair of similarities given in the form of a similarity matrix. The initial condition is that each pixel belongs to a category. Then, among all feasible edges, the edge that optimizes the objective function is selected and added to the graph using the following algorithm: Among them, H(A) represents the entropy rate term based on random walk, which can form uniform and compact clusters, B(A) is the balance term, which can make the cluster sizes similar, and λ is used to balance the weights of the entropy rate term and the balance term.

[0048] In this embodiment 1, a hybrid convolutional neural network is used, in which two-dimensional convolution is used to learn the spatial features of the image without losing spectral information, and three-dimensional convolution is used to extract spatial-spectral fusion features.

[0049] Specifically, a 2D-3D hybrid convolutional neural network was constructed. 2D convolution focuses on learning spatial features from images without losing spectral information, while 3D convolution is used to extract spatial-spectral features and reduce network parameters. The first three convolutional layers use 32 3×3×3, 16 3×3×5, and 8 3×3×7 convolutional blocks, respectively. These blocks are then fed into 64 3×3 convolutional layers, with a stride of 1 in each layer. To improve the model's expressiveness, ReLU is used as the activation function:

[0050] f Relu (x)=max(0,w (i)T x j +b (i) );

[0051] Among them, w (i)T x j +b (i) The linear transformation that the input vector x undergoes in the previous layer of the neural network, and the nonlinear result of the final output depends on the current position of the neuron in the network structure. Simultaneously train multiple sets of parameters and select the largest activation value as the activation value of the next layer. Among them, w represents the weight, x j represents neurons, w (i)T represents the weight of back propagation, b (i) Indicates bias.

[0052] In this embodiment 1, the attention mechanism module incorporates spectral attention mechanism and spatial attention mechanism. Specifically, the dual attention mechanism derives an attention map based on the two independent dimensions of spectral and spatial order, then multiplies the attention map by the input feature map and performs adaptive feature refinement. Given a feature map G∈R H×W×C After the extraction module, the one-dimensional spectral feature map M is obtained spe ∈R 1×1×C and two-dimensional spatial feature map M spa ∈R H×W×1 , where H, W, and C represent height, bandwidth, and number of spectra, respectively. The dual attention mechanism can be expressed as follows:

[0053]

[0054]

[0055] in, represents the multiplication of corresponding elements, g′ represents the output after the spectral attention mechanism, and g″ represents the output after the spatial attention mechanism.

[0056] In this embodiment 1, after the output of the attention mechanism module, it passes through two Dropout layers, which also includes dense connections and ReLU activation functions, and finally obtains the result through the softmax classification layer.

[0057] In this embodiment 1, the back propagation algorithm of the optimizer is trained by using a classification loss function, and the parameters are updated by back propagation. The classification loss function is expressed as follows:

[0058]

[0059] in, and They represent the true value and the predicted value respectively, M represents the total number of mini-batch samples, and L represents the total number of feature coverage classes.

[0060] Example 2

[0061] In this embodiment 2, a hyperspectral remote sensing image classification method based on a convolutional neural residual network with an attention mechanism based on superpixel preprocessing is provided. The residual connection is adopted to effectively reduce the gradient vanishing phenomenon. The dual attention mechanism is adopted to enable the network to more completely extract spatial features and spectral features, and suppress the influence of noise. The entropy rate superpixel is used to preprocess the hyperspectral image, and more emphasis is placed on the relationship between different bands, thereby improving the classification accuracy and making the classification results more accurate and clear.

[0062] In this embodiment 1, the overall architecture of the attention mechanism convolutional neural residual network is as follows: Figure 1 As shown, the image first undergoes dimensionality reduction (PCA), then passes through the ERS (Entropy Rate Superpixel) algorithm, and after another PCA pass, enters the convolutional network (Conv). Each convolutional layer uses a residual normalization (BN) connection between the activation function layer (ReLU). The Flatten layer is used to "flatten" the input, converting multidimensional input into one dimension. It is often used in the transition from convolutional layers to fully connected layers. The Dense layer maps the previously extracted feature associations to the output space.

[0063] The specific steps are as follows:

[0064] First, the hyperspectral image is subjected to dimensionality reduction. This process uses the entropy-rate superpixel algorithm to generate uniformly sized and compact superpixels that fit the hyperspectral image boundaries. Because pixel-level feature extraction is crucial for classification, the differences between different spectral features cannot be ignored. The image is represented as P = (V, E), where V represents the vertex, consisting of pixels, and E represents the edge weight, consisting of a pair of similarities given in the form of a similarity matrix. The initial condition is that each pixel belongs to a class. Then, among all feasible edges, the edge that optimizes the objective function is selected and added to the graph using the following algorithm:

[0065] m A axH(A)+λB(A) s.t.A∈E ;

[0066] Among them, H(A) represents the entropy rate term based on random walk, which can form uniform and compact clusters, B(A) is the balance term, which can make the cluster sizes similar, and λ is used to balance the weights of the entropy rate term and the balance term.

[0067] Construct a 2D-3D hybrid convolutional neural network, such as Figure 1 As shown in the figure, 2D convolution focuses on learning spatial features of the image without losing spectral information. It is combined with 3D convolution to extract spatial-spectral features and reduce network parameters. The first three layers use 32 3×3×3, 16 3×3×5, and 8 3×3×7 convolution blocks, respectively. These are then fed into 64 3×3 convolution layers, with a stride of 1 in each layer.

[0068] In order to improve the expressiveness of the model, ReLU is used as the activation function:

[0069] f Relu (x)=max(0,w (i)T x j +b (i) )

[0070] Among them, w (i)T x j +b (i) The linear transformation that the input vector x of the previous layer of the neural network undergoes, and the nonlinear result of the final output depends on the current position of the neuron in the network structure.

[0071] In this embodiment 2, in order to prevent the gradient from disappearing, a residual connection is added to the convolutional layer, and then the spectral attention mechanism and the spatial attention mechanism are added respectively.

[0072] Finally, after two Dropout layers, which also include dense connections and ReLU activation functions, the result is finally obtained through the softmax classification layer.

[0073] In Example 2, the dual attention mechanism described above generates an attention map based on two independent dimensions: spectral and spatial order. This attention map is then multiplied by the input feature map and adaptively refined. Integrating the attention module into the convolutional network adds negligible overhead while significantly improving the accuracy of image classification and object detection.

[0074] Using the above network, given a feature map G∈R H×W×C , we can get the one-dimensional spectral feature map M spe ∈R 1 ×1×C and two-dimensional spatial feature map M spa ∈R H×W×1 , where H, W, and C represent height, bandwidth, and number of spectra, respectively. The dual attention mechanism can be expressed as follows:

[0075]

[0076]

[0077] in, represents the multiplication of corresponding elements, g′ represents the output after the spectral attention mechanism, and g″ represents the output after the spatial attention mechanism.

[0078] In this embodiment 2, the Adam optimizer with high computational efficiency and low memory requirements is used. The back propagation algorithm of the optimizer is trained by using softmax loss, and the parameters are updated by back propagation. The classification loss function is expressed as follows:

[0079]

[0080] in, and They represent the true value and the predicted value respectively, M represents the total number of mini-batch samples, and L represents the total number of feature coverage classes.

[0081] In this embodiment 2, a large number of experiments were conducted on the Indian Pines dataset and the Xuzhou dataset. The results obtained by the model were compared with the GroundTruth of the dataset, and the classification results were evaluated using three evaluation indicators: OA (overall accuracy), AA (average accuracy), and Kappa coefficient. The Kappa coefficient is an indicator for measuring classification accuracy. It is obtained by multiplying the total number of pixels (N) in all ground truth classifications by the sum of the diagonal of the confusion matrix (Xkk), then subtracting the product of the total number of ground truth pixels in a certain class and the total number of classified pixels in that class, and then dividing it by the square of the total number of pixels minus the product of the total number of ground truth pixels in a certain class and the total number of classified pixels in that class, and then summing the results for all classes.

[0082] In this embodiment 2, the evaluation results are compared with the current classification model. Figure 2 As shown in the figure, (a) is the original hyperspectral remote sensing image, (b) is the classified image using SVM, (c) is the classification result of the 2D-CNN network, (d) is the classification result of the 3D-CNN network, (e) is the classification result of the SSRN network, (f) is the classification result of the DRNN network, (g) is the classification result of the method described in Example 2, and (h) is the real classified annotated image. The hyperspectral remote sensing image method described in Example 2 has higher classification accuracy and better classification effect, and has certain practicality.

[0083] Example 3

[0084] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the hyperspectral remote sensing image classification method described above is implemented. The method includes:

[0085] Perform dimensionality reduction on hyperspectral remote sensing images;

[0086] Extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0087] Extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0088] The optimizer is used to process the image after extracting the spectral key information and spatial key information to obtain the image classification result.

[0089] Example 4

[0090] Embodiment 4 of the present invention provides a computer program (product), including a computer program, wherein when the computer program is run on one or more processors, the computer program is used to implement the hyperspectral remote sensing image classification method described above, the method comprising:

[0091] Perform dimensionality reduction on hyperspectral remote sensing images;

[0092] Extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0093] Extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0094] The optimizer is used to process the image after extracting the spectral key information and spatial key information to obtain the image classification result.

[0095] Example 5

[0096] Embodiment 5 of the present invention provides an electronic device, comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the hyperspectral remote sensing image classification method described above, the method comprising:

[0097] Perform dimensionality reduction on hyperspectral remote sensing images;

[0098] Extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; extract the spectral features of the spatial feature map to obtain a spatial-spectral fusion feature map;

[0099] Extract spectral key information and spatial key information from the spatial-spectral fusion feature map;

[0100] The optimizer is used to process the image after extracting the spectral key information and spatial key information to obtain the image classification result.

[0101] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0105] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. A hyperspectral remote sensing image classification method, characterized in that: include: Perform dimensionality reduction on hyperspectral remote sensing images; Extract the spatial features of the image after dimensionality reduction processing to obtain a spatial feature map; Extract the spectral features of the spatial feature map to obtain the spatial-spectral fusion feature map; adopt a hybrid convolutional neural network, in which two-dimensional convolution is used to learn the spatial features of the image without losing spectral information, and cooperate with three-dimensional convolution to extract spatial-spectral fusion features. Specifically, a 2D-3D hybrid convolutional neural network is constructed. The first three convolutional layers use 32 3×3×3, 16 3×3×5, and 8 3×3×7 convolution blocks, respectively, and then send them to 64 3×3 convolutional layers. The step size in each layer is set to 1; ReLU is used as the activation function: ； in, Represents the input vector of the previous layer of the neural network The linear transformation experienced and the final nonlinear output result depend on the current position of the neuron in the network structure; multiple sets of parameters are trained at the same time, and the one with the largest activation value is selected as the activation value of the next layer; represents the weight, represents neurons, represents the weight of back propagation, Indicates bias; The attention mechanism module adds spectral attention mechanism and spatial attention mechanism respectively. Specifically, the dual attention mechanism obtains the attention map according to the two independent dimensions of spectral and spatial order, then multiplies the attention map by the input feature map and performs adaptive feature refinement. Given a feature map , after the extraction module, we get a one-dimensional spectral feature map and two-dimensional spatial feature maps ,in, 、 、 Represent height, bandwidth, and spectrum quantity respectively; the dual attention mechanism is expressed as follows: in, Indicates the multiplication of corresponding elements, represents the output after the spectral attention mechanism, Represents the output result after the spatial attention mechanism; After the output of the attention mechanism module, it passes through two Dropout layers, which also include dense connections and ReLU activation functions, and finally the result is obtained through the softmax classification layer; Extract spectral key information and spatial key information from the spatial-spectral fusion feature map; The optimizer is used to process the image after extracting the spectral key information and spatial key information to obtain the image classification result.

2. The hyperspectral remote sensing image classification method according to claim 1, characterized in that: The dimensionality reduction processing of the hyperspectral remote sensing image includes: using the entropy rate superpixel algorithm to generate superpixels suitable for the boundaries of the hyperspectral image.

3. The hyperspectral remote sensing image classification method according to claim 2, characterized in that: In the entropy rate superpixel algorithm, the image is represented in the form of a vertex-edge network, where the vertices are composed of pixels and the weights of the edges connecting the vertices are composed of a pair of similarities given in the form of a similarity matrix.

4. The hyperspectral remote sensing image classification method according to claim 3, characterized in that: Each pixel belongs to a class, and among all feasible edges, the edge that optimizes the objective function is selected and added to the image.

5. The hyperspectral remote sensing image classification method according to claim 1, characterized in that: The backpropagation algorithm for the optimizer is trained using the classification loss function, and the parameters are updated through backpropagation.

6. A hyperspectral remote sensing image classification system, characterized in that: include: Dimensionality reduction module, used to perform dimensionality reduction processing on hyperspectral remote sensing images; The extraction module is used to extract the spatial features of the image after dimensionality reduction processing and obtain a spatial feature map; to extract the spectral features of the spatial feature map and obtain a spatial-spectral fusion feature map; a hybrid convolutional neural network is used, in which two-dimensional convolution is used to learn the spatial features of the image without losing spectral information, and three-dimensional convolution is used to extract spatial-spectral fusion features. Specifically, a 2D-3D hybrid convolutional neural network is constructed. The first three convolutional layers use 32 3×3×3, 16 3×3×5, and 8 3×3×7 convolution blocks, respectively, and then send them to 64 3×3 convolutional layers. The step size in each layer is set to 1; ReLU is used as the activation function: ； in, Represents the input vector of the previous layer of the neural network The linear transformation experienced and the final nonlinear output result depend on the current position of the neuron in the network structure; multiple sets of parameters are trained at the same time, and the one with the largest activation value is selected as the activation value of the next layer; represents the weight, represents neurons, represents the weight of back propagation, Indicates bias; The attention mechanism module adds spectral attention mechanism and spatial attention mechanism respectively. Specifically, the dual attention mechanism obtains the attention map according to the two independent dimensions of spectral and spatial order, then multiplies the attention map by the input feature map and performs adaptive feature refinement. Given a feature map , after the extraction module, we get a one-dimensional spectral feature map and two-dimensional spatial feature maps ,in, 、 、 Represent height, bandwidth, and spectrum quantity respectively; the dual attention mechanism is expressed as follows: in, Indicates the multiplication of corresponding elements, represents the output after the spectral attention mechanism, Represents the output result after the spatial attention mechanism; After the output of the attention mechanism module, it passes through two Dropout layers, which also include dense connections and ReLU activation functions, and finally the result is obtained through the softmax classification layer; Attention mechanism module, used to extract spectral key information and spatial key information from the spatial-spectral fusion feature map; The classification module is used to process the image after extracting key information using the optimizer to obtain image classification results.

7. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the hyperspectral remote sensing image classification method according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the hyperspectral remote sensing image classification method according to any one of claims 1 to 5.

Images