Hyperspectral remote sensing image classification method based on dynamic suturable neural network
By dynamically sewing neural networks to connect space and spectral network branches, combined with reinforcement learning strategies, the problem of insufficient generalization ability in hyperspectral image classification is solved, and more efficient feature fusion and recognition effects are achieved.
Patent Information
- Application Number
- CN202510604545.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-12
- Publication Date
- 2025-08-05
AI Technical Summary
In the existing hyperspectral image classification methods, the generalization ability of spatial feature and spectral feature fusion strategy network is limited, and it is difficult to effectively balance the contribution of spatial knowledge and spectral knowledge.
Using a method based on dynamic suturable neural network, the spatial network branches and spectral network branches are connected through suture modules, combined with ViT and 1D CNN network models, the reinforcement learning strategy is used to dynamically select suture locations to generate suture hypernetworks, enhancing the flexibility and generalization capabilities of the network.
It improves the accuracy and generalization ability of hyperspectral image classification, can better identify geographic categories, and improves the adaptability and performance of the overall model.
Smart Images
Figure CN120431402A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of hyperspectral remote sensing image classification and relates to a hyperspectral image classification method. Background Art
[0002] Hyperspectral images contain rich spatial and spectral information, which is of great value for land cover identification. Therefore, they are widely used in various fields such as agriculture, urban planning, and environmental monitoring. Among them, hyperspectral image classification, as a key technology for interpreting hyperspectral images, aims to assign different semantic labels to each pixel in the hyperspectral image, which is a fundamental but challenging task in this field.
[0003] In recent years, with the development of deep learning, hyperspectral image classification methods have made significant progress. Currently, most existing hyperspectral classification methods are based on two main design approaches. The simplest and most direct approach is to design a backbone network with a three-dimensional convolutional structure to directly learn the complete spatial-spectral integrated features of hyperspectral images. However, this approach struggles to balance the contributions of spatial and spectral knowledge in the integrated features. Another main approach is to design two distinct network branches to extract spatial and spectral features from hyperspectral images, respectively, and then design specialized strategies or modules to interact or fuse the learned subdomain features. This approach allows for flexible design of each network branch, thereby balancing the contributions of spatial and spectral information in the overall network decision-making. However, this approach not only requires researchers to invest significant time and effort in designing specific modules or feature interaction mechanisms, but also these feature interaction and fusion modules and strategies struggle to capture the nonlinear relationship between spatial and spectral information in hyperspectral images. Furthermore, the vast majority of existing methods still focus on designing networks with fixed structures and performing full parameter training. Obviously, these fixed-structured networks often have limited generalization capabilities. Summary of the Invention
[0004] The present invention aims to solve the problem that the existing network for fusion strategy of spatial features and spectral features of hyperspectral images has limited generalization ability.
[0005] A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network, comprising:
[0006] The acquired hyperspectral image X is subjected to recognition processing using a stitching network for hyperspectral image recognition to obtain a hyperspectral remote sensing image classification; the stitching network includes a spatial network branch and a spectral network branch, and a stitching module arranged between the spatial network branch and the spectral network branch, the stitching module being used to map the output intermediate feature space of the first half of one network branch to the input intermediate feature space of the second half of another network branch; the network branch to which the intermediate feature space is mapped outputs a recognition result of the hyperspectral remote sensing image classification.
[0007] Furthermore, the stitching module is used to map the output intermediate feature space of the first half of the spatial network branch to the input intermediate feature space of the second half of the spectral network branch.
[0008] Preferably, the spatial network branch and the spectral network branch adopt ViT and 1D CNN network models.
[0009] Furthermore, the 1D CNN network model adopted by the spectral network branch includes a 1D CNN module and an MLP module.
[0010] Preferably, the stitching module adopts linear layers.
[0011] Furthermore, the process of setting the position of the stitching module between the spatial network branch and the spectral network branch includes the following steps:
[0012] Obtaining raw hyperspectral images Where H, W and B represent the height, width and number of channels of HSI respectively. HSI is hyperspectral image. express The samples marked in express The i-th pixel in n s Indicates the size of the data scale, which corresponds to the pixel size, and is actually the size of the label scale; Represents pixels around the center The HSI block with side length p is formed and Indicates all A collection composed of express The corresponding category labels, where C represents the number of categories in HSI;
[0013] L represents the number of network layers in the spatial network branch and the spectral network branch, which is defined as the network depth d; k represents the number of network layers contained in the window, which is defined as the size of the sliding window; s represents the number of network layers that the sliding window crosses at each movement, which is defined as the stride of the window;
[0014] The two network branches before stitching are recorded as the original network. When stitching the two branches, a sliding window is first placed at the starting point of the two network branches. In this sliding window, the stitching module is added to stitch the nth layer of one original network to the n+1th layer of the other original network. After all the network layers in the sliding window are stitched, the sliding window will continue to move s layers toward the end of the network and continue stitching. This process is repeated until the sliding window reaches the end of the network, completing the entire stitching process. For each stitching module, the network corresponding to the first half of the network branch of an original network, the stitching module, and the second half of the network branch of an original network is recorded as a stitched sub-network.
[0015] Set up a stitching position selection network ψ p , the output vector λ=Sigmoid(ψ p (x spa ))={λ1,λ2,...,λ N}, N is the number of stitching sub-networks contained in the stitching network; the HSI block is directly input into the stitching position selection network ψ p In the training process, the HSI block will input each stitching sub-network and the stitching position selection network at the same time, and select the confidence selection element λ. j =max{λ1,λ2,...,λ N} corresponding to the suture sub-network, that is, activate the jth suture sub-network Classify and predict the data; each sutured sub-network and the original spectral network branch have a loss value L i , and get a loss vector L i is the i-th stitching sub-network The measure of accuracy; thus the total loss of the entire model is obtained Through this training method, training iteration is performed, and finally according to the obtained λ j =max{λ1,λ2,...,λ N}Determine the final stitching sub-network, whose corresponding position is the position of the stitching module between the spatial network branch and the spectral network branch.
[0016] Preferably, during the sliding window stitching process, s≥k-1, s+k≤d needs to be satisfied.
[0017] Furthermore, the suture position selection network ψ p Use 3D CNN network.
[0018] Furthermore, the 3D CNN network used in the stitching position selection network includes a 3D convolution unit, two 2D residual units and a 2D convolution layer. The feature input obtained by the 2D convolution layer is a sigmoid function that outputs the final decision vector λ; the 3D convolution unit includes a 3D convolution layer and an activation function layer; the first 2D residual unit includes a 2D convolution layer and an activation function layer; the second 2D residual unit includes a 2D convolution layer, an activation function layer and a batch normalization layer.
[0019] Furthermore, the loss value L between each stitched sub-network and the original spectral network branch is i It is calculated using cross entropy loss.
[0020] The hyperspectral image classification algorithm proposed in this paper includes a spatial-spectral network branch stitching module. This module stitches together the advantageous parts of two network branches by inserting multiple stitching layers to generate new stitched networks with better performance. This not only enables flexible design of network branches for learning both spatial and spectral information, but also saves researchers time and effort in developing new modules. During the training phase, the present invention devises a reinforcement learning-based training strategy, introducing an additional stitching position selection network in addition to the stitching architecture. This not only helps select the most appropriate stitching sub-network for specific data but also makes the network structure flexible and versatile, greatly enhancing the adaptability and generalization of the overall method. Furthermore, based on the advantages of different network structures, the present invention selects a ViT network and a 1D CNN network for learning spatial and spectral information. A small 3D CNN network is selected as the stitching position selection network to iterate the overall model training to obtain optimal parameters. Experimental analysis shows that the proposed method can achieve OA values of 0.8721, 0.9508, and 0.9513 on three standard datasets, respectively. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 Training flowchart for the hyperspectral image light classification method based on a dynamic stitchable neural network.
[0022] Figure 2 Flowchart for testing a light classification method for hyperspectral images based on a dynamically stitchable neural network.
[0023] Figure 3 Schematic diagram of the workflow of the spatial-spectral network branch stitching module.
[0024] Figure 4 Provides a structural diagram of the components in the network architecture.
[0025] Figure 5 Schematic diagram of the stitching process of the ViT network and the 1DCNN network.
[0026] Figure 6 These are the hyperspectral pseudo-color images, training set images, test set images, and classification result images of dataset I.
[0027] Figure 7 These are the hyperspectral pseudo-color images, training set images, test set images, and classification result images of dataset II.
[0028] Figure 8 These are the hyperspectral pseudo-color images, training set images, test set images, and classification result images of dataset III. DETAILED DESCRIPTION
[0029] The present invention proposes a hyperspectral image classification method based on a dynamic stitchable neural network, which can insert multiple stitching layers between two network branches that learn spatial and spectral knowledge to generate multiple stitched new networks. This method can not only flexibly design two network branches, but also directly obtain spatial-spectral comprehensive features using the obtained stitched network. At the same time, the present invention also uses a reinforcement learning strategy, which allows the model to dynamically select the most appropriate stitching position to obtain a stitched new network based on the information of the data itself, which further enhances the generalization performance of the network, enabling the network to better learn the knowledge in different hyperspectral image blocks and be able to more accurately identify the types of land objects. This will be explained below in conjunction with specific implementation methods.
[0030] Specific implementation method 1: Combination Figure 1 To explain this embodiment,
[0031] The hyperspectral remote sensing image classification method based on a dynamic stitchable neural network described in this embodiment includes:
[0032] S1. Design a spatial-spectral network branch stitching module and use it to insert multiple stitching layers at different locations between the spatial and spectral network branches to generate a stitched supernetwork. In the stitched supernetwork, each newly generated stitched subnetwork incorporates the advantages of both original network branches.
[0033] S2. Design a reinforcement learning-based training strategy that uses a stitching location selection network to dynamically select appropriate stitching locations for the input data. This approach allows the network structure (i.e., stitching locations) to flexibly adjust based on the input data, significantly improving the overall adaptability and generalization of the entire network architecture in this method.
[0034] The present invention takes advantage of the fact that "the shallow layer of the ViT network can effectively encapsulate global spatial knowledge, while the deep layer of the 1D CNN network can capture complex spectral information knowledge". It uses the spatial-spectral network branch stitching module to stitch the ViT network and the 1D CNN network together, and uses a small 3D CNN network as the stitching position selection network to perform reinforcement learning iterative training on the overall model to obtain the optimal model parameters.
[0035] S1. Design a spatial-spectral network branch stitching module and stitch the spatial network branch and the spectral network branch:
[0036] Obtaining raw hyperspectral imagery (HSI) Where H, W and B represent the height, width and number of channels of HSI respectively; express The samples marked in express The i-th pixel (spectral vector) in X spec Indicates all The set of components, n s Indicates the size of the data scale, which corresponds to the pixel size and is actually the size of the label scale ( Indicates the number of data blocks, these three values are equal); use Represents pixels around the center The hyperspectral image block with a side length of p is formed and Indicates all A collection composed of express The corresponding category labels, where C represents the number of categories in HSI.
[0037] The processing of the spatial-spectral network branch stitching module includes:
[0038] The spatial network branch and the spectral network branch are represented as T φ and H θ Now each network is divided into L layers, where the i-th layer function of each network can be expressed as f t,φ,i and f h,θ,i , then the overall network can be expressed as:
[0039]
[0040] Among them, φ and θ represent the parameters of the two networks respectively, Indicates the natural connection between network layers (there is no structure in the middle of this network layer connection, that is, a symbol is arbitrarily defined to represent the sequential processing between networks), and L indicates the depth of the network. φ,1|L :X spa →Ys Represents network T φ The input data x spa From the input space X spa Mapping to output space Y s , that is, the space of output category labels; use H θ,1|L :X spec →Y s Represents network H θ The input data x spec From the input space X spec Mapping to output space Y s .
[0041] To stitch these two networks together, we first need to use a numbering method to fix the stitching position. Use two integer serial parameters m,n∈[1,L] to separate the two network branches from the middle, forming four neural network parts: and in and They represent the intermediate feature dimension space where the two networks are separated, also known as the intermediate feature activation space (this part expresses that the two networks are separated from the middle, and there will be intermediate feature outputs where the two networks are separated. The feature space where the intermediate features are located is the intermediate feature activation space). Now suppose there is a stitching network layer S m,n (This stitching layer structure can be any network structure, such as a linear layer or convolution. In this embodiment, the linear layer is used when stitching ViT and 1DCNN later.) The intermediate features can be output from the intermediate feature space of the first half of the first network. Mapped to the input intermediate feature space of the second half of the second network The entire data can be input into the beginning of the first network, pass through the first half of the first network, pass through the stitching layer in the middle, and then output from the second half of the second network. This is equivalent to generating a new network with the first half of the first network, the stitching layer in the middle, and the second half of the second network. This means that the stitching layer S can be used to m,n T φ,1|m With H θ,n+1|L The two network parts are stitched together to generate a stitched new network (after the network is stitched, the data is input from the first network, passed through the first half, passed through the stitching layer and output from the second half of the second network). This process can be expressed as:
[0042]
[0043] By controlling the two parameters m and n, multiple stitching layers can be inserted simultaneously at different positions between the two networks to generate a stitched supernetwork (all mapped from the first neural network to the second neural network). Assuming that N stitching layers are inserted in this process, the resulting supernetwork will contain N newly generated stitched subnetworks consisting of the first half of the first network, the corresponding stitching layers, and the second half of the second network (a new network composed of the first half of the first network, the stitching layers, and the second half of the second network). This process can be expressed as:
[0044]
[0045] Among them, SupNet(X) represents the overall generated stitched super network, Snnet represents the network stitching module, T and H are the aforementioned spatial network branch and spectral network branch; represents the newly generated i-th stitched sub-network in the super-network.
[0046] S2. Design a method to select the insertion position of the stitching layer in the spatial-spectral network branch stitching module.
[0047] Specifically, the spatial-spectral network branch stitching module uses a sliding window method to select the location to insert the stitching layer between the two networks.
[0048] Define the parameters of the sliding window: L represents the number of artificially divided network layers, which is defined as the network depth d (d = L); k represents the number of network layers included in the window, which is defined as the size of the sliding window; s represents the number of network layers that the sliding window crosses each time it moves, which is defined as the stride of the window.
[0049] In the process of stitching two branches, a sliding window is first placed at the starting point of the two networks. In this sliding window, T φ and H θ Specifically, the nth layer of an original network is stitched to the n+1th layer of another original network, such as Figure 1 As shown, this is equivalent to inserting k-1 stitching layers into each sliding window. Once all network layers within the sliding window are stitched (here, all network layers contained within the sliding window are stitched with the nth layer and the n+1th layer of another network), the sliding window will continue to move s layers toward the end of the network and continue stitching the newly entered network layers. This process is repeated until the sliding window reaches the end of the network, completing the entire stitching process. In order to prevent repeated insertion of stitching layers and at the same time ensure that the sliding window does not exceed the end of the network, it is obvious that the parameters d, k, and s must satisfy the following constraints:
[0050] s≥k-1,s+k≤d(4)
[0051] It should be noted that sliding windows can actually overlap more than one network layer. The above constraints effectively prevent duplicate stitching, thereby saving computing power and cost, and reducing hardware requirements. Furthermore, users can choose the stride size based on hardware conditions. If hardware conditions are limited, a larger stride can be chosen for sliding.
[0052] To better illustrate the overall stitching process, we take the parameter set d = 9, k = 3, and s = 3 as an example and use the spatial-spectral network branch stitching module to learn the spatial information network T. φ and for learning spectral information network H θ The whole suture process is as follows Figure 3 Obviously, during this process, some stitching layers are inserted and 6 new stitching sub-networks are generated. The overall process can be expressed as:
[0053]
[0054] from Figure 3 As can be seen in Figure 3, the complexity of the stitched hypernetwork can be adjusted by controlling the total depth d, window size k, and stride s. Obviously, when the values of d and k increase and the value of s decreases, the structure of the stitched hypernetwork becomes more complex and contains more newly generated stitched networks.
[0055] Then, a reinforcement learning-based training strategy is used to enable the overall network to dynamically select the appropriate suture position based on the data.
[0056] In order to select the most appropriate suture position based on specific input data, the present invention designs a training strategy based on reinforcement learning. Specifically, the present invention introduces an additional suture position selection network outside the suture supernetwork framework (this network structure is arbitrary, that is, a separate network model, which only needs to ensure that the output size matches the number of suture subnetworks, that is, the number of inserted suture layers matches. In the following embodiments, the suture position selection network selects a three-dimensional convolutional structure) to help specific data select the subnetwork generated by the specific suture position in the suture supernetwork. In order to improve the accuracy of the suture position selection network decision-making, the entire method adopts a reinforcement learning strategy during the training process.
[0057] During model inference, the HSI block is first directly input into the stitching position selection network ψ p , and the data is finally output through a sigmoid function. The final vector λ is the decision vector used to select the stitching sub-network.
[0058] The dimension of the output decision vector λ is set according to the number of subnetworks contained in the stitched supernetwork to match the number of newly generated stitched subnetworks (i.e., the number of stitched networks). Therefore, each element of the decision vector corresponds to a newly generated stitched network. Assuming that the stitched supernetwork contains N generated stitched subnetworks, the process can be expressed as:
[0059] λ=Sigmoid(ψ p (x spa ))={λ1,λ2,…,λ N}(6)
[0060] Each element in λ has a value between 0 and 1, and these values can be viewed as the confidence level of the corresponding network selected.
[0061] Since each batch of data can only pass through one stitching sub-network in the inference process, the confidence selection element λ should be selected in the decision vector j =max{λ1,λ2,…,λ N}(j∈[1,N]) corresponds to the (highest confidence) suture sub-network (during the training process, data will be input into each generated suture sub-network and suture position selection network at the same time. The dimension of the suture position selection network output is determined according to the number of sub-networks. Each element in the decision vector output by the suture position selection network is the confidence of the corresponding original suture sub-network), that is, the jth suture sub-network is activated Make classification predictions on the data.
[0062] In order to ensure that the decision of the stitching position selection network is more accurate, the present invention uses a reinforcement learning technique to train the overall network architecture. Specifically, during the training process, the original HSI block is simultaneously input into the stitching position selection network and each stitching sub-network. Since each stitching sub-network has an original spectral network branch at the end (the original network represents the network before stitching), H θ This allows the model to obtain a loss vector L in addition to the decision vector λ, which is composed of the loss values output by each stitching sub-network. This process can be expressed as:
[0063]
[0064] The smaller the value of the element Li in the loss vector L, the more accurate the prediction of the corresponding i-th network. Therefore, L i can be considered as the i-th stitching subnetwork A measure of precision.
[0065] In order to improve the performance of all new stitching networks and also improve the decision-making ability of the policy network (stitching location selection network), we define the total loss of the entire model as the weighted sum of the network’s accuracy and confidence, expressed as:
[0066]
[0067] With this training approach, during backpropagation, the model tends to reduce the values of all elements in the loss vector L, allowing each stitching sub-network to be fine-tuned. At the same time, the model tends to increase the elements (confidence values) of the decision vector corresponding to networks with smaller loss values (better performance), thereby improving the accuracy of the decisions made by the stitching location selection network.
[0068] S3. Select the appropriate network structure according to the characteristics of each part and perform training iterations.
[0069] Considering that the ViT network can better capture global information, while the 1D CNN network focuses more on one-dimensional detail information, this paper selects the ViT network as the T φ The network learns spatial information and selects 1D CNN network as H θ The network learns spectral information. After pre-training, the two networks are directly divided into d=6 layers, and the spatial-spectral network branch stitching module is used to stitch the two networks together. Since the role of the stitching layer in the overall model is only to transfer features from one activation space to another, its structure and initialization method should be as simple as possible. In practical applications, the present invention uses the simplest linear layer as the stitching layer and uses the default Kaimiing method for initialization. The ViT network extracts and records spatial knowledge by patchembedding and attention modules; the 1D CNN network contains a 1D CNN module and an MLP module for extracting and recording spectral knowledge. The structures of the above network components are as follows Figure 4 As shown, Figure 4 This is a structural diagram of the component modules of different networks in the network architecture.
[0070] The 3D CNN network used in the stitching position selection network includes a 3D convolution unit, two 2D residual units and a 2D convolution layer. The feature input obtained by the 2D convolution layer is a sigmoid function that outputs the final decision vector λ; the 3D convolution unit includes a 3D convolution layer and an activation function layer; the first 2D residual unit includes a 2D convolution layer and an activation function layer; the second 2D residual unit includes a 2D convolution layer, an activation function layer and a batch normalization layer.
[0071] Obviously, in this case, when the sliding window parameters are set to k = 2, s = 1, the number of stitching layers inserted between the two networks is the largest. At this time, five stitching layers are inserted between the two networks and five new stitching sub-networks are generated, such as Figure 5 As shown, this process can be expressed as:
[0072]
[0073] Since the hyperspectral dataset is small in scale and has great similarity, in order to improve the overall generalization ability of the model, the present invention uses two data enhancement techniques, mixup and cutmix, to preprocess the original data.
[0074] Considering that the 3D CNN structure can fully learn the spectral-spatial information in HSI, the present invention adopts a small five-output 3D CNN network as the stitching position selection network ψ p To learn the relevant knowledge of spectral-spatial information and generate the decision vector λ vc , the process can be expressed as:
[0075] λ vc =Sigmoid(3DCNN(x spa ))={λ1,λ2,λ3,λ4,λ5}(10)
[0076] At the same time, during the training process, the original data is directly input into the spectral space stitching hypernetwork to obtain the loss vector L vc , which can be expressed as:
[0077]
[0078] Each element in the loss vector is calculated using the cross entropy loss of the corresponding stitched network tail (which is also the tail of the original 1D CNN network), and the calculation formula is as follows:
[0079]
[0080] In order to improve the performance of the 3D CNN stitching position selection network and fine-tune each newly stitched network, the present invention weights the accuracy of the loss vector and the confidence value of the decision vector as the total loss of the network. The calculation formula is as follows:
[0081]
[0082] Among them, y ic represents the true label corresponding to the data, and Represents the predicted label of the cth class output by the i-th stitching sub-network.
[0083] The overall reasoning process of the model is as follows Figure 2 As shown in FIG, first, the test data is input into the suture position selection network to obtain the decision vector of the test phase. Then, according to the decision vector, the suture position to be passed is selected to activate the new suture network corresponding to the element with the highest value in the decision vector. Finally, the original test data is input into the activated suture network, and the result is output as the final classification result. The classification result of the present invention is shown in FIG. Figure 6-Figure 8 shown. Figure 6 These are the hyperspectral pseudo-color images, training set images, test set images, and classification result images of dataset I Houston 2013 (Houston2013); (a) is the pseudo-color image, (b) is the training set image, (c) is the test set image, and (d) is the classification result image. Figure 7 The hyperspectral pseudo-color image, training set image, test set image and classification result image of Dataset II University of Pavia (PaviaU) dataset; (a) is the pseudo-color image, (b) is the training set image, (c) is the test set image, and (d) is the classification result image. Figure 8 These are the hyperspectral pseudo-color images, training set images, test set images, and classification result images of the Yancheng (YC) dataset III; (a) is the pseudo-color image, (b) is the training set image, (c) is the test set image, and (d) is the classification result image.
[0084] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.
Claims
1. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network, characterized in that: include: The obtained hyperspectral image X is processed by using a stitching network for hyperspectral image recognition to obtain a hyperspectral remote sensing image classification; the stitching network includes a spatial network branch and a spectral network branch, and a stitching module provided between the spatial network branch and the spectral network branch, the stitching module being used to map the output intermediate feature space of the first half of one network branch to the input intermediate feature space of the second half of another network branch; The network branch to which the intermediate feature space is mapped outputs the recognition result of hyperspectral remote sensing image classification.
2. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 1, characterized in that: The stitching module is used to map the output intermediate feature space of the first half of the spatial network branch to the input intermediate feature space of the second half of the spectral network branch.
3. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 1, characterized in that: The spatial network branch and spectral network branch use ViT and 1D CNN network models.
4. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 3, characterized in that: The 1D CNN network model used by the spectral network branch includes a 1D CNN module and an MLP module.
5. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 1, characterized in that: The stitching module uses linear layers.
6. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to any one of claims 1 to 4, characterized in that: The process of setting the position of the stitching module between the spatial network branch and the spectral network branch includes the following steps: Obtaining raw hyperspectral images Where H, W and B represent the height, width and number of channels of HSI respectively. HSI is hyperspectral image. express The samples marked in express The i-th pixel in n s Indicates the size of the data scale, which corresponds to the pixel size, and is actually the size of the label scale; Represents pixels around the center The HSI block with side length p is formed and Indicates all A collection composed of express The corresponding category labels, where C represents the number of categories in HSI; L represents the number of network layers in the spatial network branch and the spectral network branch, which is defined as the network depth d; k represents the number of network layers contained in the window, which is defined as the size of the sliding window; s represents the number of network layers that the sliding window crosses at each movement, which is defined as the stride of the window; The two network branches before stitching are recorded as the original network. When stitching the two branches, a sliding window is first placed at the starting point of the two network branches. In this sliding window, the stitching module is added to stitch the nth layer of one original network to the n+1th layer of the other original network. After all the network layers in the sliding window are stitched, the sliding window will continue to move s layers toward the end of the network and continue stitching. This process is repeated until the sliding window reaches the end of the network, completing the entire stitching process. For each stitching module, the network corresponding to the first half of the network branch of an original network, the stitching module, and the second half of the network branch of an original network is recorded as a stitched sub-network. Set up a stitching position selection network ψ p , the output vector λ=Sigmoid(ψ p (x spa ))={λ1,λ2,...,λ N }, N is the number of stitching sub-networks contained in the stitching network; the HSI block is directly input into the stitching position selection network ψ p In the training process, the HSI block will input each stitching sub-network and the stitching position selection network at the same time, and select the confidence selection element λ. j =max{λ1,λ2,…,λ N } corresponding to the suture sub-network, that is, activate the jth suture sub-network Classify and predict the data; each sutured sub-network and the original spectral network branch have a loss value L i , and get a loss vector L i is the i-th stitching sub-network F i S (x spa ) accuracy; and then the total loss of the entire model is obtained Through this training method, training iteration is performed, and finally according to the obtained λ j =max{λ1,λ2,...,λ N }Determine the final stitching sub-network, whose corresponding position is the position of the stitching module between the spatial network branch and the spectral network branch.
7. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 6, characterized in that: During the sliding window stitching process, it is necessary to satisfy s≥k-1,s+k≤d.
8. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 6, characterized in that: The suture position selection network ψ p Use 3D CNN network.
9. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 8, characterized in that: The 3D CNN network used in the stitching position selection network includes a 3D convolution unit, two 2D residual units and a 2D convolution layer. The feature input obtained by the 2D convolution layer is a sigmoid function that outputs the final decision vector λ; the 3D convolution unit includes a 3D convolution layer and an activation function layer; the first 2D residual unit includes a 2D convolution layer and an activation function layer; the second 2D residual unit includes a 2D convolution layer, an activation function layer and a batch normalization layer.
10. A hyperspectral remote sensing image classification method based on a dynamic stitchable neural network according to claim 6, characterized in that: The loss value L of each stitched sub-network and the original spectral network branch i It is calculated using cross entropy loss.