Mask and contrast learning-based hyperspectral image classification method, medium and equipment

Abstract

The invention discloses a mask and contrast learning-based hyperspectral image classification method, medium and equipment, and relates to the field of hyperspectral image classification, and the mask and contrast learning-based hyperspectral image classification method mainly comprises the steps of preprocessing a hyperspectral image to obtain a preprocessed hyperspectral image; constructing a two-way selective large-kernel spatial-spectral convolutional network, and pre-training the two-way selective large-kernel spatial-spectral convolutional network by using the preprocessed hyperspectral image to obtain a pre-trained two-way selective large-kernel spatial-spectral convolutional network; obtaining a hyperspectral image classification network according to the cross attention module and a pre-trained dual-path selective large-kernel spatial spectrum convolutional network; and classifying a to-be-classified hyperspectral image by using the hyperspectral image classification network to obtain a classification result. By implementing the hyperspectral image classification method, medium and equipment based on mask and contrast learning provided by the invention, the classification precision of the hyperspectral image under the condition of small samples can be improved.

Description

Technical Field

[0001] The present invention relates to the field of hyperspectral image classification, and more specifically to a hyperspectral image classification method, medium and device based on mask and contrast learning. Background Art

[0002] Currently, methods based on self-supervised learning have made important progress in solving the problem of scarce labeled samples in hyperspectral image classification tasks. In recent years, mask modeling and contrastive learning have received widespread attention due to their superior performance.

[0003] The mask modeling method improves the model's ability to extract global semantic information of the image by randomly masking part of the image and reconstructing it. In the hyperspectral image classification task, this method mainly uses local pixel blocks as input, and the category of the pixel block is determined by its center pixel. Based on this feature, Jia et al. (Jia S, Wang Y, Jiang S, et al. A Center-masked Transformer for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024.) improved the original mask modeling method and proposed a center mask strategy. By masking the center pixel of the pixel block, the model's attention to the center pixel is increased, thereby improving the accuracy of classification. However, due to the local similarity of hyperspectral images, adjacent pixels have high similarity, and the masked part of the center can be easily restored through its neighboring area.

[0004] Contrastive learning methods construct positive and negative sample pairs to bring similar data instances closer together and dissimilar data farther apart in the representation space. This method improves the separability of data without using labels. Positive sample pairs are usually generated by applying data augmentation operations such as random cropping, rotation, color jitter, and Gaussian blur to the same data sample, while data pairs from different samples are considered negative samples. In hyperspectral images, pixel blocks have small sizes, and the effect achieved by data augmentation is limited.

[0005] To solve this problem, Guan et al. introduced a comparative learning method between spatial and spectral information in the literature (Guan P, Lam E Y. Cross-domain contrastive learning for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-13.). The network compares the central pixel of the pixel block with a single band and uses a mask to enhance the data. This method allows the model to pay more attention to the information shared between spatial and spectral information, reduce the attention to redundant information, and thus improve the separability of features.

[0006] The above methods have effectively improved the accuracy of hyperspectral image classification under small sample sizes. However, due to the local similarity of hyperspectral images, adjacent pixels have high similarity, and the central masked part can be easily restored through its neighboring area. In addition, using only the information of a single band cannot fully exploit the potential of the rich band information in hyperspectral images. Summary of the invention

[0007] The purpose of the present invention is to provide a hyperspectral image classification method, medium and device based on mask and contrast learning, which can improve the classification accuracy of hyperspectral images in small sample conditions.

[0008] The present invention provides a hyperspectral image classification method based on mask and contrast learning, comprising the following steps: S1: acquiring a hyperspectral image, preprocessing the hyperspectral image, and obtaining a preprocessed hyperspectral image; S2: constructing a dual-path selective large-kernel spatial-spectral convolutional network, which is used to extract spatial features and spectral features of the hyperspectral image; S3: using the preprocessed hyperspectral image, pre-training the dual-path selective large-kernel spatial-spectral convolutional network, and obtaining a pre-trained dual-path selective large-kernel spatial-spectral convolutional network; S4: constructing a cross-attention module, and obtaining a hyperspectral image classification network according to the cross-attention module and the pre-trained dual-path selective large-kernel spatial-spectral convolutional network; S5: using the hyperspectral image classification network to classify the hyperspectral image to be classified, and obtaining a classification result.

[0009] Further, step S1 specifically includes: obtaining a hyperspectral image, using a principal component analysis method to reduce the dimension of the band of the hyperspectral image and divide it into pixel blocks of a fixed size, to obtain a preprocessed hyperspectral image, the preprocessed hyperspectral image includes a spatial part and a spectral part, such as the formula: , , in, Represents the first The center pixel of the pixel block, Represents the first The center pixel of the pixel block is set to 0. Indicates the length of the pixel block, Indicates the width of the pixel block, Indicates the number of channels of the pixel block, represents the central pixel set, Indicates the set of parts where the central pixel is set to 0.

[0010] Furthermore, the dual-path selective large-kernel spatial-spectral convolutional network comprises a spatial branch, a spectral branch and a multi-layer perceptron; the output end of the spatial branch and the output end of the spectral branch are connected to the input end of the multi-layer perceptron after matrix multiplication; the spatial branch comprises a first feature extraction module, a second feature extraction module, a third feature extraction module and an adaptive pooling layer connected in series in sequence; the first feature extraction module comprises a first feature extraction module connected in series in sequence Convolutional layer, first GELU activation function layer, selective large kernel convolutional layer, second Convolutional layer, first addition operator and feed-forward neural network layer, first The input end of the convolutional layer is also connected to the other input end of the first addition operator; the structure of the second feature extraction module and the structure of the third feature extraction module are the same as the structure of the first feature extraction module; the structure of the spectral branch is the same as the structure of the spatial branch.

[0011] Furthermore, the selective large kernel convolution layer includes a first Convolutional layer, Dilated convolution layer, concatenation layer, global pooling layer, average pooling layer, Large kernel convolution layer, Sigmoid activation function, attention selection layer; The expansion rate of the dilated convolutional layer is 2; the first The input of the convolutional layer and The input connection of the dilated convolutional layer is first The output of the convolutional layer and The output of the dilated convolutional layer is connected to the two inputs of the concatenation layer, the output of the concatenation layer is connected to the input of the global pooling layer and the input of the average pooling layer, and the output of the global pooling layer and the output of the average pooling layer are connected to The input connection of the large kernel convolution layer, The output of the large kernel convolution layer is connected to the input of the Sigmoid activation function, and the output of the Sigmoid activation function is connected to the input of the attention selection layer. The output of the convolutional layer and The output of the dilated convolutional layer is connected to the input of the attention selection layer.

[0012] Furthermore, the feedforward neural network layer includes a second Convolutional layer, second GELU activation function layer and maximum pooling layer, the pooling kernel of the maximum pooling layer is 2.

[0013] Furthermore, step S3 specifically includes: S31: obtaining a mask image by using a similarity calculation method according to the preprocessed hyperspectral image; S32: constructing a total loss function, such as the formula: , , , , Among them, L is the total loss function, , , , represents four learnable parameters that sum to 1, is the contrast loss, is the mean square error loss, n represents the number of samples, Indicates mask image, Indicates The mask image is reconstructed by a dual-path selective large kernel spatial spectral convolutional network. Represents a function that measures the distance between two samples. Here, cosine similarity is used. Represents the spectral characteristics, Represents spatial features, represents the temperature coefficient, Represents a layer of linear neural network; T represents matrix transpose; S33: Using the total loss function and the mask image, the two-way selective large kernel spatial spectral convolutional network is pre-trained to obtain a pre-trained two-way selective large kernel spatial spectral convolutional network.

[0014] Further, step S31 specifically includes: S311: according to the pre-processed hyperspectral image, obtain the similarity between each pixel in the pixel block and the central pixel, such as formula: , , , in, is the similarity between each pixel in the pixel block and the central pixel, is the fusion factor, is the cosine similarity between each pixel in the pixel block and the central pixel, is the spatial distance between each pixel in the pixel block and the central pixel, Indicates The first pixel in the pixels, Indicates The center pixel of the pixel block, , Respectively represent the pixel blocks The horizontal and vertical coordinates of pixels, , Respectively represent the horizontal coordinate and vertical coordinate of the central pixel; S312: according to the similarity between each pixel in the pixel block and the central pixel and the pre-processed hyperspectral image, the pixel blocks in the pixel block are sorted, and the pixels in the preset sorting are selected for masking operation to obtain a mask image.

[0015] Furthermore, step S4 specifically includes: S41: Construct a cross-attention module, as shown in the formula: , , in, The final output feature is obtained by multiplying the attention matrix with the spatial feature and pooling it, which is the output of the cross attention module. Represents a global pooling operation; Represents the spectral characteristics, Represents spatial features, The attention matrix is ​​obtained by calculating the attention weight for each feature of the spectral feature and the spatial feature. express Activation function; S42: Based on the cross attention module and the pre-trained dual-path selective large kernel spatial-spectral convolutional network, a hyperspectral image classification network is obtained.

[0016] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned hyperspectral image classification method based on mask and contrast learning are implemented.

[0017] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the above-mentioned hyperspectral image classification method based on mask and contrast learning are implemented.

[0018] The hyperspectral image classification method, medium and device based on mask and contrast learning provided by the present invention have the following beneficial effects: The present invention extracts the spatial and spectral features of hyperspectral images by designing a dual-path selective large-core space-spectrum convolutional network; designs two pre-training tasks of center-guided mask reconstruction and multi-level contrastive learning, and performs pre-training without using label information; designs a cross-attention module to fuse the obtained spatial and spectral features and classify the hyperspectral image to obtain the classification result of the whole image; wherein, the dual-path selective large-core space-spectrum convolutional network can comprehensively utilize the spatial and spectral features of the hyperspectral image to improve the classification accuracy of the model; designs two pre-training tasks of center-guided mask reconstruction and multi-level contrastive learning to strengthen the interaction of spatial and spectral features, as well as the attention to the central pixel, and improve the robustness of the model; and better classifies the hyperspectral image by combining the two methods and introducing center-guided mask reconstruction and multi-level contrastive learning tasks. The present invention solves the current problems of difficulty in labeling hyperspectral image data, scarcity of labeled samples, and limited performance of deep learning models, and improves the classification accuracy of hyperspectral images in small sample conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which: Figure 1 It is a flow chart of a hyperspectral image classification method based on mask and contrast learning provided by the present invention; Figure 2 It is a schematic diagram of a dual-path selective large-kernel spatial-spectral convolutional network provided by the present invention; Figure 3 It is a structural block diagram of the computer device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0021] Figure 1 A schematic diagram of a hyperspectral image classification method based on mask and contrast learning of this embodiment is shown. In this embodiment, the hyperspectral image classification method based on mask and contrast learning includes the following steps: S1: Acquire a hyperspectral image, preprocess the hyperspectral image, and obtain a preprocessed hyperspectral image; In an exemplary embodiment, step S1 specifically includes: obtaining a hyperspectral image, using a principal component analysis method to reduce the dimension of the band of the hyperspectral image and divide it into pixel blocks of fixed size to obtain a preprocessed hyperspectral image, wherein the preprocessed hyperspectral image includes a spatial part and a spectral part, as shown in the formula: , , in, Represents the first The center pixel of the pixel block, Represents the first The center pixel of the pixel block is set to 0. Indicates the length of the pixel block, Indicates the width of the pixel block, Indicates the number of channels of the pixel block, represents the central pixel set, Indicates the set of parts where the central pixel is set to 0; For an exemplary embodiment, in step S1, a hyperspectral image is collected, dimension reduction is performed using principal component analysis (PCA), and channel normalization is performed for preprocessing, and then the image is divided into regular pixel blocks, and the labels of the pixel blocks are determined by their central pixels. Specifically, principal component analysis (PCA) is used to reduce the dimension of the bands of the hyperspectral image and divide them into pixel blocks of fixed size; the central pixel set is taken out. And the part where the center pixel is set to 0 Parts are used as spectral input and spatial input respectively, where Indicates the length of the pixel block, Indicates the width of the pixel block, Indicates the number of channels of the pixel block; S2: Construct a dual-path selective large kernel spatial-spectral convolutional network, which is used to extract the spatial and spectral features of hyperspectral images; In an exemplary embodiment, the dual-path selective large kernel space-spectrum convolution network includes a spatial branch, a spectral branch and a multi-layer perceptron; the output end of the spatial branch and the output end of the spectral branch are connected to the input end of the multi-layer perceptron after matrix multiplication; the spatial branch includes a first feature extraction module, a second feature extraction module, a third feature extraction module and an adaptive pooling layer connected in series in sequence; the first feature extraction module includes a first feature extraction module connected in series in sequence Convolutional layer, first GELU activation function layer, selective large kernel convolutional layer, second Convolutional layer, first addition operator and feed-forward neural network layer, first The input end of the convolutional layer is also connected to the other input end of the first addition operator; the structure of the second feature extraction module and the structure of the third feature extraction module are the same as the structure of the first feature extraction module; the structure of the spectral branch is the same as the structure of the spatial branch; In an exemplary embodiment, the selective large kernel convolution layer includes a first Convolutional layer, Dilated convolution layer, concatenation layer, global pooling layer, average pooling layer, Large kernel convolution layer, Sigmoid activation function, attention selection layer; The expansion rate of the dilated convolutional layer is 2; the first The input of the convolutional layer and The input connection of the dilated convolutional layer is first The output of the convolutional layer and The output of the dilated convolutional layer is connected to the two inputs of the concatenation layer, the output of the concatenation layer is connected to the input of the global pooling layer and the input of the average pooling layer, and the output of the global pooling layer and the output of the average pooling layer are connected to The input connection of the large kernel convolution layer, The output of the large kernel convolution layer is connected to the input of the Sigmoid activation function, and the output of the Sigmoid activation function is connected to the input of the attention selection layer. The output of the convolutional layer and The output of the dilated convolutional layer is connected to the input of the attention selection layer; In an exemplary embodiment, the feedforward neural network layer includes a second Convolutional layer, second GELU activation function layer and maximum pooling layer, the pooling kernel of the maximum pooling layer is 2; For an exemplary embodiment, in step S2, a dual-path selective large-kernel space-spectrum convolutional network for spatial and spectral information is designed to extract features from the hyperspectral image to obtain spatial and spectral features; specifically, symmetrical spectral and spatial feature extraction branches are constructed, each branch consisting of three stacked feature extraction modules; each feature extraction module includes a selective large-kernel convolution layer and a feedforward neural network layer; first, a selective large-kernel convolution layer is constructed, including a convolution layer with a convolution kernel of 3, a dilated convolution layer with a convolution kernel of 3 and a dilation rate of 2, a large-kernel convolution layer with a convolution kernel of 7, and an attention selection layer; its working process is: the hyperspectral image pixel block X is subjected to the convolution layer and the dilated convolution layer to obtain features of two scales and ;Will and Splice on the channel and use global pooling and average pooling on the channel to obtain a dual-channel spatial feature ; Use a large kernel convolution with a convolution kernel of 7 to extract large-scale spatial features and use the Sigmoid activation function to obtain the spatial attention map ; The obtained spatial attention map is compared with the features of the two scales and Channel-by-channel dot multiplication to obtain attention features ; Secondly, construct a feedforward neural network layer with a serial structure, including a convolution layer with a convolution kernel of 3, a GELU activation function layer and a maximum pooling layer with a pooling kernel of 2; S3: Using the preprocessed hyperspectral image, pre-train the dual-path selective large-kernel spatial-spectral convolutional network to obtain a pre-trained dual-path selective large-kernel spatial-spectral convolutional network; In an exemplary embodiment, step S3 specifically includes: S31: Obtain a mask image using a similarity calculation method based on the preprocessed hyperspectral image; In an exemplary embodiment, step S31 specifically includes: S311: Based on the preprocessed hyperspectral image, the similarity between each pixel in the pixel block and the central pixel is obtained, such as the formula: , , , in, is the similarity between each pixel in the pixel block and the central pixel, is the fusion factor, is the cosine similarity between each pixel in the pixel block and the central pixel, is the spatial distance between each pixel in the pixel block and the central pixel, Indicates The first pixel in the pixels, Indicates The center pixel of the pixel block, , Respectively represent the pixel blocks The horizontal and vertical coordinates of pixels, , Represent the horizontal and vertical coordinates of the center pixel respectively; S312: sorting the pixel blocks in the pixel block according to the similarity between each pixel in the pixel block and the central pixel and the pre-processed hyperspectral image, selecting pixels in a preset sorting order to perform a mask operation, and obtaining a mask image; S32: Construct the total loss function, such as formula: , , , , Among them, L is the total loss function, , , , represents four learnable parameters that sum to 1, is the mean square error loss, is the contrast loss corresponding to the i-th feature extraction module, n represents the number of samples, Indicates mask image, Indicates The mask image is reconstructed by a dual-path selective large kernel spatial spectral convolutional network. Represents a function that measures the distance between two samples. Here, cosine similarity is used. Represents the spectral characteristics, Represents spatial features, represents the temperature coefficient, represents a layer of linear neural network; T represents matrix transpose; S33: Pre-training the dual-path selective large kernel spatial spectral convolutional network using the total loss function and the mask image to obtain a pre-trained dual-path selective large kernel spatial spectral convolutional network; For an exemplary embodiment, in step S3, a center-guided mask reconstruction and multi-level contrastive learning task are designed to pre-train the network without using label information; first, the center-guided mask reconstruction comprehensively utilizes the spectral and spatial information of the center pixel to mask the relevant pixels in the pixel block and reconstruct, that is, according to the similarity with the center pixel , sort the pixel blocks within the pixel block, select pixels with high similarity for masking operation (set to 0), and obtain the mask image; after passing through the feature extraction network, use a two-layer fully connected network to restore the input, and use the mean square error loss to constrain the reconstruction:

[0022] Then, multi-level contrastive learning is introduced after each module of the spatial and spectral branches; spectral features and spatial features are obtained after each feature extraction module; for spectral features, average pooling is used to convert them into a single vector; for spatial features, the vector at its center position is taken out; for spectral features and spatial features belonging to the same pixel block, they are regarded as positive sample pairs, and for spectral features and spatial features from different pixel blocks, they are regarded as negative sample pairs, and contrast loss is used. To constrain:

[0023]

[0024] Where n represents the number of samples, Represents the spectral characteristics, Represents spatial features, represents the temperature coefficient, Represents a function that measures the distance between two samples. Here, cosine similarity is used. Represents a one-layer linear neural network. The network has three feature extraction modules in total, and three contrast losses will be obtained; The reconstruction loss and contrast loss are weighted and summed to get the total loss function L:

[0025] in, , , , represents four learnable parameters that sum to 1, Finally, the mask image and the total loss function are used to pre-train the dual-path selective large kernel spatial-spectral convolutional network to obtain a pre-trained dual-path selective large kernel spatial-spectral convolutional network. S4: construct a cross-attention module, and obtain a hyperspectral image classification network based on the cross-attention module and the pre-trained dual-path selective large-kernel spatial-spectral convolutional network; In an exemplary embodiment, step S4 specifically includes: S41: Construct a cross-attention module, as shown in the formula: , , in, The final output feature is obtained by multiplying the attention matrix with the spatial feature and pooling it, which is the output of the cross attention module. Represents a global pooling operation; Represents the spectral characteristics, Represents spatial features, The attention matrix is ​​obtained by calculating the attention weight for each feature of the spectral feature and the spatial feature. express Activation function; S42: Based on the cross attention module and the pre-trained dual-path selective large kernel spatial-spectral convolutional network, a hyperspectral image classification network is obtained; For an exemplary embodiment, in step S4, a cross attention module is designed to fuse the obtained spatial and spectral features, and a small number of labeled samples are used to fine-tune the network and classify the hyperspectral image to obtain the classification result of the whole image; specifically, first, the attention weights of each feature of the obtained spectral feature and the spatial feature are calculated to obtain the attention matrix :

[0026] in Represents the softmax activation function, multiplying the obtained attention matrix with the spatial feature points and pooling them to obtain the final output features :

[0027] in, Represents a global pooling operation; Then, the obtained features are classified using a two-layer nonlinear fully connected network to obtain the final classification result of each pixel block, that is, the multi-layer perceptron in the pre-trained two-way selective large kernel spatial spectrum convolutional network is replaced with a cross attention module to obtain a hyperspectral image classification network; S5: Use the hyperspectral image classification network to classify the hyperspectral image to be classified and obtain the classification result.

[0028] In some embodiments, the above-mentioned hyperspectral image classification method based on mask and contrast learning can also be implemented in the following manner. To illustrate the effect of the present invention, the following embodiments are provided for experimental comparison.

[0029] The experiment was verified using three hyperspectral image datasets: PaviaU, Houston2013, and Xuzhou. For fairness, each method used the same 5 labeled samples per class for training, and the remaining samples were used as the test set. The experimental results were obtained by taking the average of 10 experiments.

[0030] Overall accuracy OA, average accuracy AA and Kappa coefficient are the most common and widely used objective measurement indicators of classification methods.

[0031] , The closer OA is to 1, the better the classification effect is.

[0032] , Where n represents the number of categories, Represents the classification accuracy of category i. AA reflects the classification balance of categories.

[0033] , in represents the overall accuracy, i.e. OA, represents the expected accuracy of random classification, , in represents the actual number of samples of the category in the confusion matrix, represents the number of predicted samples in the confusion matrix, N is the total number of samples, and the kappa coefficient reflects the consistency of the classification effect.

[0034] The technical solution of the present application is experimentally compared and verified with the deep learning method and self-supervised learning method in the field of hyperspectral image classification, as well as References 1 and 2. The comparison of classification accuracy obtained from the experiment is shown in Table 1.

[0035] Table 1: Comparison of classification accuracy of different methods

[0036] It can be seen that for three different types of hyperspectral image data sets, the technical solution of the present application is superior to the comparative method.

[0037] This embodiment provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned hyperspectral image classification method based on mask and contrast learning are implemented. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memory.

[0038] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned hyperspectral image classification method based on mask and contrast learning are implemented.

[0039] like Figure 3As shown, the computer device 120 may include: at least one processor 121, such as a central processing unit (CPU), at least one communication interface 123, a memory 124, and at least one communication bus 122. Among them, the communication bus 122 is used to realize the connection and communication between these components. Among them, the communication interface 123 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 123 may also include a standard wired interface and a wireless interface. The memory 124 may be a high-speed random access memory (Random Access Memory, RAM), or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 124 may also be at least one storage device located away from the aforementioned processor 121. Among them, the memory 124 stores an application program, and the processor 121 calls the program code stored in the memory 124 to perform any of the above method steps. Among them, the communication bus 122 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 122 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3Only one line is used to represent it, but it does not mean that there is only one bus or one type of bus. Among them, the memory 124 may include a volatile memory (volatile memory), such as a random-access memory (random-access memory, RAM); the memory may also include a non-volatile memory (non-volatile memory), such as a flash memory (flash memory), a hard disk drive (hard disk drive, HDD) or a solid-state drive (solid-state drive, SSD); the memory 124 may also include a combination of the above-mentioned types of memory. Among them, the processor 121 may be a central processing unit (central processing unit, CPU), a network processor (network processor, NP) or a combination of CPU and NP. Among them, the processor 121 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit (application-specific integrated circuit, ASIC), a programmable logic device (programmable logic device, PLD) or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), a generic array logic (generic array logic, GAL) or any combination thereof. Optionally, the memory 124 is also used to store program instructions. The processor 121 can call the program instructions to implement the hyperspectral image classification method based on mask and contrast learning as in this embodiment.

[0040] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A hyperspectral image classification method based on mask and contrast learning, characterized in that: The following steps are involved: S1: Acquire a hyperspectral image, and preprocess the hyperspectral image to obtain a preprocessed hyperspectral image; S2: constructing a dual-path selective large-kernel spatial-spectral convolutional network, wherein the dual-path selective large-kernel spatial-spectral convolutional network is used to extract spatial features and spectral features of hyperspectral images; S3: using the preprocessed hyperspectral image, pre-training the dual-path selective large-kernel spatial-spectral convolutional network to obtain a pre-trained dual-path selective large-kernel spatial-spectral convolutional network; S4: constructing a cross attention module, and obtaining a hyperspectral image classification network according to the cross attention module and the pre-trained dual-path selective large kernel spatial-spectral convolutional network; S5: Classify the hyperspectral image to be classified using the hyperspectral image classification network to obtain a classification result.

2. The hyperspectral image classification method based on mask and contrast learning according to claim 1, characterized in that: Step S1 specifically includes: obtaining a hyperspectral image, using a principal component analysis method to reduce the dimension of the band of the hyperspectral image and divide it into pixel blocks of fixed size to obtain a preprocessed hyperspectral image, wherein the preprocessed hyperspectral image includes a spatial part and a spectral part, as shown in the formula: ， ， in, Represents the first The center pixel of the pixel block, Represents the first The center pixel of the pixel block is set to 0. Indicates the length of the pixel block, Represents the width of the pixel block, Indicates the number of channels of the pixel block, represents the central pixel set, Indicates the set of parts where the central pixel is set to 0.

3. The hyperspectral image classification method based on mask and contrast learning according to claim 1, characterized in that: The dual-path selective large-kernel spatial-spectral convolutional network includes a spatial branch, a spectral branch and a multi-layer perceptron; the output end of the spatial branch and the output end of the spectral branch are connected to the input end of the multi-layer perceptron after matrix multiplication; the spatial branch includes a first feature extraction module, a second feature extraction module, a third feature extraction module and an adaptive pooling layer connected in series in sequence; the first feature extraction module includes a first feature extraction module connected in series in sequence Convolutional layer, first GELU activation function layer, selective large kernel convolutional layer, second convolutional layer, a first addition operator and a feed-forward neural network layer, the first The input end of the convolutional layer is also connected to the other input end of the first addition operator; the structure of the second feature extraction module and the structure of the third feature extraction module are the same as the structure of the first feature extraction module; the structure of the spectral branch is the same as the structure of the spatial branch.

4. The hyperspectral image classification method based on mask and contrast learning according to claim 3 is characterized in that: The selective large kernel convolution layer includes a first Convolutional layer, Dilated convolution layer, concatenation layer, global pooling layer, average pooling layer, Large kernel convolution layer, Sigmoid activation function, attention selection layer; The expansion rate of the dilated convolutional layer is 2; the first The input of the convolutional layer and the The input end of the dilated convolutional layer is connected to the first The output of the convolutional layer and the The output end of the dilated convolution layer is connected to the two input ends of the splicing layer respectively, the output end of the splicing layer is connected to the input end of the global pooling layer and the input end of the average pooling layer, and the output end of the global pooling layer and the output end of the average pooling layer are connected to the The input end of the large kernel convolution layer is connected. The output end of the large kernel convolution layer is connected to the input end of the Sigmoid activation function, and the output end of the Sigmoid activation function is connected to the input end of the attention selection layer. The output of the convolutional layer and the The output of the dilated convolutional layer is connected to the input of the attention selection layer.

5. The hyperspectral image classification method based on mask and contrast learning according to claim 3, characterized in that: The feedforward neural network layer includes a second A convolutional layer, a second GELU activation function layer and a maximum pooling layer, wherein the pooling kernel of the maximum pooling layer is 2.

6. The hyperspectral image classification method based on mask and contrast learning according to claim 1, characterized in that: Step S3 specifically includes: S31: Obtaining a mask image using a similarity calculation method according to the preprocessed hyperspectral image; S32: Construct the total loss function, such as the formula: ， ， ， ， Among them, L is the total loss function, , , , represents four learnable parameters that sum to 1, is the mean square error loss, is the contrast loss corresponding to the i-th feature extraction module, n represents the number of samples, Indicates mask image, Indicates The mask image is reconstructed by a dual-path selective large kernel spatial spectral convolutional network. Represents a function that measures the distance between two samples. Here, cosine similarity is used. Represents the spectral characteristics, Represents spatial features, represents the temperature coefficient, represents a layer of linear neural network; T represents matrix transpose; S33: Pre-train the dual-path selective large kernel spatial-spectral convolutional network using the total loss function and the mask image to obtain a pre-trained dual-path selective large kernel spatial-spectral convolutional network.

7. The hyperspectral image classification method based on mask and contrast learning according to claim 6, characterized in that: Step S31 specifically includes: S311: According to the pre-processed hyperspectral image, the similarity between each pixel in the pixel block and the central pixel is obtained, such as the formula: ， ， ， in, is the similarity between each pixel in the pixel block and the central pixel, is the fusion factor, is the cosine similarity between each pixel in the pixel block and the central pixel, is the spatial distance between each pixel in the pixel block and the central pixel, Indicates The first pixel in the pixels, Indicates The center pixel of the pixel block, , Respectively represent the pixel blocks The horizontal and vertical coordinates of pixels, , Respectively represent the horizontal and vertical coordinates of the center pixel; S312: Sort the pixel blocks in the pixel block according to the similarity between each pixel in the pixel block and the central pixel and the preprocessed hyperspectral image, select pixels in a preset sorting to perform a mask operation, and obtain a mask image.

8. The hyperspectral image classification method based on mask and contrast learning according to claim 1, characterized in that: Step S4 specifically includes: S41: Construct a cross-attention module, as shown in the formula: ， ， in, The final output feature is obtained by multiplying the attention matrix with the spatial feature and pooling it, which is the output of the cross attention module. Represents a global pooling operation; Represents the spectral characteristics, Represents spatial features, The attention matrix is ​​obtained by calculating the attention weight for each feature of the spectral feature and the spatial feature. express Activation function; S42: Obtain a hyperspectral image classification network according to the cross attention module and the pre-trained dual-path selective large kernel spatial-spectral convolutional network.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hyperspectral image classification method based on mask and contrast learning are implemented as described in any one of claims 1 to 8.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the hyperspectral image classification method based on mask and contrast learning are implemented as described in any one of claims 1 to 8.

Images