A knowledge-guided spatial-spectral collaborative representation method and apparatus for remote sensing image classification

By employing a knowledge-guided spatial-spectral collaborative representation method, and utilizing the feature information of panchromatic and multispectral images, a classification model is constructed, which solves the problem of low accuracy in remote sensing image classification and achieves higher classification accuracy.

CN118447298BActive Publication Date: 2026-11-17XIDIAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410507519.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-25
Publication Date
2026-11-17
Estimated Expiration
2044-04-25

AI Technical Summary

Technical Problem

Existing remote sensing image classification methods fail to fully consider the respective advantages and corresponding auxiliary features of panchromatic and multispectral images, resulting in low accuracy of classification results.

Method used

A knowledge-guided spatial-spectral collaborative representation method is adopted. By preprocessing panchromatic and multispectral images, spatial location enhancement learning, and cross-spatial aggregation learning respectively, and combining them with a time-frequency collaborative spectral enhancement module, a classification model is constructed to make full use of their respective feature information.

Benefits of technology

It improves the accuracy of remote sensing image classification, enhances the feature extraction capabilities of panchromatic and multispectral images, and improves the performance of classification models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118447298B_ABST
    Figure CN118447298B_ABST
Patent Text Reader

Abstract

This invention provides a knowledge-guided spatial-spectral collaborative representation method and apparatus for remote sensing image classification, relating to the field of computer vision image processing technology. The method includes: selecting training and testing datasets from various land cover categories; sequentially inputting each first feature corresponding to the panchromatic image into a preprocessing module, a spatial location enhancement learning module, and a cross-spatial aggregation learning module to output a first final output feature; inputting each second feature corresponding to the multispectral image into a time-frequency collaborative spectral enhancement module to output a second final output feature; sequentially inputting the first and second final output features into a global average pooling layer and a fully connected layer to construct a classification model; training the classification model using the training dataset; inputting the testing dataset into the trained classification model; and outputting the classification result. This method fully considers the respective advantageous and auxiliary feature information of the panchromatic and multispectral images, resulting in high accuracy of the final classification result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer vision image processing technology, and in particular to a remote sensing image classification method and apparatus based on knowledge-guided spatial-spectral collaborative representation. Background Technology

[0002] With the continuous advancement of sensor technology and the constant improvement of facilities, more diverse remote sensing images can be acquired, including panchromatic (PAN) images and multispectral (MS) images. Numerous Earth observation satellites can simultaneously acquire PAN and MS images within the same coverage area. PAN images are single-channel images with richer spatial resolution and more detailed spatial information than MS images. MS images are multi-channel images, containing four channels: RGB (primary colors) and a near-infrared channel, providing rich spectral information. Due to the differences in reflectivity of land cover across different spectral bands, MS images can record more detailed surface information. The fine spatial information in PAN images and the rich spectral information in MS images are complementary. This complementarity allows for the effective extraction and classification of unique features from various remote sensing images, offering significant potential for the development of remote sensing image classification.

[0003] Currently, remote sensing image classification typically employs a method that fuses PAN and MS images. However, existing methods for fusing PAN and MS images only consider the dominant features of a single image, neglecting the representation of the dominant features of the other image within that single image. This results in insufficient consideration of modal information from individual images, leading to low accuracy in the final classification results. Summary of the Invention

[0004] The purpose of this invention is to provide a remote sensing image classification method and apparatus based on knowledge-guided spatial-spectral collaborative representation, thereby solving the problem of low accuracy in the final classification results.

[0005] To address the aforementioned technical problems, the embodiments of the present invention provide the following technical solutions:

[0006] The first aspect of this invention provides a remote sensing image classification method based on knowledge-guided spatial-spectral collaborative representation, the method comprising:

[0007] Obtain panchromatic images, multispectral images, and corresponding class labels for the multispectral images. The class labels include multiple land cover categories.

[0008] The panchromatic image and the multispectral image are respectively normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, and the corresponding first features and second features are output respectively. Training datasets and test datasets are selected from various land cover categories according to preset rules.

[0009] Each first feature is sequentially input into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module to output the first final output feature. Each second feature is input into the time-frequency co-processing spectral enhancement module to output the second final output feature. The spatial location enhancement learning module includes a pooling layer, a 1×1 convolutional layer, and a Sigmoid activation function layer. The cross-spatial aggregation learning module includes a two-dimensional pooling layer and a Softmax function layer. The time-frequency co-processing spectral enhancement module includes an internal information extraction submodule, a hidden minute detail submodule, a two-dimensional pooling layer, a Softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The internal information extraction submodule includes a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule includes a 3×3 convolutional layer.

[0010] The first and second final output features are sequentially input into a global average pooling layer and a fully connected layer to construct a classification model.

[0011] The classification model is trained using the training dataset to obtain a well-trained classification model.

[0012] The test dataset is input into the trained classification model, which then classifies the test dataset and outputs the classification results.

[0013] A second aspect of this application provides a remote sensing image classification device based on knowledge-guided spatial-spectral collaborative representation, the device comprising:

[0014] The acquisition module is used to acquire panchromatic images, multispectral images, and class label images corresponding to the multispectral images. The class label images include multiple land cover categories.

[0015] The processing and selection module is used to perform normalization, neighborhood division and input into two 3×3 convolutional layers on the panchromatic image and the multispectral image respectively, and output multiple first features and multiple second features respectively. According to preset rules, the training dataset and test dataset are selected from various land cover categories respectively.

[0016] The input processing module is used to sequentially input each first feature into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module, and output the first final output feature. It also inputs each second feature into the time-frequency co-processing spectral enhancement module and outputs the second final output feature. The spatial location enhancement learning module is a module including a pooling layer, a 1×1 convolutional layer, and a Sigmoid activation function layer. The cross-spatial aggregation learning module is a module including a two-dimensional pooling layer and a Softmax function layer. The time-frequency co-processing spectral enhancement module is a module including an internal information extraction submodule, a hidden minute detail submodule, a two-dimensional pooling layer, a Softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The internal information extraction submodule is a submodule including a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule is a submodule including a 3×3 convolutional layer.

[0017] The building module is used to sequentially input the first and second final output features into the global average pooling layer and the fully connected layer to build a classification model;

[0018] The training module is used to train the classification model using the training dataset to obtain a trained classification model.

[0019] The classification module is used to input the test dataset into the trained classification model, classify the test dataset using the trained classification model, and output the classification result.

[0020] Compared to existing technologies, the knowledge-guided spatial-spectral collaborative representation remote sensing image classification method and apparatus provided by this invention sequentially inputs the first features corresponding to the panchromatic image into a preprocessing module, a spatial location enhancement learning module, and a cross-spatial aggregation learning module, outputting a first final output feature. It also inputs the second features corresponding to the multispectral image into a time-frequency collaborative spectral enhancement module, outputting a second final output feature. In the branch modules of the constructed and trained classification model, the respective advantageous feature information and corresponding auxiliary feature information of the panchromatic image and the multispectral image can be fully considered, resulting in higher accuracy of the final classification result of the classification model. Attached Figure Description

[0021] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, with the same or corresponding reference numerals denoteing the same or corresponding parts, wherein:

[0022] Figure 1 The flowchart of a knowledge-guided spatial-spectral collaborative representation method for remote sensing image classification is illustrated schematically. Figure 1 ;

[0023] Figure 2 The flowchart of a knowledge-guided spatial-spectral collaborative representation method for remote sensing image classification is illustrated schematically. Figure 2 ;

[0024] Figure 3 The classification results of the images to be classified are shown schematically.

[0025] Figure 4 A schematic diagram of a remote sensing image classification device based on knowledge-guided spatial-spectral collaborative representation is shown. Detailed Implementation

[0026] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0027] It should be noted that, unless otherwise stated, the technical or scientific terms used in this invention should have the ordinary meaning as understood by one of ordinary skill in the art.

[0028] The methods described in the embodiments of the present invention will be explained in detail below.

[0029] Figure 1 A flowchart illustrating a remote sensing image classification method for knowledge-guided spatial-spectral collaborative representation in an embodiment of the present invention is shown. See [link to flowchart illustration]. Figure 1 As shown, the method may include:

[0030] S101. Obtain the panchromatic image, multispectral image, and the corresponding class label image of the multispectral image.

[0031] The class label images include multiple land cover categories.

[0032] Both PAN and MS images are registered images.

[0033] You can obtain the class label images corresponding to PAN images, MS images, and MS images from the dataset.

[0034] S102. Normalize the panchromatic image and the multispectral image, divide them into neighborhoods, and input them into two 3×3 convolutional layers. Output the corresponding first features and multiple second features respectively. Then, select the training dataset and test dataset from each land cover category according to the preset rules.

[0035] Specifically, the panchromatic image is sequentially normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, outputting multiple corresponding first features. Similarly, the multispectral image is sequentially normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, outputting multiple corresponding second features.

[0036] Then, according to preset rules, training and test datasets are selected from each land cover category. Specifically, according to a preset number of samples, sample land cover is selected twice from each land cover category. The first sample land cover is used as the training dataset, and the second sample land cover is labeled and used as the test dataset.

[0037] S103. Input each first feature sequentially into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module, output the first final output feature, and input each second feature into the time-frequency collaborative spectral enhancement module, output the second final output feature.

[0038] The spatial location enhancement learning module comprises a pooling layer, a 1×1 convolutional layer, and a sigmoid activation function layer. The cross-spatial aggregation learning module comprises a 2D pooling layer and a softmax function layer. The time-frequency co-processing spectral enhancement module comprises an interior information extraction submodule, a hidden subtle detail submodule, a 2D pooling layer, a softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The interior information extraction submodule comprises a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden subtle detail submodule comprises a 3×3 convolutional layer.

[0039] S104. Input the first final output feature and the second final output feature into the global average pooling layer and the fully connected layer in sequence to construct the classification model.

[0040] For the first final output feature Second final output features The features are cascaded along the channel dimension and then sequentially input into a global average pooling layer and a fully connected layer to construct a classification model.

[0041] S105. Train the classification model using the training dataset to obtain a trained classification model.

[0042] The classification model is trained using the training dataset, and training is stopped at a specified number of steps to obtain the trained classification model.

[0043] S106. Input the test dataset into the trained classification model, use the trained classification model to classify the test dataset, and output the classification result.

[0044] The sample blocks of the test dataset are used as input to the trained classification model, and the output of the trained classification model is the classification category obtained by classifying each pixel in the test dataset.

[0045] Based on the above Figure 1 As can be seen from the implementation method, the embodiments of the present invention acquire panchromatic images, multispectral images, and corresponding class label images of multispectral images; the panchromatic images and multispectral images are respectively normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, respectively, outputting multiple first features and multiple second features, and training datasets and test datasets are selected from various land cover categories according to preset rules; each first feature is sequentially input into a preprocessing module, a spatial location enhancement learning module, and a cross-spatial aggregation learning module, outputting a first final output feature, and each second feature is input into a time-frequency co-processing spectral enhancement module, outputting a second final output feature; the first final output feature and the second final output feature are sequentially input into a global average pooling layer and a fully connected layer to construct a classification model; the classification model is trained using the training dataset to obtain a trained classification model; the test dataset is input into the trained classification model, and the trained classification model is used to classify the test dataset, outputting the classification result. In this way, the first features corresponding to the panchromatic image are sequentially input into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module to output the first final output feature. The second features corresponding to the multispectral image are input into the time-frequency co-processing spectral enhancement module to output the second final output feature. In the branch modules of the constructed and trained classification model, the respective advantageous feature information and corresponding auxiliary feature information of the panchromatic image and the multispectral image can be fully considered, so that the final classification result of the classification model has high accuracy.

[0046] As a refinement and extension of the above embodiments, Figure 2 The flowchart of a knowledge-guided spatial-spectral collaborative representation remote sensing image classification method in an embodiment of the present invention is as follows. Figure 2 See Figure 2 As shown in the embodiment of the present invention, a remote sensing image classification method based on knowledge-guided spatial-spectral collaborative representation may include:

[0047] S201. Obtain the panchromatic image, multispectral image, and the corresponding class label image of the multispectral image.

[0048] The class label images include multiple land cover categories.

[0049] Both PAN and MS images are registered images.

[0050] You can obtain the class label images corresponding to PAN images, MS images, and MS images from the dataset.

[0051] S202. Normalize the panchromatic image and the multispectral image, divide them into neighborhoods, and input them into two 3×3 convolutional layers. Output the corresponding first features and multiple second features respectively. Then, select the training dataset and test dataset from each land cover category according to the preset rules.

[0052] Specifically, the PAN and MS images are sequentially normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, respectively, outputting multiple first features and multiple second features. Training and test datasets are then selected from various land cover categories according to preset rules, including:

[0053] Step A1: Normalize the panchromatic image and the multispectral image to obtain a normalized panchromatic image and a normalized multispectral image.

[0054] The values ​​of all pixels in the matrices corresponding to the PAN and MS images are normalized to the range [0,1] to achieve normalization.

[0055] Step A2: Divide the normalized panchromatic image into multiple panchromatic images according to the first preset neighborhood.

[0056] The first preset neighborhood is a 64×64 region. Find the pixels with corresponding class labels in the normalized PAN image, and take a 64×64 region of the PAN image neighborhood centered on each pixel as a sample block pair. The neighborhood of each block serves the classification of the center point, and multiple segmented PAN images can be obtained.

[0057] Step A3: Divide the normalized multispectral image according to the second preset neighborhood to obtain multiple divided multispectral images.

[0058] The second preset neighborhood is a 16×16 region. Find the pixels with corresponding class labels in the normalized MS image, and take the MS image neighborhood of 16×16 as the center of each pixel as a sample block pair. The neighborhood of each block serves the classification of the center point, and multiple divided MS images can be obtained.

[0059] Step A4: Input the multiple panchromatic images and the multiple multispectral images into two 3×3 convolutional layers respectively, and output multiple first features corresponding to the multiple panchromatic images and multiple second features corresponding to the multiple multispectral images.

[0060] Step A5: Select two sample land cover items from each land cover category according to the preset number of samples. Use the first sample land cover item as the training dataset and label the second sample land cover item as the test dataset.

[0061] The preset sample size is 2% of the samples from each land cover category. Approximately 2% of the samples from each land cover category in the class map are randomly and discretely selected twice. The land cover samples selected in the first selection are used as the training dataset, and the surrounding area of ​​the land cover samples selected in the second selection is blacked out with a neighborhood size of 50 pixels. The training and testing are completely separated to obtain the test dataset.

[0062] Steps S203-S206 below are the specific operations of sequentially inputting each first feature into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module to output the first final output feature, and inputting each second feature into the time-frequency collaborative spectral enhancement module to output the second final output feature.

[0063] S203. Input each first feature into the preprocessing module and output the weight coefficients between each channel corresponding to each first feature.

[0064] The preprocessing module consists of a 3×3 convolutional layer and a Softmax function layer.

[0065] Specifically, each first feature is input into the preprocessing module, and the weight coefficients between each channel corresponding to each first feature are output, including:

[0066] Step B1: Input each first feature into a 3×3 convolutional layer, and adjust each feature output by the 3×3 convolutional layer according to the first preset size and the second preset size to obtain each first intermediate feature and each second intermediate feature.

[0067] Each first feature The input is fed into a 3×3 convolutional layer, and the output features of the 3×3 convolutional layer are adjusted according to the first preset size and the second preset size respectively to obtain the first intermediate features F. d and each of the second intermediate features F f The first and second preset sizes are different and can be set manually in advance.

[0068] Step B2: Perform matrix multiplication on each first intermediate feature and each second intermediate feature to obtain each third intermediate feature.

[0069] Each of the third intermediate features is represented in matrix form, and each matrix includes multiple rows of matrix elements.

[0070] Each first intermediate feature F d and each of the second intermediate features F f Perform matrix multiplication to obtain each third intermediate feature F. a .

[0071] Step B3: Perform row-by-row addition on the multi-row matrix elements in each of the third intermediate features to obtain the features of each variable.

[0072] Each third intermediate feature F a The multi-row matrix elements are added row by row to obtain the characteristic F of each variable. h .

[0073] Step B4: Input the features of each variable into the Softmax function layer, and output the weight coefficients F between each channel corresponding to each variable. e .

[0074] S204. For each first feature input spatial location enhancement learning module, output the third final output feature.

[0075] The spatial location reinforcement learning module consists of a pooling layer, a 1×1 convolutional layer, and a sigmoid activation function layer.

[0076] Specifically, in the spatial location enhancement learning module for each first feature input, the third final output feature is output, including:

[0077] Step C1: For each first feature Pooling operations are performed along the height and width directions of the panchromatic image to obtain the height feature F. H and width feature F W .

[0078] Step C2: Transfer the height feature F H and width feature F W The data is cascaded and input into a 1×1 convolutional layer, outputting cascaded features F. C .

[0079] Cascade Feature F C It contains location information in space.

[0080] Step C3: Concatenate feature F C The two separated features are then input into a Sigmoid activation function layer, and the output height position F is calculated. H1 and the position F in width W1 .

[0081] Step C4: Position F in width W1 Adjust according to the third preset size to obtain the position F in the adjustment width. W2 .

[0082] The third preset size is a size that is set in advance by the user.

[0083] Step C5: Broadcast the signal to each first feature. Position F at height H1 And adjust the position F on the width W2Perform element-wise multiplication and adjust the resulting features according to the fourth preset size to obtain the third final output feature F. SLE .

[0084] The fourth preset size is a size that is set in advance by the user.

[0085] S205. Input the third final output feature and weight coefficients into the cross-space aggregation learning module to output the first final output feature.

[0086] The cross-space aggregation learning module consists of a two-dimensional pooling layer and a Softmax function layer.

[0087] Specifically, the third final output feature and weight coefficients are input into the cross-space aggregation learning module, which outputs the first final output feature, including:

[0088] Step D1: Convert the third final output feature F SLE Perform a two-dimensional pooling operation to obtain the pooled channel information F. b .

[0089] Step D2: Pool the channel information F b Input the softmax function layer and adjust the output features according to the fifth preset size to obtain the corresponding weight information F. c .

[0090] The fifth preset size is a size that is set in advance by the user.

[0091] Step D3: For each first intermediate feature F d and weight information F c Perform matrix multiplication and size adjustment sequentially to obtain the first spatial feature F. m1 .

[0092] Step D4: For the third final output feature F SLE With weighting coefficient F e By performing matrix multiplication and size adjustment sequentially, the second spatial feature F is obtained. m2 .

[0093] By analyzing the third final output feature F SLE With weighting coefficient F e Perform matrix multiplication operations sequentially, and obtain the second spatial feature F after adjusting the size. m2 This helps to generate a higher level of pixel-level attention.

[0094] Step D5: For the first spatial feature F m1 Second spatial feature F m2Perform element-wise addition, and then input the resulting features into the Softmax function layer to output the fourth final feature F. CSA .

[0095] Step D6: For the fourth final output feature F CSA With each first feature Perform element-wise multiplication to obtain the first final output feature.

[0096] The cross-spatial aggregation learning module captures pixel-level pairwise relationships and highlights global contextual information across all pixels. This allows for better focus on high-level feature maps at the pixel level. Furthermore, each parallel sub-network, i.e., each module, establishes local cross-channel interactions. This not only encodes inter-channel information and adjusts the importance of different channels, but also preserves precise spatial structure information within each channel.

[0097] S206. Input each of the second features into the time-frequency coordinated spectral enhancement module and output the second final output feature.

[0098] The time-frequency co-processed spectral enhancement module comprises an inner information extraction submodule, a hidden minute detail submodule, a 2D pooling layer, a Softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The inner information extraction submodule includes a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule includes a 3×3 convolutional layer.

[0099] Specifically, each of the second features is input into the time-frequency coordinated spectral enhancement module, and the second final output feature is output, including:

[0100] Step E1: According to each second feature The corresponding preset ratio of the number of channels, for each second feature The fourth intermediate feature MS is obtained by partitioning. up and the fifth intermediate feature MS low .

[0101] The preset ratio for the number of channels can be 0.5. For example, if each second feature corresponds to 64 channels, then according to the preset ratio of 0.5, each second feature can be... The fourth intermediate feature MS is divided into two 32-channel corresponding features. up and the fifth intermediate feature MS low .

[0102] Step E2: Input the fourth intermediate feature into the internal information extraction submodule and output the sixth intermediate feature.

[0103] The internal information extraction submodule consists of a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer.

[0104] Specifically, the fourth intermediate feature is input into the internal information extraction submodule, and the sixth intermediate feature is output, including:

[0105] Step E21: Perform a 3×3 grouped convolution on the fourth intermediate feature according to the preset number of groups to obtain the eleventh intermediate feature.

[0106] The preset number of groups can be 2. For the fourth intermediate feature MS... up Perform 3×3 grouped convolutions, with 2 groups, to obtain the eleventh intermediate feature.

[0107] Step E22: Transfer the fourth intermediate feature MS up The input is processed into a 1×1 convolutional layer, which performs a 1×1 convolution and outputs the twelfth intermediate feature.

[0108] Step E23: Transfer the fourth intermediate feature MS up The input is processed into a 3×3 convolutional layer, which performs a 3×3 convolution and outputs the thirteenth intermediate feature.

[0109] Step E24: For the thirteenth intermediate feature Perform a Fourier transform to obtain the frequency characteristics.

[0110] Step E25: Multiply the learning parameters and frequency features to obtain the spectral components, and perform an inverse Fourier transform on the spectral components to obtain the time-domain features.

[0111] Construct learnable parameters ω c and its frequency characteristics Multiplication is performed to obtain the most important spectral components for image classification tasks. Then, the spectral components are subjected to inverse Fourier transform to obtain the temporal features.

[0112] Step E26: For the eleventh intermediate feature Twelfth intermediate feature and temporal features Performing element-wise addition yields the sixth intermediate feature.

[0113] Step E3: Input the fifth intermediate feature into the hidden minute details submodule and output the seventh intermediate feature.

[0114] The hidden minute details submodule is a submodule that includes a 3×3 convolutional layer.

[0115] Specifically, the fifth intermediate feature is input into the hidden minute details submodule, and the seventh intermediate feature is output, including:

[0116] Step E31: Transfer the fifth intermediate feature MS low The input is a 3×3 convolutional layer, and the output is the fourteenth intermediate feature.

[0117] Step E32: For the fourteenth intermediate feature and the fifth intermediate feature MS low Performing element-wise addition yields the seventh intermediate feature.

[0118] Step E4: Concatenate the sixth and seventh intermediate features along the channel dimension to obtain the eighth intermediate feature. Then, perform two-dimensional pooling operations on the eighth intermediate feature and input it into the Softmax function layer. Output the weights of each channel corresponding to the eighth intermediate feature. Multiply the weights of the eighth intermediate feature and each channel to obtain the optimized feature.

[0119] The sixth intermediate feature and the seventh intermediate feature Concatenating along the channel dimension yields the eighth intermediate feature. And the eighth intermediate feature Perform a two-dimensional pooling operation to obtain a feature S1 containing only channel information. Input this feature S1 into a Softmax function layer, which outputs the weights β1 for each channel corresponding to the eighth intermediate feature. Multiplying the result by the weight β1 of each channel yields the optimized feature.

[0120] Step E5: For each second feature Perform max pooling and mean pooling operations respectively to obtain the ninth intermediate feature. and the tenth intermediate feature And for the ninth intermediate feature and the tenth intermediate feature The concatenated features are then sequentially fed into a 3×3 convolutional layer and a softmax function layer to obtain spatial information features. Spatial information characteristics and each second feature Element-wise multiplication yields spatially enhanced features.

[0121] Step E6: Enhance spatial features and optimized features Perform element-wise addition to obtain the second final output feature.

[0122] The time-frequency co-processed spectral enhancement module captures different types of features by separating channels. It extracts rich, representative features and subtle latent features through lightweight convolution operations. Fourier transform and learnable parameters accurately capture lines and edges in the image, determine the weight of each frequency component, and identify the spectral components most important for classification.

[0123] The modules corresponding to PAN and MS images each take into account their respective dominant modal information and corresponding supplementary modal information.

[0124] S207. Input the first final output feature and the second final output feature into the global average pooling layer and the fully connected layer in sequence to construct a classification model.

[0125] For the first final output feature Second final output features The features are cascaded along the channel dimension and then sequentially input into a global average pooling layer and a fully connected layer to construct a classification model.

[0126] For the branches corresponding to PAN and MS images, there are four stages, forming a pyramid structure.

[0127] Specifically as follows:

[0128] For the specific operation of the PAN image branch, the first stage consists of a first layer: a 3×3 convolutional layer with a stride of 2, 1 input channel, and 32 output channels; the second layer consists of a 3×3 convolutional layer with a stride of 2, 32 input channels, and 64 output channels; then, a location-guided spatial enhancement module A is used to perform element-wise multiplication.

[0129] The second stage: First layer: 3×3 convolutional layer with a stride of 1, 64 input channels, and 128 output channels; followed by two location-guided spatial augmentation modules A;

[0130] The third stage: First layer: 3×3 convolutional layer with a stride of 1, 128 input channels, and 256 output channels; followed by spatial augmentation module A with 3 positions guided.

[0131] The fourth stage: the first layer: a 3×3 convolutional layer with a stride of 1, 256 input channels and 512 output channels; followed by a spatial augmentation module A with a location guidance.

[0132] The first final output feature can be obtained through the operations of the PAN image branch described above.

[0133] For the specific operation of the MS image branch, the first stage consists of the following layers: the first layer is a 3×3 convolutional layer with a stride of 1, 4 input channels, and 32 output channels; the second layer is a 3×3 convolutional layer with a stride of 1, 32 input channels, and 64 output channels; followed by a time-frequency co-located spectral enhancement module B.

[0134] The second stage: First layer: 3×3 convolutional layer with a stride of 1, 64 input channels, and 128 output channels; followed by two time-frequency co-located spectral enhancement modules B;

[0135] The third stage: First layer: 3×3 convolutional layer with a stride of 1, 128 input channels and 256 output channels; followed by 3 time-frequency co-located spectral enhancement modules B;

[0136] The fourth stage: the first layer: a 3×3 convolutional layer with a stride of 1, 256 input channels and 512 output channels; followed by a time-frequency co-located spectral enhancement module B.

[0137] The second final output feature can be obtained through the operations of the MS image branch described above.

[0138] S208. Train the classification model using the training dataset to obtain a trained classification model.

[0139] Before training the classification model using the training dataset to obtain a trained classification model, the following steps are required:

[0140] The loss function for the classification model can be constructed using the following formula, where cross-entropy loss can be used:

[0141]

[0142] Where p(x) is the true probability distribution, q(x) is the predicted probability distribution for classification, and x is each pixel in the image.

[0143] Adjust the hyperparameters: Specifically, set the batch size to 64, the learning rate to 0.002, and the number of training epochs to 50.

[0144] Specifically, the sample blocks of the training dataset are used as input to the classification model. Multiple features corresponding to the MS image and PAN image are simultaneously input into the two branches of the classification model. The category of each pixel in the training dataset is used as the output of the classification model. The network parameters of the classification model are optimized by solving the error between the above categories and the corresponding true categories on the ground map, and then minimizing the error through backpropagation. Training is stopped at a specified number of training steps to obtain the trained classification model.

[0145] S209. Input the test dataset into the trained classification model, use the trained classification model to classify the test dataset, and output the classification result.

[0146] The sample blocks of the test dataset are used as input to the trained classification model, and the output of the trained classification model is the classification category obtained by classifying each pixel in the test dataset.

[0147] By embedding spatial location information through a cross-spatial aggregation learning module, spatial location information under different channels can be enhanced, thereby enhancing spatial features and generating better pixel-level attention to the feature map. The time-frequency combined analysis enables adaptive capture of the most important spectral components for the classification task through learnable parameters, thereby reducing information redundancy and enhancing the generalization of the network structure. In this method, the module branches corresponding to panchromatic images and multispectral images not only consider the dominant modal information of the images, but also the auxiliary feature information of each other, enhancing the feature extraction capability of single images and improving the classification performance of the network.

[0148] Different types of features are captured by separating channels. Rich, representative features and subtle latent features are extracted through lightweight convolution operations. Fourier transform and learnable parameters accurately capture lines and edges in the image, determine the weight of each frequency component, and identify the spectral components most important for classification.

[0149] This invention addresses the characteristics of different modalities under single-modal data, constructs a time-frequency collaborative feature extraction strategy, enhances the representation ability of spatial location information, proposes a novel network model for remote sensing data, and thus effectively improves classification performance.

[0150] Experimental simulation of this invention embodiment:

[0151] The simulation conditions are as follows: hardware platform: HP-Z840 workstation, RTX2080TI-12GB-GPU, 64GB RAM; software platform: Python, PyTorch deep learning framework.

[0152] Simulation content and results:

[0153] Figure 3The classification results of the images to be classified are illustrated schematically. The images used in the simulation experiment of this invention are remote sensing images of Hohhot city. The PAN image consists of 8004×8404 pixels, and the MS image consists of 2001*2101*4 pixels. The labeled classification scenes include 11 categories: water areas, roads, bare land, vegetation, densely built-up urban areas, urban residential areas, suburbs, farmland, unidentified white-roofed buildings, red-roofed buildings, and blue-roofed buildings. The results are shown below. Figure 3 The classification results include these 11 categories.

[0154] After training, the overall accuracy (OA) of the correctly classified samples in the total test samples of the method of this invention was 0.7377, the average precision (AA) was 0.7696, and the Kappa coefficient was 0.7039. This demonstrates that focusing on spatial location information and important spectral components is indeed beneficial to improving the fusion classification results. Moreover, fully considering the information of different modalities in a single image can further improve the pixel-by-pixel classification performance of remote sensing images.

[0155] Table 1 lists the method of this invention and other methods (AM). 3 The performance of Net, CALC, and DCTransformer in geomorphological classification in Hohhot City was compared, and the overall accuracy (OA), average accuracy (AA), and kappa coefficient of each method were statistically analyzed.

[0156] Table 1 Comparison of geomorphic classification performance in Hohhot city

[0157] method Overall accuracy (%) Average accuracy (%) Kappa (%) <![CDATA[AM 3 Net]]> 72.31 75.91 68.95 CALC 72.50 75.70 69.02 DCTransformer 59.59 61.23 54.21 Method of the present invention 73.77 76.96 70.39

[0158] As can be seen from the comparative experiments in Table 1, the method of this invention, compared with other methods, does indeed help improve classification performance in multimodal image fusion classification tasks by focusing on each other's auxiliary feature information.

[0159] In summary, this invention proposes a knowledge-guided spatial-spectral collaborative representation method for remote sensing image classification, which solves the problem of low accuracy in classification results caused by insufficient consideration of modal information of single images in the current technology, and can be used for the fusion classification of multimodal images.

[0160] Based on the same inventive concept, as an implementation of the above-mentioned knowledge-guided spatial-spectral collaborative representation remote sensing image classification method, this embodiment of the invention also provides a knowledge-guided spatial-spectral collaborative representation remote sensing image classification device. Figure 4 This is a structural diagram of the device in an embodiment of the present invention. See also: Figure 4 As shown, the device may include:

[0161] The acquisition module 401 is used to acquire panchromatic images, multispectral images, and class label images corresponding to multispectral images. The class label images include multiple land cover categories.

[0162] The processing selection module 402 is used to perform normalization processing, neighborhood division and input into two 3×3 convolutional layers on the panchromatic image and the multispectral image acquired by the acquisition module 401, respectively, and output multiple first features and multiple second features respectively. According to preset rules, the training dataset and test dataset are selected from various land cover categories respectively.

[0163] The input processing module 403 is used to sequentially input the first features output by the processing selection module 402 into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module, and output the first final output features. It also inputs the second features output by the processing selection module 402 into the time-frequency co-processing spectral enhancement module, and outputs the second final output features. The spatial location enhancement learning module is a module including a pooling layer, a 1×1 convolutional layer, and a Sigmoid activation function layer. The cross-spatial aggregation learning module is a module including a two-dimensional pooling layer and a Softmax function layer. The time-frequency co-processing spectral enhancement module is a module including an inner information extraction submodule, a hidden minute detail submodule, a two-dimensional pooling layer, a Softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The inner information extraction submodule is a submodule including a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule is a submodule including a 3×3 convolutional layer.

[0164] The construction module 404 is used to sequentially input the first final output feature output by the input processing module 403 and the second final output feature output by the input processing module 403 into the global average pooling layer and the fully connected layer to construct a classification model.

[0165] The training module 405 is used to train the classification model constructed by the construction module 404 by processing the training dataset selected by the selection module 402, so as to obtain the trained classification model.

[0166] The classification module 406 is used to input the test dataset selected by the processing selection module 402 into the trained classification model trained by the training module 405, and use the trained classification model to classify the test dataset and output the classification result.

[0167] The processing and selection module 402 is specifically used to normalize the panchromatic image and the multispectral image to obtain normalized panchromatic images and normalized multispectral images; to divide the normalized panchromatic image according to a first preset neighborhood to obtain multiple divided panchromatic images; to divide the normalized multispectral image according to a second preset neighborhood to obtain multiple divided multispectral images; to input the multiple divided panchromatic images and the multiple divided multispectral images into two 3×3 convolutional layers respectively, and to output multiple first features corresponding to the multiple divided panchromatic images and multiple second features corresponding to the multiple divided multispectral images; and to select sample land cover twice from each land cover category according to a preset number of samples, using the first sample land cover as the training dataset and the second sample land cover as the test dataset.

[0168] The input processing module 403 is specifically used to input each first feature into the preprocessing module and output the weight coefficients between each channel corresponding to each first feature. The preprocessing module includes a 3×3 convolutional layer and a Softmax function layer. The first features are input into the spatial location enhancement learning module and output the third final output feature. The third final output feature and the weight coefficients are input into the cross-spatial aggregation learning module and output the first final output feature. The second features are input into the time-frequency co-processing spectral enhancement module and output the second final output feature.

[0169] The input processing module 403 inputs each first feature into the preprocessing module and outputs the weight coefficients between each channel corresponding to each first feature, including: inputting each first feature into a 3×3 convolutional layer, and adjusting each feature output by the 3×3 convolutional layer according to a first preset size and a second preset size to obtain each first intermediate feature and each second intermediate feature; performing matrix multiplication on each first intermediate feature and each second intermediate feature to obtain each third intermediate feature, each third intermediate feature being represented in matrix form, each matrix including multiple rows of matrix elements; performing row-by-row addition on the multiple rows of matrix elements in each third intermediate feature to obtain each variable feature; and inputting each variable feature into a Softmax function layer to output the weight coefficients between each channel corresponding to each variable.

[0170] The input processing module 403 inputs each first feature into the spatial location enhancement learning module and outputs a third final output feature, including: performing pooling operations on each first feature along the height and width directions of the panchromatic image to obtain height features and width features; concatenating the height features and width features and inputting them into a 1×1 convolutional layer to output concatenated features; separating the concatenated features and inputting the two separated features into a Sigmoid activation function layer to output the height position and width position; adjusting the width position according to a third preset size to obtain the adjusted width position; and performing element-wise multiplication operations on each first feature, the height position, and the adjusted width position through a broadcast mechanism, and adjusting the features after the element-wise multiplication operation according to a fourth preset size to obtain the third final output feature.

[0171] The input processing module 403 inputs the third final output feature and weight coefficients into the cross-space aggregation learning module and outputs the first final output feature, including: performing a two-dimensional pooling operation on the third final output feature to obtain pooled channel information; inputting the pooled channel information into the Softmax function layer and adjusting the output features according to a fifth preset size to obtain corresponding weight information; performing matrix multiplication and size adjustment on each first intermediate feature and weight information in sequence to obtain a first spatial feature; performing matrix multiplication and size adjustment on the third final output feature and weight coefficients in sequence to obtain a second spatial feature; performing element-wise addition on the first spatial feature and the second spatial feature, and inputting the feature after element-wise addition into the Softmax function layer to output a fourth final output feature; and performing element-wise multiplication on the fourth final output feature and each first feature to obtain the first final output feature.

[0172] The input processing module 403 inputs each second feature into the time-frequency co-processed spectral enhancement module and outputs the second final output feature, including: dividing each second feature according to a preset ratio of the number of channels corresponding to each second feature to obtain a fourth intermediate feature and a fifth intermediate feature; inputting the fourth intermediate feature into the internal information extraction submodule to output a sixth intermediate feature; inputting the fifth intermediate feature into the hidden minute details submodule to output a seventh intermediate feature; cascading the sixth and seventh intermediate features in the channel dimension to obtain an eighth intermediate feature, and sequentially performing two-dimensional pooling operations and inputting the eighth intermediate feature into Softma The x-function layer outputs the weights of each channel corresponding to the eighth intermediate feature. The eighth intermediate feature and the weights of each channel are multiplied to obtain the optimized feature. Max pooling and mean pooling operations are performed on each second feature to obtain the ninth and tenth intermediate features. The ninth and tenth intermediate features are concatenated and then fed into a 3×3 convolutional layer and a softmax function layer to obtain spatial information features. The spatial information features and each second feature are multiplied element-wise to obtain spatially enhanced features. The spatially enhanced features and the optimized feature are added element-wise to obtain the second final output feature.

[0173] The input processing module 403 inputs the fourth intermediate feature into the internal information extraction submodule and outputs the sixth intermediate feature, including: performing a 3×3 grouped convolution on the fourth intermediate feature according to a preset number of groups to obtain the eleventh intermediate feature; inputting the fourth intermediate feature into a 1×1 convolutional layer to output the twelfth intermediate feature; inputting the fourth intermediate feature into a 3×3 convolutional layer to output the thirteenth intermediate feature; performing a Fourier transform on the thirteenth intermediate feature to obtain the frequency feature; multiplying the learning parameters and the frequency feature to obtain the spectral component, and performing an inverse Fourier transform on the spectral component to obtain the time-domain feature; and performing element-wise addition on the eleventh intermediate feature, the twelfth intermediate feature, and the time-domain feature to obtain the sixth intermediate feature.

[0174] The input processing module 403 inputs the fifth intermediate feature into the hidden minute details submodule and outputs the seventh intermediate feature, including: inputting the fifth intermediate feature into a 3×3 convolutional layer and outputting the fourteenth intermediate feature; performing element-wise addition on the fourteenth intermediate feature and the fifth intermediate feature to obtain the seventh intermediate feature.

[0175] It should be noted that the above description of the remote sensing image classification device embodiment based on knowledge-guided spatial-spectral collaborative representation is similar to the description of the method embodiment described above, and has similar beneficial effects. For technical details not disclosed in the embodiments of the remote sensing image classification device based on knowledge-guided spatial-spectral collaborative representation of this invention, please refer to the description of the method embodiment of this invention for understanding.

[0176] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A remote sensing image classification method based on knowledge-guided spatial-spectral collaborative representation, characterized in that, The method includes: Obtain a panchromatic image, a multispectral image, and a class label image corresponding to the multispectral image, wherein the class label image includes multiple land cover categories; The panchromatic image and the multispectral image are respectively normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, respectively, and outputting multiple first features and multiple second features. Training datasets and test datasets are selected from various land cover categories according to preset rules. Each first feature is sequentially input into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module to output the first final output feature. Each second feature is input into the time-frequency co-processing spectral enhancement module to output the second final output feature. The spatial location enhancement learning module includes a pooling layer, a 1×1 convolutional layer, and a sigmoid activation function layer. The cross-spatial aggregation learning module includes a two-dimensional pooling layer and a softmax function layer. The time-frequency co-processing spectral enhancement module includes an inner information extraction submodule, a hidden minute detail submodule, a two-dimensional pooling layer, a softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The inner information extraction submodule includes a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule includes a 3×3 convolutional layer. The first final output feature and the second final output feature are sequentially input into a global average pooling layer and a fully connected layer to construct a classification model; The classification model is trained using the training dataset to obtain a trained classification model; The test dataset is input into the trained classification model, and the trained classification model is used to classify the test dataset and output the classification result.

2. The method according to claim 1, characterized in that, The panchromatic image and the multispectral image are respectively normalized, divided into neighborhoods, and input into two 3×3 convolutional layers, which output multiple first features and multiple second features respectively. Training and test datasets are selected from various land cover categories according to preset rules, including: The panchromatic image and the multispectral image are normalized to obtain a normalized panchromatic image and a normalized multispectral image. The normalized panchromatic image is divided according to a first preset neighborhood to obtain multiple divided panchromatic images; The normalized multispectral image is divided according to a second preset neighborhood to obtain multiple divided multispectral images; The multiple panchromatic images and the multiple multispectral images are respectively input into two 3×3 convolutional layers, and the outputs multiple first features corresponding to the multiple panchromatic images and multiple second features corresponding to the multiple multispectral images. According to the preset number of samples, two sample land features are selected from each of the land feature categories. The first sample land feature is used as the training dataset, and the second sample land feature is labeled as the test dataset.

3. The method according to claim 1, characterized in that, The process of sequentially inputting each first feature into the preprocessing module, the spatial location enhancement learning module, and the cross-spatial aggregation learning module to output a first final output feature, and inputting each second feature into the time-frequency collaborative spectral enhancement module to output a second final output feature, includes: Each first feature is input into the preprocessing module, and the weight coefficients between each channel corresponding to each first feature are output. The preprocessing module is a module including a 3×3 convolutional layer and a Softmax function layer. The first features are input into the spatial location enhancement learning module, and the third final output feature is output. The third final output feature and the weight coefficients are input into the cross-space aggregation learning module, and the first final output feature is output. The second features are input into the time-frequency coordinated spectral enhancement module, and the second final output features are output.

4. The method according to claim 3, characterized in that, The step of inputting each first feature into the preprocessing module and outputting the weight coefficients between each channel corresponding to each first feature includes: Each first feature is input into a 3×3 convolutional layer, and each feature output by the 3×3 convolutional layer is adjusted according to a first preset size and a second preset size to obtain each first intermediate feature and each second intermediate feature. Each first intermediate feature and each second intermediate feature are multiplied by a matrix to obtain each third intermediate feature. Each third intermediate feature is represented in matrix form, and each matrix includes multiple rows of matrix elements. The multi-row matrix elements in each of the third intermediate features are added row by row to obtain the features of each variable. The features of each variable are input into the Softmax function layer, and the weight coefficients between each channel corresponding to each variable are output.

5. The method according to claim 4, characterized in that, The step of inputting each of the first features into the spatial location enhancement learning module and outputting a third final output feature includes: Each of the first features is pooled along the height and width directions of the panchromatic image to obtain height features and width features; The height and width features are concatenated and input into a 1×1 convolutional layer to output the concatenated features. The cascaded features are separated, and the two separated features are input into the Sigmoid activation function layer respectively, outputting the position in height and the position in width; The position on the width is adjusted according to a third preset dimension to obtain the position on the adjusted width; Through a broadcast mechanism, element-wise multiplication is performed on each of the first features, the position at the height, and the position at the adjusted width. The features after the element-wise multiplication are then adjusted according to a fourth preset size to obtain the third final output feature.

6. The method according to claim 5, characterized in that, The step of inputting the third final output feature and the weight coefficients into the cross-space aggregation learning module and outputting the first final output feature includes: The third final output feature is subjected to two-dimensional pooling to obtain the pooled channel information. The pooled channel information is input into the Softmax function layer, and the output features are adjusted according to the fifth preset size to obtain the corresponding weight information. Perform matrix multiplication and size adjustment on each first intermediate feature and the weight information in sequence to obtain the first spatial feature; The third final output feature and the weight coefficient are sequentially subjected to matrix multiplication and size adjustment to obtain the second spatial feature; Element-wise addition is performed on the first spatial feature and the second spatial feature, and the feature after element-wise addition is input into the Softmax function layer to output the fourth final output feature; The fourth final output feature is multiplied element-wise with each of the first features to obtain the first final output feature.

7. The method according to claim 3, characterized in that, The step of inputting each of the second features into the time-frequency coordinated spectral enhancement module and outputting the second final output feature includes: According to a preset ratio of the number of channels corresponding to each of the second features, the second features are divided to obtain the fourth intermediate feature and the fifth intermediate feature; The fourth intermediate feature is input into the internal information extraction submodule, and the sixth intermediate feature is output. The fifth intermediate feature is input into the hidden minute detail submodule, and the seventh intermediate feature is output. The sixth and seventh intermediate features are concatenated along the channel dimension to obtain the eighth intermediate feature. The eighth intermediate feature is then subjected to two-dimensional pooling and input into a Softmax function layer. The weights of each channel corresponding to the eighth intermediate feature are output. The weights of the eighth intermediate feature and each channel are multiplied to obtain the optimized feature. Max pooling and mean pooling operations are performed on each of the second features to obtain the ninth and tenth intermediate features. The ninth and tenth intermediate features are concatenated and then sequentially input into a 3×3 convolutional layer and a Softmax function layer to obtain spatial information features. The spatial information features and each of the second features are multiplied element-wise to obtain spatial enhancement features. The spatial enhancement feature and the optimized feature are added element-wise to obtain the second final output feature.

8. The method according to claim 7, characterized in that, The step of inputting the fourth intermediate feature into the internal information extraction submodule and outputting the sixth intermediate feature includes: According to the preset number of groups, the fourth intermediate feature is subjected to a 3×3 grouped convolution to obtain the eleventh intermediate feature; The fourth intermediate feature is input into a 1×1 convolutional layer, and the twelfth intermediate feature is output. The fourth intermediate feature is input into a 3×3 convolutional layer, and the thirteenth intermediate feature is output. Perform a Fourier transform on the thirteenth intermediate feature to obtain the frequency feature; The learning parameters are multiplied by the frequency features to obtain the spectral components, and the spectral components are then subjected to an inverse Fourier transform to obtain the time-domain features. The sixth intermediate feature is obtained by performing element-wise addition on the eleventh intermediate feature, the twelfth intermediate feature, and the time-domain feature.

9. The method according to claim 7, characterized in that, The step of inputting the fifth intermediate feature into the hidden minute detail submodule and outputting the seventh intermediate feature includes: The fifth intermediate feature is input into a 3×3 convolutional layer, and the fourteenth intermediate feature is output. The seventh intermediate feature is obtained by performing element-wise addition on the fourteenth intermediate feature and the fifth intermediate feature.

10. A remote sensing image classification device based on knowledge-guided spatial-spectral collaborative representation, characterized in that, The device includes: The acquisition module is used to acquire panchromatic images, multispectral images, and class label images corresponding to the multispectral images, wherein the class label images include multiple land cover categories; The processing and selection module is used to perform normalization processing, neighborhood division and input into two 3×3 convolutional layers on the panchromatic image and the multispectral image respectively, outputting multiple first features and multiple second features respectively, and selecting training datasets and test datasets from various land cover categories according to preset rules. An input processing module is used to sequentially input each first feature into a preprocessing module, a spatial location enhancement learning module, and a cross-spatial aggregation learning module, outputting a first final output feature, and input each second feature into a time-frequency co-processing spectral enhancement module, outputting a second final output feature. The spatial location enhancement learning module includes a pooling layer, a 1×1 convolutional layer, and a sigmoid activation function layer. The cross-spatial aggregation learning module includes a two-dimensional pooling layer and a softmax function layer. The time-frequency co-processing spectral enhancement module includes an inner information extraction submodule, a hidden minute detail submodule, a two-dimensional pooling layer, a softmax function layer, a max pooling layer, a mean pooling layer, and a 3×3 convolutional layer. The inner information extraction submodule includes a 3×3 convolutional layer, a 1×1 convolutional layer, a Fourier transform layer, and an inverse Fourier transform layer. The hidden minute detail submodule includes a 3×3 convolutional layer. A construction module is used to sequentially input the first final output feature and the second final output feature into a global average pooling layer and a fully connected layer to construct a classification model; The training module is used to train the classification model using the training dataset to obtain a trained classification model. The classification module is used to input the test dataset into the trained classification model, use the trained classification model to classify the test dataset, and output the classification result.