Small-sample hyperspectral crop classification method coupled with multi-dimensional pyramid and Transform

Abstract

The invention provides a small-sample hyperspectral crop classification method coupled with a multi-dimensional pyramid and Transform, and relates to the technical field of hyperspectral remote sensing image classification. According to the technical scheme provided by the invention, the hyperspectral data features are extracted in parallel, and spectral feature expressions are mined from different angles. Multi-level receptive field features are captured through progressive expansion convolution to generate a multi-dimensional feature map, and receptive field expansion from local to global is achieved. The linear projection feature pyramid module integrates the multi-level features to obtain multi-scale fusion features. The multi-scale fusion features are input into a Transform module to execute self-attention calculation to obtain deep joint features, a global dependency relationship between long-distance pixels is established, long-range correlation crossing space and spectrum dimensions in a hyperspectral image is fully captured, the ability to distinguish crops with similar spectral features is significantly improved, and the method is suitable for large-scale popularization and application. And finally, the crop classification precision in a complex farmland scene can be improved.

Description

Technical Field

[0001] This application relates to the field of hyperspectral remote sensing image classification technology, specifically to a small-sample hyperspectral crop classification method that couples a multidimensional pyramid with a Transformer. Background Technology

[0002] Crop classification is one of the core tasks in the field of agricultural remote sensing, and it is of great significance for precision agricultural management, crop growth monitoring, yield prediction, and the rational allocation of agricultural resources. Accurate identification and classification of different crops in remote sensing imagery enables rapid surveys of large-scale farmland, providing a scientific basis for agricultural production decisions. Hyperspectral remote sensing imagery, due to its rich spectral information, can capture the reflectance characteristics of different ground features across hundreds of consecutive bands, exhibiting stronger ground feature identification capabilities compared to traditional multispectral imagery, thus demonstrating great potential in fine-grained crop classification.

[0003] In existing technologies, convolutional neural network (CNN)-based methods are widely used for crop classification tasks in hyperspectral remote sensing imagery. These methods extract spatial and spectral features through convolutional layers to identify different crop categories. However, in practical applications, especially when dealing with complex farmland scenarios characterized by "small sample sizes" (scarce labeled data), "different spectral characteristics of the same crop under different conditions," and "different spectral characteristics of different crop categories," CNNs rely on local convolutional kernels for feature extraction. This limits their receptive field and makes it difficult to establish global dependencies between distant pixels. Consequently, they lack sufficient discriminative power between different crop categories with highly similar spectral features, ultimately resulting in poor crop classification accuracy. Summary of the Invention

[0004] This application provides a small-sample hyperspectral crop classification method that couples a multidimensional pyramid with a Transformer, which can improve the accuracy of crop classification in complex farmland scenarios.

[0005] Firstly, this application provides a small-sample hyperspectral crop classification method that couples a multidimensional pyramid with a Transformer, the method comprising: Acquire hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions; The hyperspectral data features of the image patch are extracted by using multiple feature extraction branches set in parallel in the multidimensional feature extraction module. Progressive dilation convolution is used to capture multi-level receptive field features corresponding to each feature extraction branch from hyperspectral data features, generating multi-dimensional feature maps; By inputting the multidimensional feature map into a pre-constructed linear projection feature pyramid module, multi-scale fused features are obtained. The multi-scale fused features are input into the Transformer module, and a self-attention computation operation is performed on the multi-scale fused features to obtain deep joint features. The deep joint features are input into a fully connected layer for mapping, and the crop category labels corresponding to the image patches are output.

[0006] By employing the above technical solution, multiple feature extraction branches set in parallel within the multi-dimensional feature extraction module are used to extract hyperspectral data features, mining spectral feature representations from different perspectives. Progressive dilated convolution captures multi-level receptive field features to generate multi-dimensional feature maps, achieving receptive field expansion from local to global. The linear projection feature pyramid module integrates multi-level features to obtain multi-scale fusion features. These multi-scale fusion features are input into the Transformer module to perform self-attention computation, obtaining deep joint features. This overcomes the limitations of the local receptive field in convolutional neural networks, establishes global dependencies between distant pixels, fully captures long-range correlations across spatial and spectral dimensions in hyperspectral images, significantly improves the ability to distinguish crops with similar spectral features, and ultimately enhances crop classification accuracy in complex farmland scenarios.

[0007] Secondly, this application provides a small-sample hyperspectral crop classification system that couples a multidimensional pyramid with a Transformer, the system comprising: The image patch extraction module is used to acquire hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions. The feature extraction module is used to extract hyperspectral data features of image patches by using multiple feature extraction branches set in parallel in the multidimensional feature extraction module. The feature map generation module is used to capture multi-level receptive field features corresponding to each feature extraction branch from hyperspectral data features using progressive dilated convolution, and generate multi-dimensional feature maps. The feature fusion module is used to input the multi-dimensional feature map into the pre-constructed linear projection feature pyramid module to obtain multi-scale fused features; The feature processing module is used to input multi-scale fused features into the Transformer module, perform self-attention calculation on the multi-scale fused features, and obtain deep joint features; The output module is used to input deep joint features into the fully connected layer for mapping and output crop category labels corresponding to the image patches.

[0008] Thirdly, this application provides a computer storage medium that stores multiple instructions adapted for loading by a processor and executing any of the methods described above.

[0009] Fourthly, this application provides an electronic device including a processor, a memory, and a transceiver. The memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform any of the methods described above.

[0010] In summary, the beneficial effects of the technical solution of this application include: By employing the above technical solution, multiple feature extraction branches set in parallel within the multi-dimensional feature extraction module are used to extract hyperspectral data features, mining spectral feature representations from different perspectives. Progressive dilated convolution captures multi-level receptive field features to generate multi-dimensional feature maps, achieving receptive field expansion from local to global. The linear projection feature pyramid module integrates multi-level features to obtain multi-scale fusion features. These multi-scale fusion features are input into the Transformer module to perform self-attention computation, obtaining deep joint features. This overcomes the limitations of the local receptive field in convolutional neural networks, establishes global dependencies between distant pixels, fully captures long-range correlations across spatial and spectral dimensions in hyperspectral images, significantly improves the ability to distinguish crops with similar spectral features, and ultimately enhances crop classification accuracy in complex farmland scenarios. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating a small-sample hyperspectral crop classification method that couples a multidimensional pyramid with a Transformer, as provided in an embodiment of this application. Figure 2 This is a flowchart illustrating another small-sample hyperspectral crop classification method coupled with a multidimensional pyramid and a Transformer, provided in an embodiment of this application. Figure 3 This is a schematic diagram of an architecture for multi-level receptive field feature extraction based on progressive dilated convolution provided in an embodiment of this application; Figure 4 This is a color comparison image of the model effect in a real-world scenario provided by an embodiment of this application; Figure 5 This is a color comparison image of the model effect in another real-world scenario provided by the embodiments of this application; Figure 6 This is a schematic diagram of the structure of a small-sample hyperspectral crop classification system that couples a multidimensional pyramid and a Transformer, according to an embodiment of this application. Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0012] Explanation of reference numerals in the attached figures: 700, electronic device; 701, processor; 702, communication bus; 703, user interface; 704, network interface; 705, memory. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0014] In the description of the embodiments of this application, words such as "illustrative," "for example," or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "illustrative," "for example," or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Rather, the use of words such as "illustrative," "for example," or "for example" is intended to present the relevant concepts in a specific manner.

[0015] Please see Figure 1 This document presents a flowchart illustrating a small-sample hyperspectral crop classification method coupled with a multidimensional pyramid and a Transformer, as provided in an embodiment of this application. This method can be implemented using a computer program, a microcontroller, or run on a small-sample hyperspectral crop classification system based on the von Neumann architecture and coupled with a multidimensional pyramid and a Transformer. The computer program can be integrated into the application or run as a standalone utility application. The specific steps of the small-sample hyperspectral crop classification method coupled with a multidimensional pyramid and a Transformer are described in detail below.

[0016] S101: Acquire the hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions; Hyperspectral image data refers to remote sensing image data containing dozens to hundreds of continuous spectral bands. Each pixel records spectral reflectance information from multiple bands, from visible light to near-infrared. For example, a hyperspectral image of a farmland area may contain 270 bands with wavelengths ranging from 400nm to 2500nm.

[0017] An image block represents a three-dimensional data cube extracted from a certain pixel according to a preset spatial size. It has a fixed number of pixels in the length and width directions, and maintains the same number of bands as the original dataset in the band dimension.

[0018] Spectral dimension is used to represent the spectral response characteristics at different wavelengths, reflecting the material composition and physicochemical properties of ground objects. For example, different crops have different chlorophyll absorption peak positions in specific bands.

[0019] Spatial dimension refers to the spatial distribution information of image patches on a two-dimensional plane, including spatial structural features such as the texture, shape, and neighborhood relationship of crops. For example, cornfields and rice paddies exhibit different patterns in spatial texture.

[0020] Specifically, firstly, hyperspectral image data of the target area is acquired using remote sensing sensors or data storage platforms. This data contains complete spectral and spatial information of the study area. Then, image patch extraction is performed on the acquired image data. This extraction process employs a sliding window mechanism, where the window moves sequentially across the image's spatial plane. At each pixel position, a fixed-size spatial neighborhood window is extracted, centered on that pixel. This window encompasses all neighboring pixels surrounding the central pixel, while preserving all spectral band data for each pixel within the window. This process iterates through each pixel to be classified in the image, extracting corresponding image patches for each pixel. These image patches contain both the structural information of the spatial neighborhood and complete spectral curve information, forming the basic processing units for subsequent feature extraction and classification.

[0021] In some embodiments, image patch extraction can be achieved in several ways: Optionally, first, the size parameter of the spatial window is determined, then each pixel position of the hyperspectral image is traversed, and for each position, a square or rectangular spatial neighborhood is formed by expanding outwards from that pixel. Next, the complete spectral band data of all pixels within that neighborhood is extracted from the original image, and finally, the extracted three-dimensional data block is stored as the image patch corresponding to the central pixel. It is understood that other window extraction strategies or data organization methods can also be used to obtain image patches, which are not limited here.

[0022] S102: Use multiple feature extraction branches set in parallel in the multidimensional feature extraction module to extract hyperspectral data features of image patches respectively; The multidimensional feature extraction module refers to a network structure module containing multiple parallel feature extraction channels, used to extract feature information of image patches from different dimensional perspectives. Parallel configuration means that multiple independent feature extraction branches are configured simultaneously within the module. These branches are structurally independent but run synchronously in time, processing the same input data together. A feature extraction branch is an independent network channel that uses a specific convolutional method to process data of a specific dimension. Each branch is responsible for performing feature extraction operations on the image patch from a particular dimensional perspective. Hyperspectral data features refer to the data representation extracted from the image patch through the feature extraction branches. These features contain discriminative information about the image patch from different dimensions; for example, features extracted by one branch might focus on spectral variation patterns, while features extracted by another branch might focus on spatial texture structures.

[0023] Specifically, the image patches extracted in the previous step are simultaneously input into the parallel branches of the multidimensional feature extraction module, with each branch receiving the complete image patch data as input. Each branch performs feature extraction on the image patch according to its designed convolutional kernel type and convolution method. Different branches use convolutional kernels of different dimensions; for example, some branches use one-dimensional convolutional kernels to extract features along the spectral dimension, some use two-dimensional convolutional kernels to extract features in the spatial dimension, and some use three-dimensional convolutional kernels to extract features in both the spectral and spatial dimensions simultaneously. Each branch computes in parallel within independent network channels, using its own convolutional layer structure and learnable weight parameters to perform convolution operations on the input data, extracting discriminative features from its respective dimensional perspective. Due to the parallel computing architecture, the feature extraction processes of each branch are performed simultaneously without blocking each other, improving computational efficiency and achieving the goal of capturing features from multiple dimensions simultaneously. After processing by each branch, hyperspectral data features from different dimensions are obtained, and these features characterize the data properties of the image patch from their respective perspectives.

[0024] In some embodiments, multi-branch feature extraction can be implemented in several ways: Optionally, the image patch is first copied multiple times and fed into various parallel branches. Then, in the first branch, a one-dimensional convolutional kernel is used to perform convolution operations along the spectral dimension to extract spectral features. In the second branch, a two-dimensional convolutional kernel is used to perform convolution operations on the spatial plane to extract spatial features. In the third branch, a three-dimensional convolutional kernel is used to simultaneously process spectral and spatial dimensions to extract joint features. Finally, the features output by each branch are collected as hyperspectral data features. Optionally, a parallel network architecture containing multiple independent branches is first constructed. Then, different numbers of convolutional layers and convolutional kernel sizes are configured for each branch to adapt to the feature extraction requirements of different dimensions. Next, the forward propagation computation process of each branch is started simultaneously, and each branch independently performs convolution, activation, and other operations. Finally, the multi-dimensional features are summarized after each branch has completed its computation. It is understood that other parallel computing methods or branch configuration strategies can also be used to achieve multi-dimensional feature extraction, which is not limited here.

[0025] S103: Use progressive dilation convolution to capture the multi-level receptive field features corresponding to each feature extraction branch from the hyperspectral data features, and generate a multi-dimensional feature map; Progressive dilated convolution refers to a dilated convolution operation where the dilation rate parameter gradually increases as the network depth increases. This expands the receptive field by inserting intervals of different sizes between convolution kernel elements. The dilation rate controls the spacing between kernel elements; progressively increasing the dilation rate means that shallow layers use smaller intervals to capture local details, while deeper layers use larger intervals to capture global context. The receptive field represents the range of input data that the convolution operation can cover and perceive. Dilated convolution expands the receptive field without increasing the number of parameters by using interval sampling. Multi-level receptive field features refer to features extracted by stacking convolutional layers with different dilation rates, covering different scales and containing multiple levels of feature information, from fine-grained local to coarse-grained global. Multidimensional feature maps represent the feature representations generated after progressive dilated convolution processing for each feature extraction branch. For example, a branch undergoing multiple dilated convolutions may generate a feature map containing information from multiple receptive field levels.

[0026] Specifically, progressive dilated convolution operations are applied to the hyperspectral data features output by each feature extraction branch, constructing a multi-layered stacked dilated convolutional network structure within each branch. In the shallow dilated convolutional layers, a small dilation rate parameter is set, resulting in small spacing between convolutional kernel elements. The convolution operation is performed within a relatively compact neighborhood, thereby capturing local details and correlation patterns between nearby elements. As the network depth increases, the dilation rate parameter of subsequent dilated convolutional layers is gradually increased, widening the spacing between convolutional kernel elements. This allows the convolution operation to sample across a larger range, capturing global contextual information and dependencies between distant elements over a broader area. Through this progressively increasing dilation rate, the network can capture receptive field features at different scales at different levels, preserving fine-grained local features in shallow layers and acquiring coarse-grained global features in deeper layers, forming a multi-layered feature representation from fine to coarse. After processing by progressive dilated convolution, each branch generates its own multi-dimensional feature map, which integrates rich feature information extracted from multiple receptive field levels by each branch.

[0027] In some embodiments, feature extraction via progressive dilated convolution can be implemented in various ways: Optionally, a stacked structure containing multiple dilated convolutional layers is first constructed in each branch. Then, a small dilation rate is set for the first dilated convolutional layer, for example, starting from a base value, and dilation convolution is performed on the input hyperspectral data features to obtain the first layer features. Next, an increased dilation rate is set for the second layer, and dilation convolution is performed on the first layer features to obtain the second layer features. This process is repeated layer by layer, increasing the dilation rate and performing convolution operations. Finally, the multi-level receptive field features obtained from each layer are integrated to form a multi-dimensional feature map. It is understood that other dilation rate setting strategies or feature integration methods can also be used to capture multi-level receptive field features, which are not limited here.

[0028] S104: Input the multi-dimensional feature map into the pre-constructed linear projection feature pyramid module to obtain multi-scale fused features; The linear projection feature pyramid module refers to a network module that constructs multi-scale feature representations through linear projection operations, organizing features at different scale levels using a pyramid structure. Linear projection representation achieves a linear mapping operation that transforms the feature space by performing matrix multiplication between the weight matrix and the input features. This operation maps the input features from the original feature space to a new target feature space. The feature pyramid representation organizes features into a pyramid-like hierarchical structure. The pyramid contains multiple scale levels, with different levels corresponding to different feature resolutions or levels of abstraction. For example, the bottom layer of the pyramid contains high-resolution detailed features, while the top layer contains low-resolution abstract features. Multi-scale fusion features refer to the comprehensive feature representation obtained by fusing features from different scale levels in the pyramid. This feature simultaneously contains information from multiple scale levels, including both detailed and global information.

[0029] Specifically, the multidimensional feature map generated in the previous step is input into a pre-constructed linear projection feature pyramid module for processing. The module first performs a linear projection transformation on the input multidimensional feature map, mapping the features to different feature subspaces through multiple linear projection layers. Each linear projection layer uses different weight matrix parameters. The linear projection calculation is achieved by performing matrix multiplication between the input feature vector and the weight matrix, yielding the projected feature vector. By using different projection parameters, the module can generate feature representations at multiple scales. These features are organized into a pyramid structure, forming a feature pyramid containing multiple scale levels. Different levels of the pyramid obtain different feature resolutions through different projection parameters and possible downsampling or upsampling operations. The bottom layer retains a higher feature resolution to preserve detailed information, the top layer reduces the feature resolution to obtain more abstract global information, and the middle layers fall between the two. Then, the module performs a fusion operation on the features at each level of the pyramid, integrating feature information from different scales into a unified feature representation. The fusion operation can be achieved through feature concatenation, weighted summation, or other fusion strategies. The resulting multi-scale fused feature integrates information from each level of the pyramid, containing both fine-grained local details and coarse-grained global context.

[0030] S105: Input the multi-scale fused features into the Transformer module, perform self-attention calculation on the multi-scale fused features, and obtain deep joint features; The Transformer module refers to a deep learning network module based on a self-attention mechanism, capable of modeling global dependencies between elements in an input feature sequence. The self-attention computation operation represents the core computational mechanism of the Transformer, achieving global information interaction and aggregation within the sequence by calculating the relevance weights between each element in the feature sequence and all other elements in the sequence. Self-attention computation involves mapping the input features to query vectors, key vectors, and value vectors through three linear transformations, then calculating the similarity between the query and key to obtain attention weights, and finally using the attention weights to perform a weighted sum of the value vectors to obtain the output features. Deep joint features refer to the feature representation obtained after the self-attention computation of the Transformer module. This feature integrates global information from all positional elements in the input feature sequence, fully modeling long-range dependencies between elements. For example, if two distant positions in the feature sequence have a strong correlation, the self-attention mechanism can capture and integrate this correlation.

[0031] Specifically, the multi-scale fused features obtained in the previous step are input into the Transformer module for processing. The Transformer module first converts the input multi-scale fused features into a sequence, flattening the spatial locations of the features into different positions within the sequence, making the features a one-dimensional positional sequence. Then, a self-attention calculation operation is performed on this feature sequence. The self-attention calculation process includes the following stages: First, the input feature sequence is mapped to a query matrix, a key matrix, and a value matrix through three independent linear transformation layers. The linear transformation is achieved by matrix multiplication of the input features with the weight matrix. Each feature position is mapped to its corresponding query vector, key vector, and value vector. Next, the similarity score between the query matrix and the key matrix is ​​calculated. The similarity calculation is achieved by matrix multiplication of the transpose of the query matrix and the key matrix. The result of the matrix multiplication is an attention score matrix, where each element represents the correlation between a query position and a key position; a larger element value indicates a stronger correlation between the features at the two positions. Then, the attention score matrix is ​​normalized to obtain the attention weight matrix. Normalization is achieved by applying a normalization function to each row of the score matrix, ensuring that the sum of the weights in each row is one. The normalized weights represent the attention allocation of each query position to all key positions. Finally, the calculated attention weight matrix is ​​used to perform a weighted summation of the value matrix. This is achieved through matrix multiplication between the attention weight matrix and the value matrix. The output feature at each position in the calculation result is the weighted sum of the value vectors of all positions according to their corresponding attention weights. Through this self-attention mechanism, the output feature at each position incorporates information from all positions in the sequence, achieving global information interaction and aggregation. After self-attention calculation, a deep joint feature is obtained, which fully integrates the global contextual information and inter-position dependencies in the multi-scale fusion feature.

[0032] In some embodiments, self-attention computation can be implemented in several ways: Optionally, firstly, the multi-scale fused features are flattened into a one-dimensional sequence and positional encoding is added to preserve positional information. Then, the sequence features are linearly transformed through three linear layers to obtain a query matrix, a key matrix, and a value matrix. Next, the matrix product of the query matrix and the transpose of the key matrix is ​​calculated and divided by a scaling factor to obtain an attention score. Then, a normalization function is applied to the attention score to obtain an attention weight matrix. Finally, the attention weight matrix is ​​multiplied by the value matrix to obtain the self-attention output, which is the deep joint feature. It is understood that other self-attention computation variants or Transformer architectures can also be used to extract deep joint features, which is not limited here.

[0033] S106: Input the deep joint features into the fully connected layer for mapping, and output the crop category label corresponding to the image patch.

[0034] In this context, a fully connected layer refers to a network layer structure in a neural network where each input node and each output node has connection weights. It achieves a linear mapping transformation from the input feature space to the output label space through a weight matrix. This mapping represents the process of transforming the input deep joint feature vector into an output category space representation. The fully connected layer performs this linear transformation using learned weight and bias parameters, mapping the high-dimensional feature vector to an output vector corresponding to the number of categories. Crop category labels are used to indicate the crop type to which the image patch belongs. Each label corresponds to a specific crop type, such as rice, wheat, corn, or soybean. The labels are output in the form of a category number or category name, identifying the classification of the image patch.

[0035] Specifically, the deep joint features obtained in the previous step are input into a fully connected layer for classification mapping. The fully connected layer receives the deep joint feature vector as input and performs matrix multiplication with the input feature vector using a weight matrix. During the calculation, each dimension of the input feature vector is multiplied one by one with the elements of the corresponding row of the weight matrix, and the results are accumulated. Then, the corresponding bias term is added to obtain the value of that output node. The number of output nodes in the fully connected layer equals the total number of crop categories, with each output node corresponding to one crop category. After matrix multiplication and bias summation, the original output vector is obtained. This output vector is transformed by an activation function, which converts the original output value into a category probability distribution, such that the value of each element in the output vector is between zero and one, and the sum of all element values ​​is one. Each element represents the probability score of the image patch belonging to the corresponding crop category. Then, the category with the highest probability score is selected from the probability distribution as the classification result of the image patch, and the label corresponding to that category is output as the crop category label of the image patch. The output category label indicates the crop type to which the pixel centered on the image patch belongs, completing the end-to-end classification and recognition process from the original hyperspectral image patch to the crop category.

[0036] Based on the above embodiments, as an optional implementation method, the feature extraction branches include a 1D spectral branch, a 2D spatial branch, and a 3D spatial-spectral joint branch. In step S102, multiple feature extraction branches set in parallel in the multidimensional feature extraction backbone are used to extract the hyperspectral data features of the image patch respectively. Specifically, this can be achieved through the following steps S201-S205.

[0037] For the specific implementation of the three feature extraction branches, please refer to [link / reference]. Figure 2 , Figure 2This document presents a flowchart illustrating another small-sample hyperspectral crop classification method coupled with a multidimensional pyramid and a Transformer, as provided in an embodiment of this application. The diagram shows the complete processing flow from raw hyperspectral data to the final classification result. First, starting from the raw hyperspectral data, fixed-size image patches are constructed using the length and width dimensions of random pixels. These image patches simultaneously contain spatial neighborhood information and complete spectral band information. Next, the dataset is divided into training, validation, and test sets to provide a data foundation for model training and evaluation. Then, the image patches are input in parallel into three feature extraction branches: a 1D branch, a 2D branch, and a 3D branch. These three branches extract multidimensional features from the hyperspectral data from the spectral dimension, the spatial dimension, and the spectral-spatial joint dimension, respectively. Within each branch, a receptive field pyramid is constructed using progressively dilated convolution. An increasing dilation rate sequence captures multi-level receptive field features from local to global perspectives, generating a feature map containing multi-scale information. Subsequently, the multidimensional feature maps extracted by the three branches are concatenated to form a comprehensive feature representation that integrates spectral, spatial, and spatial-spectral joint information. Next, the fused multidimensional features pass through a feature pyramid fusion module. This module reduces the feature dimensionality through linear projection and then constructs a feature pyramid using depthwise separable convolutional kernels of different sizes, further enhancing the fusion effect of multi-scale features. Finally, the fused features are input into the Transformer module, which extracts global semantic associations in the spectral-spatial dimension through a self-attention mechanism, capturing long-distance dependencies and global contextual information within the features to obtain deep joint features. These features are then mapped to the crop category space through fully connected layers, outputting the final classification result. The entire process embodies a progressive feature learning strategy from multidimensional feature extraction and multi-scale feature fusion to global semantic modeling, fully leveraging the discriminative information of hyperspectral data in both the spectral and spatial domains, and achieving high-precision classification of small sample crops.

[0038] Based on the above architecture, the following methods can be used for model initialization and dataset construction.

[0039] Setting the image patch size: The image patch size refers to the number of pixels in the length and width directions of the spatial neighborhood window extracted from the hyperspectral image. This parameter determines the size of the spatial range contained in each image patch. In this application, 11×11 can be selected as the image patch input size for example.

[0040] This step is performed during the initialization phase before model training begins, serving as a crucial parameter setting stage for data preprocessing and model configuration. Specifically, several factors need to be weighed to determine the optimal image patch size parameter. First, from a computational cost perspective, the choice of image patch size directly affects the model's computational complexity and operational efficiency. Excessively large sizes result in individual image patches containing too many pixels, significantly increasing the amount of data that subsequent feature extraction networks need to process, leading to prolonged training and inference times, increased memory consumption, and potentially exceeding hardware resource limitations. Second, from an experimental accuracy perspective, the image patch size needs to be large enough to contain sufficient discriminative information. Too small a size may result in image patches containing only limited spatial information, failing to provide sufficient feature basis for the classifier and affecting classification accuracy.

[0041] Dataset Construction: This step is performed after completing the model architecture construction and parameter initialization, but before formally starting model training. It is a crucial step in preparing experimental data. Specifically, the first step is to select samples and partition the hyperspectral image dataset. To verify the model's performance under small sample conditions, a limited sample experimental setup is adopted. Dataset partitioning uses random sampling for multiple repeated experiments, evaluating the stability and reliability of model performance by randomly selecting multiple different sample combinations. In each experiment, a limited number of pixel locations are randomly selected for each crop category in the image as training samples for that category. Image patches are extracted for these pixel locations and labeled with category labels to form the training set samples. The number of training set samples is set to include a small number of pixels for each category to ensure experiments are conducted under small sample learning scenarios. Then, a larger number of pixel locations are randomly selected for each category to construct the validation set. The number of samples in the validation set is significantly larger than that in the training set, providing sufficient evaluation data for performance monitoring during model training. Image patches also need to be extracted and labeled with categories for the validation set samples. All remaining pixels in the dataset, excluding the training and validation sets, are designated as the test set. The test set contains the majority of pixel samples in the image that were not used for training and validation, and is used to comprehensively evaluate the model's classification performance and generalization ability. For example, this application uses the same number of training samples for the training, validation, and test sets of the dataset; that is, five sets of samples are randomly selected, with 20 pixels in each set for the training set, 300 pixels for the validation set, and the remaining pixels for the test set, to verify the model's performance under conditions of a small sample size.

[0042] S201: Input the image blocks into the 1D spectral branch, the 2D spatial branch, and the 3D spatial-spectral joint branch respectively; The 1D spectral branch is a feature extraction channel specifically for processing spectral dimensional information. It extracts correlation features between different bands along the spectral band direction through one-dimensional convolution operations. The 2D spatial branch is a feature extraction channel for processing spatial dimensional information. It extracts spatial texture and structural features in the image's aspect ratio plane through two-dimensional convolution operations. The 3D spatial-spectral joint branch is a feature extraction channel that simultaneously processes spectral and spatial dimensional information. It extracts coupled features in the spectral-spatial three-dimensional space through three-dimensional convolution operations. The spectral branch focuses on the spectral curve variation patterns between bands, the spatial branch focuses on the spatial distribution patterns of neighboring pixels, and the spatial-spectral joint branch considers the synergistic relationship between the two.

[0043] In practice, the image patch is first converted to a different data format, adjusting the original 3D tensor to fit the input requirements of each branch. For the one-dimensional spectral branch, the spatial dimension of the image patch is flattened to preserve complete spectral information, making it suitable for one-dimensional convolution operations. For the two-dimensional spatial branch, the spectral dimension is used as an input channel or dimensionality reduction is performed, focusing on preserving the height and width information of the spatial dimension. For the three-dimensional spatial-spectral joint branch, the complete spatial-spectral 3D structure of the image patch is maintained without dimensionality compression. The three branches employ independent convolutional layers and parameter weights, and feature extraction is performed simultaneously through a parallel computing mechanism, ensuring computational efficiency while achieving the goal of extracting complementary features from different perspectives.

[0044] S202: Use a one-dimensional convolution kernel with a 1D spectral branch to extract spectral features from the image patch to obtain the first data features; The one-dimensional convolution kernel is a filter that performs sliding convolution operations only in the spectral dimension, capturing correlation patterns between bands by sliding across consecutive spectral bands. The first data feature is the spectral domain feature representation output by the one-dimensional spectral branch, containing correlation and discriminative information between different spectral bands. One-dimensional convolution operations can effectively extract spectral response relationships between adjacent or spaced bands, such as the differences in vegetation characteristics between the visible and near-infrared bands. These differences constitute an important basis for distinguishing different crop types.

[0045] In practice, the input image patch is first reshaped into a format suitable for one-dimensional convolution processing, ensuring that the spectral dimension is the primary direction of convolution sliding. Then, the input is processed using a first-dimensional convolutional layer, where the convolution kernel slides along the spectral dimension, performing a weighted summation operation on consecutive bands at each location. The learned weight parameters capture the response patterns between bands. After the convolution operation, batch normalization is applied to stabilize the training process, and a non-linear activation function is used to enhance the expressive power of the features. Furthermore, a progressively dilated convolution strategy is employed to construct a spectral receptive field pyramid. By increasing the dilation rate layer by layer, the convolution kernel can extract features across larger spectral intervals, expanding the receptive field range without increasing the number of parameters. Shallow convolutions with small dilation rates capture local detail features of adjacent bands, while deep convolutions with large dilation rates capture global correlation features of distant bands. Finally, the first data feature is output, which contains multi-level spectral discrimination information from local to global perspectives.

[0046] S203: Use a 2D spatial branch of a two-dimensional convolution kernel to extract spatial dimension features from image patches to obtain second data features; Specifically, a two-dimensional convolutional kernel is a filter that performs sliding convolution operations on the height and width spatial dimensions of an image to extract spatial patterns within the image plane. The second data feature is the spatial domain feature representation output by the two-dimensional spatial branch, capturing texture, edge, shape, and structural information within the pixel neighborhood. By sliding on the spatial plane, two-dimensional convolution can identify spatial distribution patterns in local areas, such as the row and column structure of crop plantings, canopy texture features, or shape information of ground feature boundaries. These spatial features provide important contextual information for crop classification.

[0047] In practical implementation, firstly, the image patch undergoes spectral dimension reduction processing. Principal component analysis or band selection methods are used to compress high-dimensional spectral information into lower-dimensional feature channels, enabling efficient spatial processing of 2D convolution. Then, a first 2D convolutional layer is applied for preliminary spatial feature extraction. The convolutional kernel slides along the height and width of the image, performing a weighted summation of neighboring pixels at each spatial location to extract local spatial patterns. Convolution computation iterates through all spatial locations of the input feature map, applying a multiplication-addition operation between the convolutional kernel weights and the corresponding neighboring pixel values ​​at each location. Next, a spatial receptive field pyramid is constructed through multiple dilated convolutions. The dilation rate is increased layer by layer, allowing the convolutional kernel to extract features over a larger spatial range. Convolutional layers with small dilation rates focus on fine-grained local textures, while those with large dilation rates focus on coarse-grained regional structures. Batch normalization and activation functions are added after each convolutional layer to enhance non-linear expression. Finally, global pooling or feature flattening is used to compress the multi-dimensional spatial feature map into a feature vector, outputting the second data feature.

[0048] S204: Use a 3D convolutional kernel with a 3D spatial-spectral joint branch to extract spectral features from the image patch to obtain the first data feature; The three-dimensional convolution kernel is a filter that performs sliding convolution operations simultaneously in the height, width, and spectral dimensions, enabling it to model both spatial neighborhood relationships and spectral band correlations. The third data feature is the spectral-spatial coupling feature representation output from the three-dimensional spatial-spectral joint branch. This feature not only contains independent spatial and spectral information but, more importantly, captures the interaction and synergistic effects between the two. By sliding within three-dimensional space, three-dimensional convolution can identify joint response patterns of specific band combinations at specific spatial locations, such as the coupling characteristics of canopy spatial distribution and spectral reflectance of a crop at a specific growth stage.

[0049] In practical implementation, firstly, the image patch is preserved to maintain its complete spatial-spectral three-dimensional structure without compression in any dimension, ensuring that the 3D convolution can simultaneously access spatial neighborhood and spectral band information. Then, a first-layer 3D convolutional layer is applied for joint feature extraction. The convolution kernel slides simultaneously in the height, width, and spectral directions, performing convolution operations on each 3D local region. The convolution calculation generates an output value by summing the 3D weights of the convolution kernel with the corresponding 3D neighborhood data element-wise at each position in 3D space. This 3D operation can capture the joint variation patterns of spatially adjacent pixels across multiple spectral bands. Next, a multi-layered cascaded 3D dilated convolution is employed, setting dilation rates in both the spatial and spectral dimensions, and expanding the 3D receptive field through interval sampling. Shallow networks extract local fine-grained spatial-spectral coupling features, while deep networks extract global coarse-grained spatial-spectral correlation features. After each convolutional layer, batch normalization and activation functions are applied for nonlinear transformation. Residual connections preserve the original information of the shallow layers to prevent information loss. Finally, global 3D pooling is used for dimensionality reduction along all dimensions, outputting a third data feature vector.

[0050] S205: The first data feature, the second data feature, and the third data feature are determined as the hyperspectral data features of the image patch.

[0051] Specifically, this step establishes the compositional structure of hyperspectral data features, which comprises three parts: a first data feature from the spectral branch, a second data feature from the spatial branch, and a third data feature from the combined spatial-spectral branch. These three features characterize the image patch from the spectral, spatial, and combined spectral-spatial dimensions, respectively. Therefore, they are collectively identified as the hyperspectral data features of the image patch, comprehensively reflecting its data characteristics across different dimensions.

[0052] Based on the above embodiments, as an optional implementation method, step S103 uses progressive dilated convolution to capture the multi-level receptive field features corresponding to each feature extraction branch from the hyperspectral data features and generate a multi-dimensional feature map. Specifically, this can be achieved through the following steps S301-S304.

[0053] Please see details. Figure 3 , Figure 3 This is a schematic diagram of an architecture for multi-level receptive field feature extraction based on progressively dilated convolution, provided in an embodiment of this application. Figure 3 This paper demonstrates the complete processing flow from hyperspectral image patch input to multidimensional feature map generation. The architecture employs a multi-branch parallel processing strategy combined with progressive dilated convolution technology to achieve feature extraction of multi-level receptive fields. The entire architecture mainly consists of four core parts: three parallel feature extraction branches, a progressive dilated convolution module group, a feature fusion module, and an attention enhancement module.

[0054] like Figure 3 As shown in the top left corner, the input is a hyperspectral image patch extracted from its spatial neighborhood. The data is in the form of a three-dimensional tensor, with dimensions represented as H×W×C Channels. Here, H represents the height of the image patch in pixels, corresponding to the previously defined patch size; W represents the width of the image patch in pixels, also corresponding to the patch size; and C represents the number of spectral bands in the hyperspectral image, containing spectral information from multiple bands, from visible light to near-infrared. This input image patch is presented in the form of a colored cube, with different colored layers representing spectral information from different bands, reflecting the three-dimensional characteristics of hyperspectral data—a data structure of two-dimensional space plus one-dimensional spectrum.

[0055] Figure 3 The left-central area shows three parallel feature extraction branches: a 1D feature extraction branch, a 2D feature extraction branch, and a 3D feature extraction branch. These three branches simultaneously receive the same input hyperspectral image patch data and perform feature extraction in parallel.

[0056] The 1D feature extraction branch is located at the top, focusing on extracting spectral dimension features. As shown in the figure, the 1D branch contains a series of convolutional layer blocks of different colors, each representing a 1D convolutional layer with a specific dilation rate. These convolutional layers perform convolution operations along the spectral dimension to capture spectral correlations and spectral curve features between different bands. The 1D convolutional layers are configured with a progressive dilation strategy, gradually increasing the dilation rate from standard 1D convolutions, enabling the network to capture spectral dependencies from adjacent bands to distant bands. The plus sign in the branch indicates feature concatenation operations, and the circle symbol indicates other operations such as normalization or activation. Finally, this branch outputs a feature representation containing multi-level spectral receptive field information.

[0057] The 2D feature extraction branch is located in the middle, focusing on extracting spatial dimension features. As shown in the figure, the 2D branch also contains multiple convolutional layer blocks of different colors, each representing a 2D convolutional layer with a specific dilation rate. These convolutional layers perform two-dimensional convolution operations on the spatial plane of the image block to extract spatial texture, edges, and structural patterns. The 2D convolutional layers employ a progressive dilation configuration: standard 2D convolutions capture local spatial neighborhood features, small dilation rate 2D convolutions expand to the surrounding area to obtain medium-scale spatial structures, and large dilation rate 2D convolutions cover a wider spatial range to extract global spatial context. This branch achieves effective capture of spatial patterns at different scales by constructing multi-layered spatial receptive fields.

[0058] The 3D feature extraction branch is located below. This branch considers both spatial and spectral dimensions for joint feature extraction. As shown in the figure, the 3D branch contains multiple 3D convolutional layer blocks, each representing a 3D convolutional layer with a specific dilation rate. The convolutional kernels of these layers perform convolution operations simultaneously in the spatial and spectral dimensions, enabling the simultaneous capture of joint spatial-spectral patterns and correlations. The 3D convolutional layers gradually expand the receptive field in three-dimensional space through a progressive dilation strategy, extracting both local spatial-spectral joint detail features and acquiring large-scale spatial-spectral contextual information, thus fully utilizing the inherent three-dimensional structure of hyperspectral data.

[0059] Figure 3 The right-hand area shows in detail the specific configuration of progressive dilated convolution in each feature extraction branch.

[0060] Figure 3 The right-hand area shows in detail the specific configuration of progressive dilated convolution in each feature extraction branch, which corresponds to the implementation details of steps S301 and S302.

[0061] 1D Convolutional Dilation Sequence: As shown in the upper right corner of the figure, the 1D feature extraction branch contains multiple 1D convolutional layers, labeled as 1D Conv Dilation-0, 1D Conv Dilation-1, 1D Conv Dilation-2, 1D Conv Dilation-3, 1D Conv Dilation-4, and 1D Conv Dilation-5. Dilation-0 corresponds to the standard 1D convolutional layer, i.e., the aforementioned standard convolutional layer, with a dilation rate of 1, used to extract spectral features between adjacent bands. Dilation-1 to Dilation-5 correspond to 1D dilated convolutional layers with different dilation rates, i.e., the aforementioned multiple dilated convolutional layers. The dilation rates are set in an increasing sequence, such as [1, 2, 3, 4, 5, 6] or [1, 2, 4, 8, 16, 32]. A larger dilation rate allows the convolutional kernel to capture long-distance spectral dependencies across more bands. Each 1D convolutional layer is represented by a block of a different color. The color difference intuitively reflects the difference in the receptive field range of different layers, forming a multi-level spectral receptive field from local to global.

[0062] 2D Convolutional Dilation Sequence: As shown in the middle right of the figure, the 2D feature extraction branch contains multiple 2D convolutional layers, labeled 2D Conv Dilation-0 to Dilation-5. Dilation-0 is a standard 2D convolutional layer that extracts local spatial neighborhood features, with its receptive field covering the pixel region immediately surrounding the central pixel. Dilation-1 to Dilation-5 are 2D dilated convolutional layers with progressively increasing dilation rates. Increasing the dilation rate allows the same-sized convolutional kernel to cover a larger spatial range; for example, a 3×3 convolutional kernel expands its actual receptive field to a 7×7 region when the dilation rate is 2. By progressively increasing the dilation rate, the 2D branch can simultaneously capture multi-scale spatial features ranging from fine-grained textures to coarse-grained spatial distributions. Different colored blocks represent different spatial scales of feature extraction capability.

[0063] 3D Convolutional Dilation Sequence: As shown in the lower right corner of the figure, the 3D feature extraction branch contains multiple 3D convolutional layers, labeled 3DConv Dilation-0 to Dilation-5. Dilation-0 is a standard 3D convolutional layer that performs convolutions simultaneously in the spatial height, width, and spectral dimensions, extracting joint spatial-spectral local features. Dilation-1 to Dilation-5 are 3D dilated convolutional layers with increasing dilation rates. The dilation rate increases synchronously or asynchronously in the three dimensions, expanding the coverage of the three-dimensional receptive field. For example, a 3D convolution with a dilation rate of 2 can expand the three-dimensional receptive field several times while keeping the number of parameters constant, capturing a wider range of joint spatial-spectral patterns. The progressive dilation configuration enables the 3D branch to construct multi-level joint feature representations from local spatial-spectral patches to global spatial-spectral context.

[0064] The figure also labels each convolutional dilation sequence with symbols for residual connections and feature concatenation. A plus sign inside a circle indicates a residual connection operation, and a plus sign inside a box indicates a feature concatenation operation. ReLU and BN represent the activation function and batch normalization layer, respectively. These designs correspond to the specific implementation of feature extraction in step S303.

[0065] Figure 3 The central area shows how the features extracted from the three branches are fused, which corresponds to the implementation process of step S304.

[0066] Intra-branch feature concatenation: As shown at the end of each branch in the figure, within each feature extraction branch, the first receptive field feature output from the standard convolutional layer of that branch is first concatenated with the second receptive field feature output from each dilated convolutional layer. In the figure, colored arrows guide the outputs of convolutional layers of different colors to the concatenation operation symbols, and the plus sign indicates feature concatenation along the channel dimension. The 1D branch concatenates 1D features from all its levels to form a multi-level spectral feature map; the 2D branch concatenates 2D features from all its levels to form a multi-level spatial feature map; and the 3D branch concatenates 3D features from all its levels to form a multi-level spatial-spectral joint feature map. After concatenation, each branch obtains a multi-dimensional feature map with a large number of channels, which encodes multi-scale information from local to global along the channel dimension.

[0067] Dimensionality Reduction: As shown in the "Dimensionality Reduction" module in the figure, the concatenation operation significantly increases the number of channels in the feature map. To control subsequent computational complexity and prevent feature redundancy, dimensionality reduction is necessary for the concatenated multidimensional feature map. Dimensionality reduction is typically achieved through 1×1 convolution. This convolution operation compresses the number of channels to a reasonable range while maintaining the spatial dimension, and simultaneously achieves linear combination and fusion of features from different channels. The figure shows that the multidimensional feature map of each branch, after dimensionality reduction, yields a compact feature representation with a suitable number of channels.

[0068] Cross-branch feature stitching: As shown in the central area of ​​the figure, after completing feature stitching and dimensionality reduction within each branch, the dimensionality-reduced feature maps from the 1D, 2D, and 3D branches are stitched together across branches to form a comprehensive representation that integrates spectral features, spatial features, and spatial-spectral joint features. The stitching operation is also performed along the channel dimension, connecting the feature maps of the three branches end-to-end along the channel dimension. The stitched features simultaneously contain pure spectral information, pure spatial information, and spatial-spectral joint information. This multi-perspective feature fusion can provide a more comprehensive and complementary discriminative basis for subsequent classification.

[0069] Deep feature fusion: As shown in the figure, the features after cross-branch splicing may be further processed by the dimensionality reduction or feature fusion module to integrate and refine the information of multiple branches and multiple scales, so as to obtain the final reduced global feature representation.

[0070] Figure 3 The bottom area shows the feature enhancement process based on embedding layers and attention mechanisms.

[0071] Embedding layer processing: As shown in the bottom left of the figure, the fused global features are converted into serialized token representations. The figure shows the feature map divided into multiple colored patches, each representing a token. These tokens can be embedding vectors obtained by segmenting the feature map into multiple patches. The role of the embedding layer is to convert the convolutional features into a sequence format suitable for attention mechanisms such as Transformer.

[0072] Multi-head self-attention: The token sequence is input into the attention module for processing. The diagram shows the processing flow, which includes multiple sub-modules such as "multi-head cross-attention," "feedforward," "fully connected," and "pooling." The multi-head self-attention mechanism can learn the global dependencies between tokens, capture long-distance feature interactions and contextual relationships, and compensate for the shortcomings of convolutional operations in global modeling capabilities. Through adaptive allocation of attention weights, the model can highlight important feature regions and channels while suppressing redundant or noisy information.

[0073] Feature Mapping Output: After attention enhancement and subsequent operations such as fully connected layers and pooling, the final output is a feature representation used for classification. The right side of the figure shows the classification result image with color coding; different colors represent different crop categories. This output is the model's prediction of the category to which the center pixel of the input image patch belongs.

[0074] Through the above architecture design, the model can make full use of the multi-layered receptive field constructed by progressive dilated convolution to extract discriminative features of hyperspectral images from multiple dimensions and scales, thereby achieving accurate identification and classification of complex crop distribution scenes.

[0075] S301: Obtain the preset standard convolutional layer and dilation rate sequence, and determine multiple dilated convolutional layers based on the dilation rate sequence and the standard convolutional layer. The dilation rate sequence is an increasing sequence of multiple dilation rates. In neural networks, a standard convolutional layer uses closely spaced convolutional kernels for feature extraction. The kernels slide across the input feature map with a standard stride, performing a weighted summation of neighboring pixels at each location, thus capturing spatial feature patterns within a local neighborhood. Dilation rate refers to the spacing parameter inserted between kernel elements during convolution. A dilation rate of 1 corresponds to standard convolution, while a dilation rate greater than 1 indicates spacing between kernel elements, allowing kernels of the same size to cover a larger spatial area. A dilation rate sequence is an ordered set of multiple dilation rate values ​​arranged according to a specific pattern. The sequence contains dilation rate values ​​increasing sequentially from smallest to largest, guiding the construction of dilated convolutional layers with different receptive field scales. A dilated convolutional layer is a neural network layer that uses spaced convolutional kernels for feature extraction. By inserting holes or gaps between kernel elements, it expands the receptive field, acquiring a wider range of contextual information without increasing the number of parameters or computational cost. An ascending sequence means that the values ​​in the sequence are arranged in ascending order, and the value of the next element is strictly greater than the value of the previous element, ensuring that the constructed dilated convolutional layer can gradually expand the receptive field and form multi-scale feature extraction capability.

[0076] Specifically, the configuration parameters and dilation rate sequence of the standard convolutional layer are first obtained from the preset configuration. The standard convolutional layer defines basic parameters such as kernel size and stride, while the dilation rate sequence defines incremental dilation rate values. Then, based on the number of dilation rates in the dilation rate sequence, the number of dilated convolutional layers to be created for each feature extraction branch is determined. Next, the dilation rate sequence is traversed, and for each dilation rate value in the sequence, using the configuration parameters of the standard convolutional layer as a template and the current dilation rate as the dilation parameter, dilated convolutional layers of the corresponding dimensions are created for the 1D, 2D, and 3D feature extraction branches. Finally, the standard convolutional layers are associated with the multiple dilated convolutional layers determined based on the dilation rate sequence to complete the convolutional layer configuration for each feature extraction branch.

[0077] For example, in practical applications, this application uses a dilation rate sequence d=[0, 1, 2, 5], which contains four values, where d=0 corresponds to a standard convolutional layer, and d=1, d=2, and d=5 correspond to three incrementally dilated convolutional layers. For the 1D feature extraction branch, one standard 1D convolutional layer and three consecutive 1D dilated convolutional layers are configured; for the 2D feature extraction branch, one standard 2D convolutional layer and three consecutive 2D dilated convolutional layers are configured; and for the 3D feature extraction branch, one standard 3D convolutional layer and three consecutive 3D dilated convolutional layers are configured. By setting this dilation rate sequence d=[0, 1, 2, 5], receptive field pyramid structures can be constructed in each of the three branches.

[0078] S302: Construct multi-level receptive fields corresponding to each feature extraction branch based on the standard convolutional layer and multiple dilated convolutional layers corresponding to each feature extraction branch; In this context, a multi-layered receptive field refers to a feature-perceiving structure with multiple scale coverages formed by organizing convolutional layers with different configurations in a neural network. In this application, the multi-layered receptive field is constructed by combining standard convolutional layers with multiple dilated convolutional layers with increasing dilation rates, enabling the network to simultaneously capture multi-scale information from local details to global context.

[0079] Specifically, firstly, the standard convolutional layer and multiple dilated convolutional layers determined for each feature extraction branch in step S301 are obtained. Then, for each feature extraction branch, the standard convolutional layer and multiple dilated convolutional layers are organized into a multi-level structure, establishing the data flow and processing relationships between each convolutional layer. Next, based on the mathematical principles of dilated convolution, the equivalent convolutional kernel size and receptive field size corresponding to each convolutional layer are calculated to determine the receptive field coverage of each level. Finally, the multi-level spectral receptive field of the 1D feature extraction branch, the multi-level spatial receptive field of the 2D feature extraction branch, and the multi-level spatial-spectral joint receptive field of the 3D feature extraction branch are constructed.

[0080] For example, in practical applications, this application employs a progressively dilated convolution strategy to construct multi-layered receptive fields. Its core is to achieve multi-scale feature extraction through an increasing sequence of dilation rates. The formulas for calculating the effective kernel size L and the receptive field size R_{m+1} in dilated convolution are as follows: L = l + (l-1) × rate - 1; R_{m+1}=R_m+(L_{m+1}-1)×∏_{i=1}^{m}S_i; In the formula, L represents the size of the equivalent convolutional kernel, l represents the size of the original convolutional kernel, and rate is the dilation rate. R_{m+1} is the size of the receptive field of the (m+1)th dilated convolutional layer, and S_i represents the stride of the i-th layer. The above formula can determine the equivalent convolutional kernel size and receptive field size of each convolutional layer when using the dilation rate sequence d=[0, 1, 2, 5], thus constructing a multi-layered receptive field structure from local to global in the three feature extraction branches.

[0081] S303: Use multi-level receptive fields to process hyperspectral feature data to obtain the first receptive field features corresponding to the standard convolutional layer and the second receptive field features of each dilated convolutional layer in each level. The first receptive field feature refers to the feature representation extracted by the standard convolutional layer. This feature corresponds to the local receptive field range covered by the standard convolutional kernel and mainly captures fine-grained local patterns and details between adjacent positions or adjacent bands in the input data.

[0082] The second receptive field feature refers to the feature representation extracted by each dilated convolutional layer. These features correspond to the expanded receptive field range under different dilation rate configurations, and can capture a larger range of mesoscale patterns and global contextual information in the input data.

[0083] Specifically, firstly, hyperspectral feature data is input into a multi-layered receptive field structure constructed by each feature extraction branch. Then, within each feature extraction branch, a standard convolutional layer performs convolution processing on the hyperspectral feature data, extracting features within a local receptive field to obtain the first receptive field features. Simultaneously, multiple dilated convolutional layers perform dilated convolution processing on the hyperspectral feature data according to their respective dilation rates, extracting features within receptive field ranges at different scales to obtain multiple second receptive field features. Next, multi-layered features in the spectral dimension are obtained in the 1D feature extraction branch, multi-layered features in the spatial dimension are obtained in the 2D feature extraction branch, and multi-layered features in the combined spatial-spectral dimension are obtained in the 3D feature extraction branch. Finally, the first receptive field features output by the standard convolutional layers and the second receptive field features output by each dilated convolutional layer in each branch are collected to form a multi-layered, multi-dimensional feature set.

[0084] For example, in practical applications, the 1D branch focuses on spectral features through one-dimensional convolution to uncover subtle differences in the spectral curve; the 2D branch uses two-dimensional convolution to target the HSI spatial dimension, extracting features such as spatial structure and texture; the 3D branch uses three-dimensional convolution to simultaneously associate the spatial and spectral dimensions, capturing joint "spectral-spatial" features. Specifically, for the dilation rate sequence d=[0, 1, 2, 5], the standard convolutional layer of the 1D branch outputs the first receptive field feature G_0, and the three dilated convolutional layers output the second receptive field features G_1, G_2, and G_5, respectively; the standard convolutional layer of the 2D branch outputs the first receptive field feature F_0, and the three dilated convolutional layers output the second receptive field features F_1, F_2, and F_5, respectively; the standard convolutional layer of the 3D branch outputs the first receptive field feature H_0, and the three dilated convolutional layers output the second receptive field features H_1, H_2, and H_5, respectively. Here, G_d, F_d, and H_d represent the convolutional outputs of the corresponding branches at dilation rate d.

[0085] S304: Concatenate the first receptive field feature and the corresponding second receptive field feature in each level to form a multi-level receptive field feature, and obtain the multi-dimensional feature map corresponding to each feature extraction branch.

[0086] Specifically, firstly, the first receptive field features and multiple second receptive field features output from each feature extraction branch in step S303 are obtained. Then, the features in each feature extraction branch are flattened in the spatial dimension, converting them into a unified representation. Next, within each feature extraction branch, the first receptive field features are concatenated with all second receptive field features along the channel dimension to form a fused feature containing multi-level receptive field information. Finally, 1D multidimensional feature maps corresponding to the 1D feature extraction branch, 2D multidimensional feature maps corresponding to the 2D feature extraction branch, and 3D multidimensional feature maps corresponding to the 3D feature extraction branch are obtained. These multidimensional feature maps encode multi-scale feature representations in the spectral dimension, spatial dimension, and spatial-spectral joint dimension, respectively.

[0087] For example, in practical applications, the features of each branch are flattened in the spatial dimension and then stitched together. The fusion feature of the 1D branch is calculated as X_{1D}=Concat(G_0, G_1, G_2, G_5), which stitches together the outputs of the four 1D convolutions corresponding to the dilation rate d=[0, 1, 2, 5] along the channel dimension; the fusion feature of the 2D branch is calculated as X_{2D}=Concat(F_0, F_1, F_2, F_5), which stitches together the outputs of the four 2D convolutions corresponding to the dilation rate d=[0, 1, 2, 5] along the channel dimension; the fusion feature of the 3D branch is calculated as X_{3D}=Concat(H_0, H_1, H_2, H_5), which stitches together the outputs of the four 3D convolutions corresponding to the dilation rate d=[0, 1, 2, 5] along the channel dimension. The above stitching operation can be expressed as: X_{1D}=Concat(G_0, G_1, G_2, G_5); X_{2D}=Concat(F_0, F_1, F_2, F_5); X_{3D}=Concat(H_0, H_1, H_2, H_5); Where X_{1D}, X_{2D}, and X_{3D} represent 1D, 2D, and 3D feature maps, respectively. Through this concatenation operation, the multi-level receptive field features of each branch are integrated into a unified multi-dimensional feature map representation.

[0088] Based on the above embodiments, as an optional implementation method, the multi-dimensional feature map is input into the pre-constructed linear projection feature pyramid module in step S104 to obtain multi-scale fusion features, which can be specifically implemented through the following steps S401-S405.

[0089] S401: Input the multidimensional feature maps corresponding to each feature extraction branch into the pre-constructed linear projection feature pyramid module to perform feature stitching on the multidimensional feature maps in the channel dimension to obtain the first fused feature; Among them, the linear projection feature pyramid module refers to a feature fusion module constructed by linear projection transformation and multi-scale depth separable convolution. Its design goal is to capture and fuse spatial features of different scales to form a multi-scale feature representation with a pyramid-like structure while reducing computational complexity, thereby enhancing the model's ability to perceive multi-scale information.

[0090] The first fusion feature refers to the initial fusion feature representation obtained by splicing multidimensional feature maps from different feature extraction branches in the channel dimension. This feature integrates feature information extracted from multiple branches from different perspectives.

[0091] Specifically, firstly, the multidimensional feature maps output by each feature extraction branch in step S304 are obtained, including the 1D multidimensional feature map corresponding to the 1D feature extraction branch, the 2D multidimensional feature map corresponding to the 2D feature extraction branch, and the 3D multidimensional feature map corresponding to the 3D feature extraction branch. Then, these multidimensional feature maps are input into a pre-constructed linear projection feature pyramid module. Next, within the linear projection feature pyramid module, feature stitching operations are performed on each multidimensional feature map along the channel dimension, stacking the features extracted from different branches in the channel direction. Finally, a first fused feature integrating the feature information from multiple branches is obtained.

[0092] For example, in practical applications, this application uses the 1D feature map X_{1D}, 2D feature map X_{2D}, and 3D feature map X_{3D} output by the multi-dimensional receptive field feature extraction module as input, and performs a concatenation operation along the channel dimension to form a fused feature X_{fusion}. This concatenation operation can be represented as: X_{fusion}=Concat(X_{1D}, X_{2D}, X_{3D}); Among them, X_{fusion} is the first fusion feature, which integrates multi-scale feature representations of spectral dimension, spatial dimension and spatial-spectral joint dimension.

[0093] S402: Perform linear projection transformation on the initial fused features through the linear projection layer in the linear projection feature pyramid module to reduce the feature dimensionality and obtain the second fused features after dimensionality reduction. The linear projection layer refers to a neural network layer that performs linear transformations, mapping input features to different feature spaces or changing feature dimensions through matrix multiplication. It is commonly used for feature dimensionality reduction to reduce the computational cost of subsequent processing. The linear projection transformation refers to the operation of transforming input features through a linear mapping matrix. In this application, it is used to map high-dimensional features to a low-dimensional space, reducing feature dimensionality while preserving key feature information. The second fused feature refers to the fused feature representation obtained after dimensionality reduction through the linear projection transformation. Compared to the first fused feature, it has a lower feature dimensionality, thereby reducing the computational cost of subsequent convolutional layers and improving computational efficiency.

[0094] Specifically, the first fused feature obtained in step S401 is first acquired. Then, the first fused feature is input into the linear projection layer of the linear projection feature pyramid module. Next, a linear projection transformation operation is performed on the first fused feature through the linear projection layer, mapping the feature from a high-dimensional space to a low-dimensional space, thus achieving feature dimensionality reduction. Finally, the dimensionality-reduced second fused feature is obtained, which retains key feature information while having lower computational complexity.

[0095] For example, in practical applications, this application performs dimensionality reduction processing on the first fusion feature X_{fusion} through a linear projection layer to obtain the dimensionality-reduced second fusion feature X'_{fusion}. This linear projection transformation operation can be expressed as: X' {fusion}=Linear(X {fusion}); Here, Linear(·) represents linear projection transformation. This operation can reduce the feature dimension, reduce the number of parameters and computational cost of subsequent multi-scale deep convolutional layers, and improve the overall computational efficiency of the model.

[0096] S403: Divide the second fused feature into N sub-feature maps in the channel direction, with each sub-feature map corresponding to a subset of the feature channels; Among them, a sub-feature map refers to a subset of feature maps obtained by segmenting the fused features along the channel dimension. Each sub-feature map contains a portion of the channels of the original fused features and is used in subsequent processing to achieve multi-scale feature extraction by employing different convolution kernel sizes.

[0097] Specifically, first, the dimensionality-reduced second fusion feature obtained in step S402 is acquired. Then, a feature flattening operation is performed on the second fusion feature to convert it into a representation that is easy to segment. Next, the second fusion feature is segmented along the channel dimension, dividing it into N sets of sub-feature maps, each set of sub-feature maps corresponding to a non-overlapping subset of the original feature channels. Finally, N sets of sub-feature maps are obtained, preparing for subsequent multi-scale feature extraction using convolutional kernels of different sizes.

[0098] For example, in practical applications, this application first flattens the dimensionality-reduced second fusion feature X'_{fusion}, and then divides it into N sets of feature vectors along the channel direction. This segmentation operation can be expressed as: X' {11}，X' {22},……,X' {NN}=Split(X' {fusion}); Where X' {11}，X' {22}, ..., X'_{NN} are the N sub-feature maps segmented out. Each sub-feature map has an independent subset of feature channels, which can be processed in subsequent steps using convolutional kernels of different dimensions.

[0099] S404: Construct a multi-scale depth convolutional layer, perform depthwise pointwise convolution operation on N groups of sub-feature maps respectively, and obtain N groups of target sub-feature maps. Different groups of sub-feature maps use convolution kernels of different sizes to capture spatial features of different scales. Among them, multi-scale depthwise convolutional layers refer to structures composed of multiple depthwise convolutional layers with different kernel sizes. By applying convolutional kernels of different sizes to different feature groups, parallel extraction of multi-scale spatial features is achieved. Depthwise pointwise convolution operation refers to the depthwise convolutional part of depthwise separable convolution. It is a special type of grouped convolution where the number of groups equals the number of input channels. Each input channel is processed using an independent convolutional kernel without cross-channel interaction, thus significantly reducing the number of parameters and computational cost. Target sub-feature maps refer to the sub-feature maps obtained after processing by depthwise pointwise convolution operations. Each target sub-feature map contains spatial feature information at a specific scale.

[0100] Specifically, firstly, a multi-scale depthwise convolutional layer is constructed, containing N depthwise convolutional units with different kernel sizes. Then, the N sets of sub-feature maps obtained in step S403 are acquired. Next, the corresponding depthwise convolutional unit is applied to each set of sub-feature maps, performing a pointwise depthwise convolution operation. Each input channel is processed using an independent convolutional kernel, while the number of output channels remains constant. Different sets of sub-feature maps use convolutional kernels of different sizes to capture spatial features at different scales. Finally, N sets of target sub-feature maps are obtained, each containing spatial feature representations at a specific scale.

[0101] For example, in practical applications, this application performs depthwise convolution (DC) operations on each of the N sub-feature maps after segmentation. Depthwise convolution is a special type of grouped convolution where the number of groups equals the number of input channels. Each input channel is processed using an independent convolution kernel, while the number of output channels remains unchanged. By applying a convolution kernel to each input channel separately, without cross-channel interaction, this decomposition significantly reduces the number of parameters and computational cost.

[0102] Assuming the input sub-feature map is X∈ℝ^{H×W×C_{in}}, performing a K×K dimensional depthwise convolution on it yields C_{out} output channels. Then the computational complexity ω_s of the standard convolution is: ω_s=K×K×H×W×C_{in}×C_{out}; The computational cost ω_{DPC} of depthwise pointwise convolution is: ω_{DPC}=(K×K+C_{out})×H×W×C_{in}; After depthwise pointwise convolution, the reduction in the number of parameters, ω_Δ, is shown below: ω_Δ=[(K×K+C_{out})×C_{in}] / [K×K×C_{in}×C_{out}]=1 / K^2; By using depthwise pointwise convolution operations, the computational complexity can be significantly reduced while maintaining the feature extraction capability.

[0103] S405: Reorganize the target sub-feature maps of each group and output multi-scale fusion features.

[0104] Specifically, a recombination operation is performed on each group of target sub-feature maps to integrate feature representations at different scales. Then, a feature fusion mechanism is used to merge the recombinated features, forming a unified feature representation that includes multi-scale spatial feature information. Finally, a multi-scale fused feature is output, which integrates features extracted from different scales by the linear projection feature pyramid module and can be used for subsequent classification or other task processing.

[0105] Based on the above embodiments, as an optional implementation method, the Transformer module includes an embedding layer, a multi-head self-attention layer and a multi-layer perceptron layer. In step S105, the multi-scale fused features are input into the Transformer module, and self-attention calculation operations are performed on the multi-scale fused features to obtain deep joint features. This can be specifically achieved through the following steps S501-S506.

[0106] S501: The embedding layer is used to map the multi-scale fused features into an initial serialized feature vector, and the initial serialized feature vector is subjected to layer normalization to obtain the target serialized feature vector. The embedding layer refers to the neural network layer in the Transformer module used to map input features into serialized feature vectors. It transforms spatially structured features into a sequence form suitable for Transformer processing through linear transformation or projection operations.

[0107] The initial serialized feature vector refers to the feature vector in sequence form obtained after mapping the multi-scale fused features through the embedding layer. In the Transformer architecture, it is usually called the token sequence, where each token represents a local representation of the input feature.

[0108] Target serialized feature vector: refers to the serialized feature vector obtained after layer normalization. This vector has a standardized numerical distribution and is suitable for input into a multi-head self-attention layer for subsequent processing.

[0109] Specifically, the multi-scale fusion features output in step S405 are first obtained. Then, these features are input into the embedding layer of the Transformer module. Through linear mapping operations in the embedding layer, the spatially structured multi-scale fusion features are converted into a sequence of tokens that the Transformer can process, resulting in an initial serialized feature vector. Next, layer normalization is performed on the initial serialized feature vector, standardizing the feature vector along its feature dimensions to make the feature distribution more stable. Finally, the normalized target serialized feature vector is obtained, which can be used as input to the multi-head self-attention layer.

[0110] For example, in practical applications, this application first transforms the deep global features of the hyperspectral image obtained from the linear projection feature pyramid module into a token sequence that can be processed by the Transformer through an embedding layer. Then, layer normalization is performed on the tokens to stabilize the feature distribution and provide standardized input features for subsequent multi-head self-attention computation. This preprocessing operation can alleviate covariate bias and provide a relatively stable gradient signal.

[0111] S502: Using a multi-head self-attention layer, the target serialized feature vector is mapped to the query vector space, key vector space and value vector space respectively to obtain the query vector, key vector and value vector. Among them, the multi-head self-attention layer is the core component of the Transformer module, which contains multiple parallel self-attention mechanisms. Each attention head independently learns different representation subspaces of features. Through the multi-head mechanism, the dependencies between features can be captured from multiple perspectives.

[0112] The query vector space refers to the vector space used in the self-attention mechanism to represent query information. It is a mapping space obtained by multiplying the input feature vectors by the query weight matrix. The key vector space refers to the vector space used in the self-attention mechanism to represent key information. It is a mapping space obtained by multiplying the input feature vectors by the key weight matrix. The key vectors are used to calculate the similarity with the query vectors. The value vector space refers to the vector space used in the self-attention mechanism to represent value information. It is a mapping space obtained by multiplying the input feature vectors by the value weight matrix. The value vectors carry the actual feature content and are weighted and aggregated according to the attention weights.

[0113] The query vector is the vector Q (query) obtained by mapping the target serialized feature vector to the query vector space. It represents the query request from the current position to other positions. The key vector is the vector K (key) obtained by mapping the target serialized feature vector to the key vector space. It represents the feature identifier of each position and is matched with the query vector to calculate attention weights. The value vector is the vector V (value) obtained by mapping the target serialized feature vector to the value vector space. It carries the actual feature information of each position and is weighted and aggregated according to the attention weights to generate the output features.

[0114] Specifically, the target serialized feature vector obtained in step S501 is first acquired. Then, the target serialized feature vector is input into a multi-head self-attention layer (MHSA). Next, through three sets of linear transformation matrices in the MHSA layer, a mapping operation is performed on each pixel vector feature in the target serialized feature vector, projecting it onto the query vector space Q, the key vector space K, and the value vector space V. Finally, the corresponding query vector Q, key vector K, and value vector V are obtained, and these three sets of vectors are used for subsequent attention weight calculation and feature aggregation operations.

[0115] For example, in practical applications, the multi-head self-attention layer of this application captures feature relationships in different subspaces by mapping each pixel vector feature of the input to three independent vector subspaces Q (query), K (key), and V (value). Through this multi-head mechanism, the model can learn feature representations from multiple perspectives, enhancing its ability to model complex feature relationships.

[0116] S503: Calculate the similarity matrix between the query vector and the key vector, and transform the similarity matrix into an attention weight matrix using a normalization function; The similarity matrix is ​​obtained by performing a dot product operation between the query vector Q and the key vector K. Each element in the matrix represents the relevance between a query position and a key position; a larger value indicates a higher similarity. The normalization function is used to normalize the similarity matrix, typically employing the Softmax function to convert the similarity scores into a probability distribution, ensuring that the weights in each row sum to 1. The attention weight matrix, obtained after processing with the normalization function, represents the attention weight of one position on another. Weight values ​​range from 0 to 1, and the sum of the weights in each row is 1, guiding subsequent feature weighting and aggregation.

[0117] Specifically, the process begins by obtaining the query vector Q and the key vector K. Then, the matrix multiplication between the transposes of Q and K is performed to obtain a similarity matrix, where each element reflects the similarity between the corresponding query position and the key position. Next, the similarity matrix is ​​scaled, typically by dividing it by the square root of the key vector dimension d_k to stabilize the gradient. The scaled similarity matrix is ​​then processed using the Softmax normalization function. Finally, the attention weight matrix is ​​obtained, which transforms the similarity into normalized attention weights.

[0118] Optionally, the similarity matrix can be calculated as follows: Calculate the similarity matrix between the query vector and the key vector using the similarity calculation formula; the similarity calculation formula is: ; in, Let represent the similarity matrix, Q represent the query vector, K represent the key vector, V represent the value vector, K^T represent the transpose of the key vector, and d_k represent the dimension of the key vector.

[0119] The calculation process of this formula first involves performing a matrix dot product between the transpose of the query vector and the key vector to obtain the original similarity matrix. This step physically means calculating the matching degree between each query position and all key positions; a larger dot product value indicates greater similarity and stronger correlation between the features of the two positions. Next, the original similarity matrix is ​​scaled by dividing it by the square root of the key vector dimension to prevent the dot product from becoming too large and causing gradient vanishing when the dimension is large. Then, the scaled similarity matrix is ​​normalized using the Softmax function, converting the similarity score of each row into a probability distribution so that the weights of each row sum to one. These normalized weights are the attention weights. Finally, the resulting attention weight matrix is ​​multiplied by the value vectors to achieve a weighted summation of the value vectors. The feature of each output position is a weighted combination of the value vectors of all input positions, with the weights automatically learned by the attention mechanism. This achieves the goal of dynamically aggregating global information based on feature similarity.

[0120] S504: Perform a dot product operation between the attention weight matrix and the value vector to obtain the attention weights between multi-head self-attention. The attention weights represent the global spectral-spatial dependencies between the serialized feature vectors of each target. Here, attention weight refers to the feature representation obtained by weighting and aggregating the value vector through an attention weight matrix. This feature integrates global dependency information captured based on the attention mechanism, reflecting the degree of attention each token pays to other tokens. Global spectral-spatial dependency refers to the long-distance dependency between different spatial locations and spectral bands in a hyperspectral image. Through the self-attention mechanism, the direct association between any two locations can be captured without being limited by the receptive field, thereby establishing a joint dependency pattern of global spectral and spatial dimensions.

[0121] Specifically, first, the attention weight matrix obtained in step S503 and the value vector V obtained in step S502 are obtained. Then, a matrix-dot product operation is performed between the attention weight matrix and the value vector V, and weighted aggregation is performed on each position of the value vector. Next, through this weighted aggregation operation, the feature of each output position is a weighted combination of the features of all input positions, with the weights determined by the attention weight matrix, thereby realizing feature interaction on a global scale. Finally, the attention weights between multi-head self-attention are obtained, the outputs of multiple attention heads are concatenated together, and they are projected into the output space through a linear transformation, thereby reducing dimensionality and retaining useful information.

[0122] For example, in practical applications, this application performs a dot product operation between the obtained attention weight matrix and the value vector V to obtain the attention weight of each token to other tokens in multi-head self-attention. This attention weight represents the global spectral-spatial dependency between the serialized feature vectors of each target, and can capture complex relationships such as "different spectra in the same region" and "same spectra in different regions", fully explore the spectral-spatial dimension features of hyperspectral images, enhance the understanding of complex relationships between crop classes with similar spectral features, and uncover hidden intrinsic spectral differences in crops.

[0123] S505: Perform residual connection and layer normalization on the attention weights and the target serialized feature vector to obtain the intermediate feature vector; Residual connections refer to the operation of adding input features to the transformed output features. By establishing direct connections across layers, the vanishing gradient problem in deep networks can be alleviated, gradient backpropagation can be promoted, model training can be accelerated, and performance can be improved. Intermediate feature vectors refer to feature vectors obtained after multi-head self-attention processing, residual connections, and layer normalization. This vector integrates the original input features and the globally dependent features extracted by the attention mechanism, and has a standardized numerical distribution.

[0124] Specifically, the attention weights obtained in step S504 and the target serialized feature vector obtained in step S501 are first obtained. Then, a residual concatenation operation is performed on the attention weights and the target serialized feature vector, i.e., they are added element-wise to preserve the original input information and alleviate the gradient vanishing problem. Next, layer normalization is performed on the residual concatenated feature vector, standardizing it along the feature dimension to stabilize the feature distribution and promote training convergence. Finally, an intermediate feature vector is obtained, which integrates the input features and the global dependencies captured by the self-attention mechanism, and can be used as input to the multilayer perceptron layer.

[0125] S506: Input the intermediate feature vector into the multilayer perceptron layer for feature transformation to obtain deep joint features.

[0126] In this context, a multilayer perceptron (MLP) refers to a feedforward neural network module composed of multiple fully connected layers. It typically includes two linear transformation layers and an intermediate nonlinear activation function, used to perform nonlinear transformations on the input features and adjust the feature dimensions, thereby enhancing the model's expressive power. The MLP is a fundamental type of feedforward neural network that ensures unidirectional information flow; data enters from the input layer, is processed through hidden layers, and finally outputs from the output layer, with no feedback connections between layers.

[0127] Specifically, the intermediate feature vector obtained in step S505 is first acquired. This intermediate feature vector is then input into a multilayer perceptron (MLP) layer, where layer normalization is performed before the MLP layer to stabilize the input distribution. Next, a first-layer linear transformation is applied to transform the feature dimension, using a non-linear activation function to introduce non-linear expressive power. A second-layer linear transformation then maps the feature back to the target dimension. Finally, the output of the MLP layer is residually concatenated with the intermediate feature vector and subjected to layer normalization to obtain a deep joint feature. This feature integrates comprehensive information from multi-scale feature extraction, global self-attention modeling, and non-linear feature transformation, and can be used for subsequent classification or other downstream tasks.

[0128] For example, in practical applications, the intermediate feature vector is input into the multilayer perceptron layer, and the deep joint features are calculated using the feature transformation formula; The feature transformation formula is: ; Where MLP(X) represents deep joint features, Let X represent a non-linear activation function, and let X represent an intermediate feature vector. , These represent the weight matrices for the first and second layer linear transformations, respectively. , These represent the bias vectors for the first and second layer linear transformations, respectively.

[0129] In the specific calculation, the intermediate feature vector is first multiplied by the first-layer weight matrix, and then the first-layer bias vector is added to complete the first linear transformation. This step usually expands the feature dimension several times over, for example, four times, to increase the network's expressive capacity. Next, the result of the first linear transformation is input into a non-linear activation function for processing. The activation function performs a non-linear mapping on each element, breaking the limitations of linear transformation and allowing the network to fit complex non-linear function relationships. Then, the activated features are multiplied by the second-layer weight matrix, and the second-layer bias vector is added to complete the second linear transformation. This step usually compresses the feature dimension back to its original dimension, achieving a dimensionality expansion followed by compression. Finally, the output features are residually connected to the input intermediate feature vector and layer normalization is performed to form the final deep joint features. Residual connections alleviate the gradient vanishing problem and preserve the original feature information, while layer normalization stabilizes the feature distribution and promotes training convergence.

[0130] The following is combined Figure 4 and Figure 5 This paper describes the specific implementation effect of the small-sample hyperspectral crop classification method that couples multidimensional pyramids and Transformers in this application.

[0131] Figure 4 The dataset has a spatial range of 940×475 pixels and a spatial resolution of 0.043m, containing 270 bands. This dataset mainly covers various complex crop types, including agricultural crops, and exhibits high similarity between specific land cover categories (such as Chinese cabbage and cabbage, and Chinese cabbage and bok choy). The dataset contains 22 land cover types; this study selected 19 agricultural crops as the category of interest, and the remaining categories were designated as "other".

[0132] Figure 5 The dataset has a spatial size of 400×550 pixels and a spatial resolution of approximately 0.463m. The spectral range is 400-1000nm, with a total of 270 bands. The dataset contains 9 land cover types; this study selected 7 crops as the category of interest, and the remaining categories were set as "other".

[0133] The specific evaluation method uses overall accuracy (OA), average accuracy (AA), and kappa coefficient (K) to evaluate classification accuracy, and compares it with six mainstream models. These include methods based on traditional CNNs, such as 3D-CNN, as well as Transformer-based methods, such as SSFTT, morphFormer, GAHT, GSC-ViT, and CTMixer. The accuracy evaluation of MDPC-Net compared with the other six models is shown in Tables 1 and 2.

[0134]

[0135] Table 1: MDPC-Net in Figure 4 Accuracy evaluation of the corresponding dataset and 6 models As shown in Table 1, MDPC-Net achieved an overall accuracy of 89.95% for OA (Average Orientation), 89.63% for AA (Average Orientation), and a kappa coefficient of 0.8730 for 18 crops, indicating a high overall classification performance on the WHU-HongHu dataset. While it outperformed other classic models in most categories, it still struggled to distinguish a few categories. For example, its accuracy in cotton was lower than CTMixer, but its accuracy in cotton stalks was higher, possibly due to the similarity in their spectral curves. Furthermore, the classification accuracy for Chinese cabbage and bok choy, as well as mustard greens and small mustard greens, was low, suggesting spectral confusion in these categories due to spectral and morphological features, increasing the difficulty for the model to differentiate them. Although its accuracy was better than some comparative models, it lagged behind the performance of this model in other categories.

[0136] Combination Figure 4 Compared with Table 1, Figure 4Compared to the real labels, the overall classification effect of MDPC-Net (5b) is better than the other six comparison models. Traditional CNN-based classification methods suffer from significant salt-and-pepper noise, resulting in performance far lower than Transformer-based models. This is because Transformers can capture complex long-distance dependencies in crop classification. When dealing with spectral-spatial associations of different crops, they can uncover cross-regional and cross-time-series crop feature connections, improving classification accuracy, such as distinguishing spectrally similar but dispersed crop categories. However, at plot edges and small plot junctions, MDPC-Net can accurately distinguish categories with high detail matching to the real labels. Compared to other models, MDPC-Net loses less detail and has more refined classification. This is because LPFPM can construct feature pyramids from multi-dimensional residual connections and linear projections, integrating features from different dimensions to meet the multi-scale requirements of crop classification.

[0137]

[0138] Table 2: MDPC-Net in Figure 5 Accuracy evaluation of the corresponding dataset and 6 models As shown in Table 2, MDPC-Net achieved an OA of 95.70%, AA of 97.22%, and a kappa coefficient of 0.9411 for the six crops, indicating a generally high level of accuracy. Only in the cotton category did MDPC-Net's accuracy slightly fall below SSFTT. Furthermore, the accuracy for broad-leaf and narrow-leaf soybeans was relatively low, possibly due to significant spectral differences between these two categories and the others, leading to spectral confusion. However, compared to other models, the discriminative power of these two categories was significantly improved. Moreover, comparing the standard deviations of accuracy across models, MDPC-Net exhibited the smallest standard deviations for most categories. This indicates that the model output is more stable in the face of variations within the dataset (e.g., different plots of the same crop), less affected by fluctuations in data distribution, and exhibits better robustness than the comparative models.

[0139] Combination Figure 5 Compared with Table 2, the MDPC-Net classification map shows good overall color distribution and plot outline matching. The boundaries of the "Other" category are clear and continuous. The shapes and locations of plots belonging to the corn, cotton, rice, and sesame categories are consistent with the true labels, with no obvious misclassified color blocks. Only the broad-leaved soybean portion was misclassified as narrow-leaved soybean and cotton. However, other models lose more detail in small-area categories such as cotton. This indicates that MDPC-Net has a higher degree of fidelity in restoring the global spatial structure and category distribution of crops, and the model has a stronger ability to capture category relationships in complex scenes.

[0140] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of the application.

[0141] Please see Figure 6 This illustration shows a schematic diagram of a small-sample hyperspectral crop classification system coupled with a multidimensional pyramid and a Transformer, provided in an exemplary embodiment of this application. This system can be implemented entirely or partially through software, hardware, or a combination of both. The small-sample hyperspectral crop classification system coupled with a multidimensional pyramid and a Transformer includes: The image patch extraction module is used to acquire hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions. The feature extraction module is used to extract hyperspectral data features of image patches by using multiple feature extraction branches set in parallel in the multidimensional feature extraction module. The feature map generation module is used to capture multi-level receptive field features corresponding to each feature extraction branch from hyperspectral data features using progressive dilated convolution, and generate multi-dimensional feature maps. The feature fusion module is used to input the multi-dimensional feature map into the pre-constructed linear projection feature pyramid module to obtain multi-scale fused features; The feature processing module is used to input multi-scale fused features into the Transformer module, perform self-attention calculation on the multi-scale fused features, and obtain deep joint features; The output module is used to input deep joint features into the fully connected layer for mapping and output crop category labels corresponding to the image patches.

[0142] This application also provides a computer storage medium that can store multiple instructions. The instructions are adapted to be loaded and executed by a processor as described above, using a small-sample hyperspectral crop classification method that couples a multidimensional pyramid and a Transformer. The specific execution process can be found in the detailed description of the embodiments, which will not be repeated here.

[0143] Please see Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 may include: at least one processor 701, at least one network interface 704, a user interface 703, a memory 705, and at least one communication bus 702.

[0144] The communication bus 702 is used to enable communication between these components.

[0145] The user interface 703 may include a display screen and a camera.

[0146] The network interface 704 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0147] The processor 701 may include one or more processing cores. The processor 701 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 705, and by calling data stored in memory 705. Optionally, the processor 701 may be implemented using at least one hardware form of digital signal processing, field-programmable gate array, or programmable logic array. The processor 701 may integrate one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 701 and may be implemented as a separate chip.

[0148] The memory 705 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 705 may include a non-transitory computer-readable medium. The memory 705 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 705 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 705 may also be at least one storage device located remotely from the aforementioned processor 701. Figure 7 As shown, the memory 705, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a small-sample hyperspectral crop classification method that couples a multidimensional pyramid with a Transformer.

[0149] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and practical application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

Claims

1. A small-sample hyperspectral crop classification method coupling multidimensional pyramid and Transformer, characterized in that, The method includes: Acquire hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions; The hyperspectral data features of the image patch are extracted by using multiple feature extraction branches set in parallel in the multidimensional feature extraction module. Progressive dilated convolution is used to capture multi-level receptive field features corresponding to each feature extraction branch from the hyperspectral data features, generating a multi-dimensional feature map; The multidimensional feature map is input into a pre-constructed linear projection feature pyramid module to obtain multi-scale fused features; The multi-scale fused features are input into the Transformer module, and a self-attention calculation operation is performed on the multi-scale fused features to obtain deep joint features. The deep joint features are input into a fully connected layer for mapping, and the crop category label corresponding to the image patch is output.

2. The method according to claim 1, characterized in that, The feature extraction branches include a 1D spectral branch, a 2D spatial branch, and a 3D spatial-spectral joint branch. The process of using multiple feature extraction branches set in parallel within the multidimensional feature extraction backbone to extract hyperspectral data features of the image patch includes: The image blocks are respectively input into the 1D spectral branch, the 2D spatial branch, and the 3D spatial-spectral joint branch; The first data feature is obtained by using a one-dimensional convolution kernel with a 1D spectral branch to extract spectral dimension features from the image patch. The image patch is subjected to spatial dimension feature extraction using a 2D spatial branching two-dimensional convolution kernel to obtain the second data feature; The first data feature is obtained by using a 3D convolution kernel with a 3D spatial-spectral joint branch to extract spectral dimension features from the image patch. The first data feature, the second data feature, and the third data feature are determined as the hyperspectral data features of the image block.

3. The method according to claim 1, characterized in that, The step of using progressively dilated convolution to capture multi-level receptive field features corresponding to each feature extraction branch from the hyperspectral data features, generating a multi-dimensional feature map, includes: Obtain a preset standard convolutional layer and dilation rate sequence, and determine multiple dilated convolutional layers based on the dilation rate sequence and the standard convolutional layer, wherein the dilation rate sequence is a multiple increasing sequence of dilation rates; Based on the standard convolutional layer and multiple dilated convolutional layers corresponding to each feature extraction branch, construct a multi-layered receptive field corresponding to each feature extraction branch; The hyperspectral feature data is processed using multi-level receptive fields to obtain the first receptive field features corresponding to the standard convolutional layer and the second receptive field features of each dilated convolutional layer in each level. The first receptive field feature in each level is concatenated with the corresponding second receptive field feature to form a multi-level receptive field feature, thereby obtaining the multi-dimensional feature map corresponding to each feature extraction branch.

4. The method according to claim 1, characterized in that, The step of inputting the multidimensional feature map into a pre-constructed linear projection feature pyramid module to obtain multi-scale fused features includes: The multidimensional feature maps corresponding to each feature extraction branch are input into a pre-constructed linear projection feature pyramid module to perform feature stitching on the multidimensional feature maps in the channel dimension, thereby obtaining the first fused feature; The initial fused features are subjected to linear projection transformation through the linear projection layer in the linear projection feature pyramid module to reduce feature dimensionality, thereby obtaining the second fused features after dimensionality reduction. The second fused feature is divided into N groups of sub-feature maps in the channel direction, and each group of sub-feature maps corresponds to a subset of the feature channels; Construct multi-scale depth convolutional layers and perform depthwise pointwise convolution operations on the N groups of sub-feature maps respectively to obtain N groups of target sub-feature maps, wherein different groups of sub-feature maps use convolutional kernels of different sizes to capture spatial features of different scales; The target sub-feature maps of each group are recombined to output the multi-scale fusion features.

5. The method according to claim 1, characterized in that, The Transformer module includes an embedding layer, a multi-head self-attention layer, and a multi-layer perceptron layer. The multi-scale fused features are input into the Transformer module, and a self-attention computation operation is performed on the multi-scale fused features to obtain deep joint features, including: The embedding layer is used to map the multi-scale fused features into an initial serialized feature vector, and the initial serialized feature vector is subjected to layer normalization to obtain the target serialized feature vector. Using the multi-head self-attention layer, the target serialized feature vector is mapped to the query vector space, key vector space, and value vector space respectively to obtain the query vector, key vector, and value vector. Calculate the similarity matrix between the query vector and the key vector, and transform the similarity matrix into an attention weight matrix using a normalization function; The attention weight matrix and the value vector are multiplied by a dot product to obtain the attention weights between multi-head self-attention, and the attention weights represent the global spectral-spatial dependencies between the target serialized feature vectors. The attention weights and the target serialized feature vector are subjected to residual connection and layer normalization to obtain an intermediate feature vector. The intermediate feature vector is input into the multilayer perceptron layer for feature transformation to obtain deep joint features.

6. The method according to claim 5, characterized in that, The calculation of the similarity matrix between the query vector and the key vector includes: Calculate the similarity matrix between the query vector and the key vector using the similarity calculation formula; The similarity calculation formula is as follows: ； Where MHSA(X) represents the similarity matrix, Q represents the query vector, K represents the key vector, V represents the value vector, K^T represents the key vector transpose, and d_k represents the dimension of the key vector.

7. The method according to claim 1, characterized in that, The step of inputting the intermediate feature vector into the multilayer perceptron layer for feature transformation to obtain deep joint features includes: The intermediate feature vector is input into the multilayer perceptron layer, and the deep joint features are calculated using the feature transformation formula. The feature transformation formula is: ； Where MLP(X) represents deep joint features, Let X represent a non-linear activation function, and let X represent an intermediate feature vector. , These represent the weight matrices for the first and second layer linear transformations, respectively. , These represent the bias vectors for the first and second layer linear transformations, respectively.

8. A small-sample hyperspectral crop classification system coupling a multidimensional pyramid and a Transformer, characterized in that, The system includes: The image patch extraction module is used to acquire hyperspectral image data to be classified and extract image patches containing spectral and spatial dimensions. The feature extraction module is used to extract the hyperspectral data features of the image patch by using multiple feature extraction branches set in parallel in the multidimensional feature extraction module. The feature map generation module is used to capture the multi-level receptive field features corresponding to each feature extraction branch from the hyperspectral data features using progressive dilated convolution, and generate multi-dimensional feature maps. The feature fusion module is used to input the multidimensional feature map into a pre-constructed linear projection feature pyramid module to obtain multi-scale fused features; The feature processing module is used to input the multi-scale fused features into the Transformer module, perform self-attention calculation on the multi-scale fused features, and obtain deep joint features; The output module is used to input the deep joint features into the fully connected layer for mapping and output the crop category label corresponding to the image patch.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, The device includes a processor, a memory, and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

Images