Multispectral image classification method based on adaptive feature fusion residual network

Abstract

The invention provides a multispectral image classification method based on an adaptive feature fusion residual network, and belongs to the technical field of computer vision and remote sensing image processing. The method comprises the following steps: firstly, preprocessing an original multispectral image; then, a novel adaptive feature fusion residual network model is constructed, the core of the model comprises a dual-path feature extraction backbone network, and spectral features and spatial context features are extracted respectively; a self-adaptive feature fusion module is introduced, multi-level features from double paths are dynamically learned and fused through a channel attention mechanism, and optimal integration of spectral information and spatial information is achieved; and finally, completing pixel-level or image-level classification by using the fused advanced features. According to the method, the problems of insufficient utilization of spectrum-space joint features of multispectral data and weak model generalization ability of a traditional method are effectively solved, and the precision and robustness of multispectral image classification are remarkably improved.

Description

Multispectral image classification method based on adaptive feature fusion residual network Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method for multispectral image classification using a deep learning model, specifically a multispectral image classification method based on an adaptive feature fusion residual network. Background Technology

[0002] Multispectral images contain spectral information across multiple bands, offering richer spectral features than traditional RGB images. They are widely used in land use classification, crop monitoring, environmental monitoring, and military target identification. The core challenge in this field is how to efficiently and accurately extract valuable information from multispectral images and perform classification.

[0003] Traditional multispectral image classification methods mainly rely on hand-designed features (such as SIFT, Gabor textures, etc.) combined with shallow machine learning classifiers (such as Support Vector Machines (SVM), Random Forests, etc.). These methods often have limited feature representation capabilities, struggle to capture the complex nonlinear relationship between spectral and spatial features, and require a large amount of specialized prior knowledge.

[0004] In recent years, deep learning technology, especially convolutional neural networks (CNNs), has achieved great success in image classification tasks due to its powerful automatic feature extraction capabilities. However, directly applying CNN models designed for natural images (such as VGG and ResNet) to multispectral images presents the following problems: Insufficient utilization of spectral features: Standard CNNs typically use 3x3 convolutional kernels, processing all spectral bands equally, making it difficult to highlight specific discriminative band combinations and fully exploiting the value of high-dimensional spectral information. Simple spatial-spectral feature fusion strategies: Existing methods usually employ simple concatenation or addition operations to fuse features from different sources, lacking dynamic evaluation of feature importance and failing to achieve optimal fusion.

[0005] Model Depth and Vanishing Gradients: To improve performance, networks need to be deepened, but simply deepening the network can lead to the vanishing / exploding gradient problem. Although Residual Networks (ResNet) alleviate this problem, it remains a challenge to effectively adapt them to multispectral data.

[0006] Therefore, there is an urgent need for a deep network model that can adaptively and efficiently combine the spectral and spatial features of multispectral images and has strong generalization capabilities. Summary of the Invention

[0007] (I) Technical Problems to be Solved In view of the above-mentioned technical problems, the present invention provides a multispectral image classification method based on adaptive feature fusion residual network.

[0008] (II) Technical Solution This invention provides a multispectral image classification method based on an adaptive feature fusion residual network, with the following steps: Step S1: Multispectral image data preprocessing, including data standardization, data augmentation, and dataset partitioning; Step S2: Constructing an adaptive feature fusion residual network model, which includes: a dual-path feature extraction backbone network, used to extract spectral dominant features and spatial context dominant features in parallel from the input multispectral image; an adaptive feature fusion module, embedded at different depths in the backbone network, used to receive the feature maps output by the dual paths, calculate adaptive weights through a channel attention mechanism, and perform weighted fusion of features; a classifier, which receives the final fused high-level feature map and outputs the classification result; Step S3: Training the network model constructed in Step S2 using the training set processed in Step S1, and optimizing the model parameters through backpropagation of the loss function; Step S4: Inputting the multispectral image to be classified into the trained model to obtain the final classification label map or classification confidence score.

[0009] In some embodiments of the present invention, the dual-path feature extraction backbone network consists of a spectral feature path and a spatial feature path; the spectral feature path consists of multiple one-dimensional convolutional layers or point convolutional layers, focusing on extracting pixel-level spectral response features; the spatial feature path consists of multiple two-dimensional convolutional blocks based on residual structures, focusing on extracting local and global spatial context features.

[0010] In some embodiments of the present invention, the adaptive feature fusion module specifically operates as follows: concatenating the two sets of feature maps from the dual paths along the channel dimension; performing global average pooling on the concatenated features to generate channel descriptors; using a fully connected layer and activation function to perform a nonlinear transformation on the channel descriptors to generate an adaptive weight vector for each channel; normalizing the weight vector into two sets of weights using the Softmax function, each representing the importance of the features of the two paths; and using the normalized weights to perform a weighted summation on the two sets of feature maps of the original input to obtain the fused feature map.

[0011] In some embodiments of the present invention, the adaptive feature fusion module is embedded in multiple stages of the backbone network to achieve adaptive fusion of shallow detail features and deep semantic features.

[0012] In some embodiments of the present invention, the loss function is a composite loss function that combines cross-entropy loss and L2 regularization term, which is used to prevent model overfitting while optimizing classification accuracy.

[0013] (III) Beneficial Effects As can be seen from the above technical solutions, the present invention has at least one of the following beneficial effects: The dual-path design of the present invention is specially customized for multispectral data, which optimizes the extraction process of spectral and spatial features respectively, resulting in higher efficiency. By introducing a channel attention mechanism, the model can dynamically adjust the contribution of spectral and spatial features according to the content of the input image, achieving more accurate and robust feature fusion and overcoming the limitations of simple fusion methods.

[0014] The introduction of residual structures in this invention alleviates the gradient vanishing problem in deep networks, allowing for deeper network design and thus stronger representation and generalization capabilities. It fully utilizes the joint spectral-spatial information of multispectral data, achieving higher classification accuracy in both public datasets and real-world applications. Attached Figure Description

[0015] Figure 1 is a flowchart of the overall process of this invention.

[0016] [Explanation of symbols for key components in this invention] Detailed Implementation

[0017] This invention provides a multispectral image classification method based on adaptive feature fusion residual network. To make the objectives, technical solutions and advantages of this invention clearer, the following detailed description is provided in conjunction with specific embodiments and with reference to the accompanying drawings.

[0018] Specific Implementation: As shown in Figure 1, this invention provides a multispectral image classification method based on an adaptive feature fusion residual network, with the following steps: Step S1: Multispectral image data preprocessing, including data standardization, data augmentation, and dataset partitioning; Step S2: Constructing an adaptive feature fusion residual network model, which includes: a dual-path feature extraction backbone network, used to extract spectral dominant features and spatial context dominant features in parallel from the input multispectral image; an adaptive feature fusion module, embedded at different depths in the backbone network, used to receive the feature maps output by the dual paths, calculate adaptive weights through a channel attention mechanism, and perform weighted fusion of the features; a classifier, which receives the final fused high-level feature map and outputs the classification result; Step S3: Training the network model constructed in Step S2 using the training set processed in Step S1, and optimizing the model parameters through backpropagation of the loss function; Step S4: Inputting the multispectral image to be classified into the trained model to obtain the final classification label map or classification confidence.

[0019] The dual-path feature extraction backbone network consists of a spectral feature path and a spatial feature path. The spectral feature path is composed of multiple one-dimensional convolutional layers or point convolutional layers, focusing on extracting pixel-level spectral response features. The spatial feature path is composed of multiple two-dimensional convolutional blocks based on residual structures, focusing on extracting local and global spatial context features.

[0020] The adaptive feature fusion module operates as follows: concatenates the two sets of feature maps from the dual paths along the channel dimension; performs global average pooling on the concatenated features to generate channel descriptors; uses a fully connected layer and activation function to perform a non-linear transformation on the channel descriptors to generate an adaptive weight vector for each channel; normalizes the weight vector into two sets of weights using the Softmax function, representing the importance of the features from the two paths respectively; and uses the normalized weights to perform a weighted summation on the two sets of feature maps from the original input to obtain the fused feature map.

[0021] The adaptive feature fusion module is embedded in multiple stages of the backbone network to achieve adaptive fusion of shallow detail features and deep semantic features.

[0022] The loss function is a composite loss function that combines cross-entropy loss and L2 regularization, used to optimize classification accuracy while preventing model overfitting.

[0023] This concludes the detailed description of the embodiment with reference to the accompanying drawings. Based on the above description, those skilled in the art should have a clear understanding of the present invention.

[0024] It should be noted that implementations not shown or described in the accompanying drawings or the main text of the specification are all forms known to those skilled in the art and are not described in detail. Furthermore, the definitions of the elements and methods described above are not limited to the various specific structures, shapes, or methods mentioned in the embodiments.

[0025] It should also be noted that this document provides examples of parameters containing specific values, but these parameters need not be exactly equal to the corresponding values, but can approximate the corresponding values ​​within acceptable error tolerances or design constraints. Directional terms mentioned in the embodiments, such as "up," "down," "front," "back," "left," and "right," are only for reference to the accompanying drawings and are not intended to limit the scope of protection of this invention. Furthermore, unless specifically described or steps must occur in sequence, the order of the above steps is not limited to those listed above and can be varied or rearranged according to the desired design. Moreover, the above embodiments can be used in combination with each other or with other embodiments based on design and reliability considerations; that is, technical features from different embodiments can be freely combined to form more embodiments.

[0026] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multispectral image classification method based on adaptive feature fusion residual network, characterized in that, The steps are as follows: Step S1: Multispectral image data preprocessing, including data standardization, data augmentation, and dataset partitioning; Step S2: Constructing an adaptive feature fusion residual network model, which includes: a dual-path feature extraction backbone network, used to extract spectral dominant features and spatial context dominant features in parallel from the input multispectral image; an adaptive feature fusion module, embedded at different depths of the backbone network, used to receive the feature maps output by the dual paths, calculate adaptive weights through a channel attention mechanism, and perform weighted fusion of features; a classifier, which receives the final fused high-level feature map and outputs the classification result; Step S3: Training the network model constructed in Step S2 using the training set processed in Step S1, and optimizing the model parameters through backpropagation of the loss function; Step S4: Inputting the multispectral image to be classified into the trained model to obtain the final classification label map or classification confidence.

2. The multispectral image classification method based on adaptive feature fusion residual network according to claim 1, characterized in that, The dual-path feature extraction backbone network consists of a spectral feature path and a spatial feature path. The spectral feature path is composed of multiple one-dimensional convolutional layers or point convolutional layers, focusing on extracting pixel-level spectral response features. The spatial feature path is composed of multiple two-dimensional convolutional blocks based on residual structures, focusing on extracting local and global spatial context features.

3. The multispectral image classification method based on adaptive feature fusion residual network according to claim 1 or 2, characterized in that, The adaptive feature fusion module operates as follows: it concatenates the two sets of feature maps from the dual paths along the channel dimension; it performs global average pooling on the concatenated features to generate channel descriptors; it uses a fully connected layer and activation function to perform a non-linear transformation on the channel descriptors to generate an adaptive weight vector for each channel; and it uses the Softmax function to normalize the weight vector into two sets of weights, representing the importance of the features of the two paths respectively. The two sets of feature maps from the original input are weighted and summed using normalized weights to obtain the fused feature map.

4. The multispectral image classification method based on adaptive feature fusion residual network according to claim 3, characterized in that, The adaptive feature fusion module is embedded in multiple stages of the backbone network to achieve adaptive fusion of shallow detail features and deep semantic features.

5. The multispectral image classification method based on adaptive feature fusion residual network according to claim 1, characterized in that, The loss function is a composite loss function that combines cross-entropy loss and L2 regularization, used to optimize classification accuracy while preventing model overfitting.

Images