A small sample learning-based remote sensing scene classification method and system

Abstract

The application discloses a kind of based on small sample learning's remote sensing scene classification method and system, belong to machine vision technical field, the present application aims at solving the problem of high remote sensing image labeling cost, sample scarcity and the generalization ability of traditional deep learning model is insufficient under small sample condition.This method is realized by improved MobileViT network, including increasing feature fusion module in main backbone structure, introducing channel attention mechanism in MobileNetV2 module and fusing local and global features in MobileViT module.Model training and testing are carried out through meta-training set, meta-validation set and meta-test set, and finally obtained small sample classification model can quickly adapt to new class and realize high-precision classification under the condition of only needing a small amount of labeled samples.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of machine vision, and particularly relates to a remote sensing scene classification method and system based on small sample learning. BACKGROUND

[0002] Remote sensing image scene classification is one of the core technologies in the fields of smart agriculture, disaster monitoring, and environmental management, and its purpose is to accurately classify different scenes (such as ports, beaches, forests, airports, etc.) in remote sensing images through automated algorithms. However, current remote sensing image scene classification technology faces many challenges, mainly in the aspects of sample learning requirements, feature complexity, and model generalization ability. First, the labeling of remote sensing images requires professional knowledge, and the labeling cost is high and time-consuming. For example, hyperspectral images are easily affected by atmospheric interference and cloud cover, resulting in a lack of available labeled data. Traditional deep learning models rely on large-scale labeled data for training, but they are prone to overfitting under small sample conditions and are difficult to generalize to new classes. Second, remote sensing images have significant intra-class differences (such as spectral differences caused by different lighting conditions and shooting angles) and inter-class similarities (such as the close spectra of different vegetation), which make it difficult for traditional convolutional neural networks (CNN) to fully capture multi-modal features, resulting in insufficient classification accuracy. Finally, existing small sample learning methods have insufficient feature decoupling in cross-sensor and cross-territory scenes, leading to performance degradation. For example, in disaster emergency monitoring, it is necessary to quickly identify the area of a sudden disaster, but traditional methods cannot respond in a timely manner due to their reliance on large amounts of labeled data. In summary, existing remote sensing image scene classification technology has problems such as high labeling cost, insufficient model generalization ability, and difficulty in handling feature complexity under small sample conditions, which limits its widespread application in practical applications. Therefore, there is an urgent need for a remote sensing image scene classification method that can efficiently learn and quickly generalize to new classes under small sample conditions. SUMMARY

[0003] The present application proposes a remote sensing scene classification method and system based on small sample learning to solve the above-mentioned problems of existing technology.

[0004] To achieve the above-mentioned purpose, the present application provides a remote sensing scene classification method based on small sample learning, comprising the following steps:

[0005] Obtain a remote sensing image dataset and divide it into a meta-training set, a meta-validation set, and a meta-test set;

[0006] An initial small sample classification model is constructed based on an improved MobileViT network, the improvement including adding a feature fusion module in the main backbone structure of the MobileViT network, introducing a channel attention mechanism in the MobileNetV2 module of the MobileViT network, and fusing local features with global features in the MobileViT module of the MobileViT network;

[0007] The initial small sample classification model is trained through a meta-training set and a meta-validation set, and is tested through a meta-test set, and a small sample classification model is obtained after the test is passed, and an image is classified through the small sample classification model to obtain an image classification result.

[0008] Optionally, the improved MobileViT network includes:

[0009] Multi-scale features are extracted through a hierarchical structure including the MobileNetV2 module and the MobileViT module;

[0010] The multi-scale features of different levels are upsampled to a unified resolution and then spliced to obtain fused features;

[0011] The fused features are adaptively weighted through the channel attention mechanism;

[0012] Class prototype vectors are calculated based on support set samples, and classification is achieved through distance measurement.

[0013] Optionally, the channel attention mechanism introduced in the MobileNetV2 module includes:

[0014] Global average pooling is performed on the input feature map to obtain channel statistics;

[0015] Channel weights are generated through a fully connected layer and an activation function;

[0016] The channel weights are multiplied by the original feature map to realize feature re-calibration.

[0017] Optionally, the MobileViT module includes connecting local features and global features along the channel direction through a shortcut branch to fuse local details and global context information.

[0018] Optionally, the model training adopts an N-way K-shot task training method, each task including a support set and a query set, features are extracted through the support set and class prototypes are calculated, classification is performed based on the similarity between the query set and the class prototypes, and model parameters are optimized.

[0019] Optionally, the method for calculating the class prototype includes taking the mean of all sample features of each class in the support set to obtain the prototype vector of the class.

[0020] The application further provides a remote sensing scene classification system based on small sample learning, comprising:

[0021] a dataset division module configured to obtain a remote sensing image dataset and divide the remote sensing image dataset into a meta training set, a meta validation set and a meta test set;

[0022] a model construction module configured to construct an initial small sample classification model according to an improved MobileViT network;

[0023] a training module configured to perform model training on the initial small sample classification model through the meta training set and the meta validation set;

[0024] a test module configured to test the trained model through the meta test set to obtain a small sample classification model;

[0025] a classification module configured to classify images through the small sample classification model to obtain image classification results.

[0026] Optionally, the improved MobileViT network comprises:

[0027] a feature extraction layer configured to extract multi-scale features through a hierarchical structure comprising a MobileNetV2 module and a MobileViT module;

[0028] a multi-scale feature fusion module configured to splice the features of different levels after up-sampling the features to a unified resolution to obtain fused features;

[0029] a dynamic attention weighting module configured to adaptively weight the fused features through a channel attention mechanism;

[0030] a prototype classification head configured to calculate class prototype vectors based on support set samples and realize classification through distance measurement.

[0031] The application further provides a computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the steps of the method.

[0032] The application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to realize the steps of the method.

[0033] Compared with the prior art, the application has the following advantages and technical effects:

[0034] Firstly, by means of the small sample learning framework, the application breaks through the dependence of traditional deep learning on large-scale labeled data, and only 1-5 labeled samples are needed to accurately identify new land object categories, thereby significantly reducing the artificial labeling cost and solving the problems of high labeling cost and sample scarcity of remote sensing images. Secondly, the improved MobileViT network structure enhances the collaborative modeling capability of the model for local details and global context, effectively improves the classification accuracy, and especially performs well when dealing with remote sensing images with significant intra-class differences and inter-class similarities. In addition, the introduced cross-layer connection and feature fusion mechanism optimizes the generalization ability of the model, so that it can maintain stable classification performance in multi-scale and multi-resolution remote sensing images across sensors and regions. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of this application, and their

[0036] Figure 1 An improved MobileViT network structure of an embodiment of the application;

[0037] Figure 2 A small sample learning flowchart of an embodiment of the application;

[0038] Figure 3 An improved MobileNetV2 module structure diagram of an embodiment of the application;

[0039] Figure 4 A structure diagram of a channel attention mechanism of an embodiment of the application;

[0040] Figure 5 An improved MobileViT module structure diagram of an embodiment of the application. DETAILED DESCRIPTION

[0041] It should be noted that the embodiments and features in the application can be combined with each other without conflict. The application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0043] The related technologies are introduced as follows:

[0044] MobileViT is a lightweight hybrid network architecture that skillfully combines the strengths of both Convolutional Neural Networks (CNNs) and Transformers, balancing local detail perception and global context modeling capabilities. Its core design consists of three parts:

[0045] Local feature extraction: MobileNetV2's depthwise separable convolution layers are used to efficiently extract local features such as image edges and textures, reducing computational load.

[0046] Global relationship modeling: The self-attention mechanism in lightweight Transformer blocks establishes long-range dependencies between pixels, capturing spatial layouts of ground objects such as building arrangements and river courses.

[0047] Cross-layer feature fusion: The alternating CNN-Transformer structure enables dynamic interaction between local and global features, avoiding the high computational overhead of pure Transformers.

[0048] Compared to traditional CNNs like ResNet, MobileViT reduces parameter count by over 70%, enabling both local detail recognition (e.g., vehicle outlines) and global scene correlation (e.g., road network distribution) in remote sensing image classification tasks. It is particularly suitable for resource-constrained small-sample learning scenarios, balancing high precision and edge device deployment efficiency.

[0049] Original MobileViT network structure:

[0050] MobileViT is a lightweight hybrid network architecture designed to combine the local perception capabilities of Convolutional Neural Networks (CNNs) with the global modeling advantages of Transformers, suitable for mobile devices and resource-constrained scenarios. MobileViT adopts a hierarchical design with multiple stages (Stage), each gradually expanding the receptive field and compressing spatial resolution while increasing the channel dimension. The typical structure is as follows:

[0051] Initial convolution layer: Conv3*3 in the figure, a standard 3x3 convolution that quickly extracts low-level features such as edges and corners.

[0052] MobileNetV2 module: MV2 in the figure, composed of multiple Inverted Residual Blocks, which use Depthwise Separable Convolution to extract local detail features. This lightweight approach reduces parameter count by over 80% compared to standard convolution, effectively extracting local texture and edge features.

[0053] MobileViT module: MobileViT block in the figure, which alternately uses convolution and lightweight Transformer block to realize local to global feature fusion. By self-attention, long-range context (such as road network and farmland distribution in remote sensing image) can be captured.

[0054] End convolution layer: Conv1*1 in the figure, which adjusts the channel dimension and feature fusion.

[0055] Global pooling linear layer: Global Pool Linear in the figure, which extracts the global statistics (mean) of each channel through global average pooling (GAP), so that the model is robust to spatial translation of the input image (such as the position of the building in the image does not affect the classification result). Linear layer (Linear) generates final classification score (Logits) for probability distribution after Softmax normalization.

[0056] Classification head: GAP followed by a fully connected layer to output Logits (unnormalized classification scores).

[0057] The application of the present application in smart agriculture is as follows:

[0058] The small sample learning based remote sensing scene classification method of the present application has important application value in the field of smart agriculture. Smart agriculture needs to monitor and accurately manage various agricultural scenes such as farmland, orchard and pasture in real time to improve agricultural production efficiency, optimize resource utilization and reduce environmental impact. However, traditional remote sensing image classification methods face many challenges in agricultural scenes, such as diverse farmland types, significant seasonal changes, and lack of labeled data, resulting in insufficient classification accuracy and difficulty in quickly adapting to new agricultural scenes.

[0059] The present application uses a small sample learning framework to quickly identify new agricultural scene categories with only a small amount of labeled samples (such as 1-5 images), significantly reducing labeling costs while improving the model's generalization ability. The improved MobileViT network structure can efficiently capture local texture features (such as crop planting patterns and irrigation ditches) and global spatial layout (such as farmland distribution and topography) of farmland, and enhance the ability to distinguish complex agricultural scenes through feature fusion mechanism. For example, when monitoring crop growth, the model can quickly identify crops at different growth stages and timely detect pest areas; in land use planning, it can accurately distinguish different types of farmland, woodland and water area, providing data support for rational allocation of agricultural resources.

[0060] In addition, the high efficiency and flexibility of the present application enable it to respond to dynamic changes in agricultural production in real time, such as sudden natural disasters or outbreaks of pests and diseases. By quickly classifying and analyzing remote sensing images, agricultural managers can take timely measures to reduce losses and optimize production processes. Experimental results show that the present application performs well in the classification task of agricultural-related scenes, significantly improving classification accuracy and work efficiency, and providing strong technical support for the precision management and sustainable development of smart agriculture.

[0061] Embodiment one

[0062] As Figures 1-5 shown, the present embodiment provides a remote sensing scene classification method based on small sample learning, including the following steps:

[0063] Obtain a remote sensing image dataset and divide it into a meta-training set, a meta-validation set and a meta-test set;

[0064] Construct an initial small sample classification model based on an improved MobileViT network, the improvements including adding a feature fusion module to the main backbone structure of the MobileViT network, introducing a channel attention mechanism to the MobileNetV2 module of the MobileViT network, and fusing local features with global features in the MobileViT module of the MobileViT network;

[0065] Train the initial small sample classification model through the meta-training set and the meta-validation set, and test it through the meta-test set. After the test is passed, a small sample classification model is obtained. The image is classified through the small sample classification model to obtain the image classification result.

[0066] The present embodiment uses a small sample remote sensing image scene classification method based on improved MobileViT,

[0067] Flow method (NWPU-RESISC45 dataset as an example):

[0068] 1. Dataset division;

[0069] Total dataset: 45 categories (such as airport, forest, etc.), 700 images per category.

[0070] Division ratio (example):

[0071] Meta-training set: 30 categories (used to train the meta-learning model).

[0072] Meta-validation set: 5 categories (used for hyperparameter tuning and early stopping).

[0073] Meta-test set: 10 categories (final test model generalization ability).

[0074] Key principles: three categories of categories are strictly mutually exclusive, meta-test categories are completely invisible in training and validation.

[0075] 2. Meta-Train phase;

[0076] (1) Task generation:

[0077] Generate a 5-way 1-shot task each iteration: select categories: randomly select 5 categories from the 30 categories of the meta-training set.

[0078] Sample the support set: select 1 picture per class (5 in total).

[0079] Query set: select 15 pictures per class (75 in total), non-overlapping with the support set.

[0080] Task format: input is a pair of support set and query set images, output is the classification result of the query set.

[0081] (2) Model training:

[0082] Model input support set (5 images) + query set (75 images).

[0083] Learning goal: learn task-specific features based on the support set. Calculate the loss on the query set to optimize the model's generalization ability.

[0084] Further, the model training uses the N-way K-shot task training method, each task includes a support set and a query set, the features are extracted through the support set and the class prototype is calculated, the classification is performed based on the similarity between the query set and the class prototype, and the model parameters are optimized.

[0085] N-way K-shot: way represents the number of categories, shot represents the number of pictures per category, 5-way 1-shot means 5 categories, each with 1 picture, in 5-way 1-shot task, two sets are defined, support set and query set, the goal of small sample learning task is to determine which category in the support set the query set picture belongs to.

[0086] The goal of few-shot classification is to enable the model to quickly adapt to new tasks with only a tiny amount of samples (e.g., 5 classes x 1 sample). The essential difference from traditional supervised learning is that traditional supervised learning inputs all classes and samples at once, and the model directly learns the classification boundary (e.g., ResNet classification). The training process is divided into a large number of sub-tasks (Episodes), each of which simulates a "few-shot scenario", and the model learns how to induce knowledge from a small amount of samples. Although only a small amount of data is used for each task, all meta-training classes and samples are covered multiple times through repeated random sampling. The optimization goal of meta-learning is the generalization ability at the task level, rather than the classification accuracy of a single task. Through the training of tens of thousands of tasks, the model learns how to quickly extract features from any 5 classes, rather than remembering the details of all classes.

[0087] The innovation and advantage of few-shot learning method in remote sensing image classification mainly lies in that it fundamentally changes the dependence of traditional deep learning on large-scale labeled data, and through simulating the ability of humans to quickly learn and generalize from a small amount of samples, it constructs a knowledge transfer paradigm that is closer to actual application scenarios.

[0088] The overall workflow of using MobileViT for few-shot learning is shown in Figure 2 .

[0089] 3. Meta-Val (Meta-Validation) phase;

[0090] (1) Task generation:

[0091] Generate validation tasks from the 5 classes of the meta-validation set, the method is the same as Meta-Train: generate 5-way 1-shot tasks each time.

[0092] (2) Validation process:

[0093] Fixed model parameters: use the trained model, do not perform any parameter updates.

[0094] Evaluation index: calculate the classification accuracy of the query set.

[0095] Purpose: adjust hyperparameters (such as learning rate, number of tasks).

[0096] 4. Meta-Test (Meta-Test) phase;

[0097] (1) Task generation:

[0098] Generate test tasks from the 10 classes of the meta-test set: generate multiple groups of 5-way 1-shot tasks (set 1000 tasks), ensure that all class combinations are covered.

[0099] (2) Test process:

[0100] Strictly zero-shot: the model has never seen the categories and examples of the meta-test set.

[0101] Evaluation metric: Reports the average classification accuracy.

[0102] Example results: The model achieves 70% accuracy on the 5-way 1-shot task, demonstrating its ability to quickly adapt to new categories.

[0103] Furthermore, in the MobileViT network structure, the features extracted by the three MobileViT modules are fused, and the improved network structure is as follows: Figure 1 As shown:

[0104] Feature extraction layer: The input remote sensing image (256×256×3) is preprocessed by a lightweight convolutional layer (Conv3x3) and then passes through five layers (Layer 1-5). Layers 1-2 use a pure MV2 (MobileNetV2 inverted residual module) stack, while Layers 3-5 are composed of a cascade of MV2 modules and MobileViT modules.

[0105] Multi-scale feature fusion module: Extract the MobileViT output features of Layer3 (32×32×256), Layer4 (16×16×384), and Layer5 (8×8×512), upsample them to a uniform resolution (32×32) through bilinear interpolation, align the number of channels through 1×1 convolution, and then splice them along the channel dimension to form a fused feature.

[0106] Dynamic attention weighting: After fusion of features, a channel attention module is introduced to generate channel weights through global average pooling and fully connected layers to achieve adaptive weighting of multi-scale features.

[0107] Prototype classification head: The fused features are input into the prototype head after global average pooling. The class prototype vector is calculated based on the support set samples, and the query image classification is achieved through the negative Euclidean distance metric.

[0108] Compared with the original MobileViT network, the improved network shows the following advantages in remote sensing small sample scene classification tasks:

[0109] Enhanced multi-scale feature complementarity: The original MobileViT relies on features from only a single layer (usually Layer 5), making it difficult to simultaneously capture high-resolution details (such as building textures) and low-resolution semantics (such as the overall layout of an airport). The improved network fuses multi-level features from Layers 3 to 5 to construct a hybrid representation that combines local details and global context, significantly improving its ability to discriminate complex scenes.

[0110] Small sample generalization optimization: Remote sensing images have significant intra-class differences (such as farmland in different seasons) and inter-class similarities (such as ports and wharfs). The original network is prone to overfitting due to insufficient feature diversity, while the fused features suppress noise interference (such as cloud cover) and strengthen discriminative regions (such as airport runway patterns) through attention mechanisms.

[0111] Further, the MV2 module is improved by adding an attention mechanism.

[0112] In the MobileViT architecture, the MV2 module is used as a building block to provide efficient feature extraction capabilities. This module adopts an "inverted bottleneck" structure, where features go through two steps of dimensionality increase and decrease during convolution. These techniques help reduce the computational complexity of the model while maintaining good performance. The improved algorithm adds a channel attention mechanism module to the original residual result, enhancing the representation ability of image channels. The improved MV2 module structure is as shown in Figure 3 .

[0113] Attention mechanism is used to establish a connection between two different input sequences. It can simultaneously focus on relevant elements in both sequences and learn their interaction. The role of channel attention mechanism is to enable the network to automatically identify and assign different importance to different channels, thereby more efficiently utilizing feature information. This mechanism achieves this by assigning different weights to the features of each channel, allowing the model to concentrate on more useful information and ignore relatively unimportant parts. This not only helps to improve the expressive power of the model, but also improves the model's understanding of complex data structures. The structure of the channel attention mechanism is as shown in Figure 4 .

[0114] The channel attention mechanism first uses global pooling on the input feature map to obtain a 1x1xN channel statistic. After reducing the dimensionality of the channel statistic by m times, it is then increased by m times. The activation function (sigmoid function) and (ReLU activation function) are used to ensure that the weight coefficients of each channel are normalized to between 0 and 1. In this way, the weight coefficient of each channel can be interpreted as a probability, indicating the importance of that channel. Finally, these weight coefficients are multiplied by each channel of the original input feature map. This step is feature re-labeling, which adaptively adjusts the contribution of each channel through learned channel weights. In this way, the model can emphasize the feature channels that are more important to the current task, while suppressing those that are not important.

[0115] Further, the improved MobileViT module is as shown in Figure 5 .

[0116] MobileViT is a lightweight hybrid network architecture that skillfully combines the local perception capabilities of convolutional neural networks (CNN) with the global modeling advantages of Transformers, designed specifically for mobile and resource-constrained scenarios. Its core module, the MobileViT Block, achieves local-to-global feature fusion through alternating convolution and Transformer operations. The structure of the MobileViT Block is shown in the figure, where the red lines represent the improvements made by the invention to the original MobileViT Block: Before extracting global feature information, we introduce a shortcut branch that directly connects the obtained global feature information with the local feature information along the channel direction. This improvement makes greater use of local features, allowing the network to more fully extract feature information from remote sensing scene images, especially in terms of capturing details.

[0117] After extracting image features through the MobileViT main network, small sample classification is achieved through metric learning. The core idea is that in the feature space, each class can be represented by the feature mean (prototype) of its support set samples, and classification is achieved by calculating the similarity between the query sample and all prototypes.

[0118] For an N-way K-shot task, the support set features , D is the feature dimension, and the prototype of each class is calculated as:

[0119] ;

[0120] where represents the feature of the nth class kth support sample.

[0121] For the query sample feature , calculate the negative Euclidean distance between it and all prototypes:

[0122] ;

[0123] The final classification probability is obtained by softmax normalization:

[0124] .

[0125] The following experiments were also conducted in this embodiment:

[0126] NWPU-RESISC45 is a dataset for remote sensing scene classification established by Northwestern Polytechnical University. The dataset contains 31,500 samples, divided into 45 categories, with 700 RGB images per category, and the sample size is 256x256. The dataset is divided into 25, 10, and 10 categories for training, validation, and testing.

[0127] UC Merced is a classic remote sensing scene classification dataset proposed by UC Merced Computer Vision Lab in 2010, which is used for classifying land use scenes in urban areas. The spatial resolution of the dataset is about 0.3m, the image size is 256x256, and it contains 21 classes of scenes, 100 RGB images for each class, a total of 2100 images. In the dataset, 10 classes are used for training, 6 classes are used for verification, and 5 classes are used for testing.

[0128] Experimental results:

[0129] The classification accuracy 5-way 1-shot on the NWPU-RESISC45 dataset is as shown in Table 1:

[0130] Table 1

[0131] Method Accuracy (%) MAML 45.78 Prototypical Network 46.21 MatchingNet 48.62 Relation Network 56.49 DeepEMD 62.75 ResNet50 57.93 Ours 74.67

[0132] The classification accuracy 5-way 1-shot on the UC Merced dataset is as shown in Table 2:

[0133] Table 2

[0134] Method Accuracy (%) MAML 47.84 Prototypical Network 46.87 MatchingNet 51.23 Relation Network 51.87 DeepEMD 59.75 ResNet50 52.17 Ours 75.16

[0135] According to the experimental results, the present application achieves the following effects:

[0136] 1. Breakthrough the dependence of traditional deep learning on large-scale labeled data: In view of the pain points of high labeling cost of remote sensing images and frequent dynamic addition of new classes, the small sample learning framework is used to accurately identify new ground object classes only by 1-5 labeled samples, which significantly reduces the artificial labeling cost.

[0137] 2. Improve the collaborative modeling ability of local features and global context: In view of the misclassification problem (such as confusing farmland and grassland with similar morphologies) caused by the limited field of view of local convolution in existing convolutional neural networks (such as ResNet) in remote sensing images, the MobileViT improved structure is designed to fuse lightweight convolution and self-attention mechanism, which can capture the correlation between ground object details and spatial layout.

[0138] 3. Optimize the cross-task adaptation efficiency of lightweight models: In view of the performance degradation caused by insufficient feature decoupling in existing small sample methods in cross-sensor and cross-geographical scene, the cross-layer connection and feature fusion mechanism are introduced to improve the generalization ability of the model to multi-scale and multi-resolution remote sensing images.

[0139] The present embodiment also proposes a remote sensing scene classification system based on small sample learning, which comprises:

[0140] A dataset division module is configured to obtain a remote sensing image dataset and divide the remote sensing image dataset into a meta-training set, a meta-validation set and a meta-testing set.

[0141] A model construction module is configured to construct an initial small sample classification model according to the improved MobileViT network.

[0142] A training module is configured to perform model training on the initial small sample classification model through the meta-training set and the meta-validation set.

[0143] A testing module is configured to test the trained model through the meta-testing set to obtain a small sample classification model.

[0144] A classification module is configured to classify an image through the small sample classification model to obtain an image classification result.

[0145] Further, the improved MobileViT network comprises:

[0146] A feature extraction layer is configured to extract multi-scale features through a hierarchical structure comprising a MobileNetV2 module and a MobileViT module.

[0147] A multi-scale feature fusion module is configured to splice the features of different levels after up-sampling to a unified resolution to obtain fused features.

[0148] A dynamic attention weighting module is configured to adaptively weight the fused features through a channel attention mechanism.

[0149] A prototype classification head is configured to calculate class prototype vectors based on support set samples and realize classification through distance measurement.

[0150] The embodiment also proposes a computer device comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to realize the steps of the method.

[0151] The embodiment also proposes a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to realize the steps of the method.

[0152] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A remote sensing scene classification method based on small sample learning, characterized in that: The following steps are involved: Obtain a remote sensing image dataset and divide it into a meta-training set, a meta-validation set, and a meta-test set; Building an initial small-shot classification model based on the improved MobileViT network. The improvements include adding a feature fusion module to the main backbone structure of the MobileViT network, introducing a channel attention mechanism in the MobileNetV2 module of the MobileViT network, and fusing local features with global features in the MobileViT module of the MobileViT network. The improved MobileViT network includes: Extract multi-scale features through a hierarchical structure consisting of MobileNetV2 modules and MobileViT modules; Multi-scale features at different levels are upsampled to a uniform resolution and then concatenated to obtain fused features; Adaptively weighting the fused features through a channel attention mechanism; The fused features are globally averaged and then input into the prototype head. The prototype vector is calculated based on the support set samples, and the query image classification is achieved using the negative Euclidean distance metric. The channel attention mechanism introduced in the MobileNetV2 module includes: Perform global average pooling on the input feature map to obtain channel statistics; Generate channel weights through fully connected layers and activation functions; Multiply the channel weight with the original feature map to achieve feature recalibration; The MobileViT module includes: connecting local features and global features along the channel direction through shortcut branches, fusing local details with global context information; The initial small sample classification model is trained using a meta-training set and a meta-validation set, and tested using a meta-test set. After passing the test, a small sample classification model is obtained, and the image is classified using the small sample classification model to obtain an image classification result; The model training adopts N-way K-shot task training method. Each task includes a support set and a query set. Features are extracted from the support set and the category prototype is calculated. Classification is performed based on the similarity between the query set and the category prototype, and the model parameters are optimized.

2. The method according to claim 1, characterized in that The method for calculating the category prototype includes: taking the mean of all sample features of each category in the support set to obtain the prototype vector of the category.

3. A remote sensing scene classification system based on small sample learning, characterized by: include: A dataset partitioning module is used to obtain a remote sensing image dataset and divide it into a meta-training set, a meta-validation set, and a meta-test set; A model building module for building an initial small-sample classification model based on the improved MobileViT network. The improvements include adding a feature fusion module to the main backbone structure of the MobileViT network, introducing a channel attention mechanism in the MobileNetV2 module of the MobileViT network, and fusing local features with global features in the MobileViT module of the MobileViT network. The improved MobileViT network includes: Extract multi-scale features through a hierarchical structure consisting of MobileNetV2 modules and MobileViT modules; Multi-scale features at different levels are upsampled to a uniform resolution and then concatenated to obtain fused features; Adaptively weighting the fused features through a channel attention mechanism; The fused features are globally averaged and then input into the prototype head. The prototype vector is calculated based on the support set samples, and the query image classification is achieved using the negative Euclidean distance metric. The channel attention mechanism introduced in the MobileNetV2 module includes: Perform global average pooling on the input feature map to obtain channel statistics; Generate channel weights through fully connected layers and activation functions; Multiply the channel weight with the original feature map to achieve feature recalibration; The MobileViT module includes: connecting local features and global features along the channel direction through shortcut branches, fusing local details with global context information; A training module, configured to perform model training on the initial small sample classification model using a meta-training set and a meta-validation set; The model training adopts N-way K-shot task training method. Each task includes a support set and a query set. Features are extracted from the support set and the category prototype is calculated. Classification is performed based on the similarity between the query set and the category prototype, and model parameters are optimized. The testing module is used to test the trained model using the meta-test set to obtain a small sample classification model; The classification module is used to classify images through a small sample classification model to obtain image classification results.

4. The system according to claim 3, characterized in that The improved MobileViT network includes: Feature extraction layer, which is used to extract multi-scale features through a hierarchical structure including MobileNetV2 modules and MobileViT modules; Multi-scale feature fusion module, used to upsample features at different levels to a unified resolution and then splice them to obtain fused features; A dynamic attention weighting module, configured to adaptively weight the fused features through a channel attention mechanism; The prototype classification head is used to calculate the class prototype vector based on the support set samples and perform classification through distance measurement.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 2.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images