A remote sensing scene classification method, system, storage medium and device

Abstract

The present invention discloses a remote sensing scene classification method, system, storage medium and device, the method comprising: obtaining high-resolution remote sensing image data and multi-source data, extracting remote sensing scene features based on the high-resolution remote sensing image data and the multi-source data, and characterizing the remote sensing scene features using a Lie group regional covariance feature matrix; extracting multi-layer and multi-scale features of long-distance feature information in high-resolution remote sensing image data using a parallel expansion large kernel convolution operation, and compressing the multi-layer and multi-scale features using a feature-level fusion method based on the mean within the Lie group; obtaining short-distance feature information of channels using a local channel separation and aggregation method to characterize the dependency relationship between channels, thereby classifying remote sensing scenes according to the dependency relationship. The present invention solves the problem that it is difficult to achieve an effective balance between remote sensing scene classification accuracy and model complexity, and can improve the classification accuracy of remote sensing scenes while reducing model complexity.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a remote sensing scene classification method, system, storage medium and equipment. Background Art

[0002] With the great success of convolutional neural network (CNN) models in the field of computer vision (CV), remote sensing scene classification models based on CNN have become one of the current mainstream models, and scholars have proposed many CNN-based methods for remote sensing scene classification. Then, scholars proposed a new Transformer based on a deep learning framework and applied it to the field of CV. Since the attention mechanism in Transformer can effectively calculate global long sequence inputs and learn the dependencies between them, scholars have recently proposed many remote sensing scene classification models based on Transformer.

[0003] Although the above models have achieved good classification results, they also have the following shortcomings: (1) Since the Transformer-based remote sensing scene classification model lacks convolution, it leads to inductive bias and requires more data sets than the CNN model. Therefore, its performance is slightly worse than that of the CNN model on a smaller number of data sets. (2) The Transformer-based remote sensing scene classification model is larger and more complex than the CNN model in terms of the number of model parameters and the amount of calculation. Therefore, its robustness is weaker in some resource-constrained scenarios. (3) Although the Transformer-based remote sensing scene classification model can effectively capture global information, it may also lose key local information in the scene. It can be seen that the CNN-based scene classification model is still one of the important research directions.

[0004] The above models and methods have achieved good results, but they still have challenges: (1) Some CNN-based models mainly focus on the high-level features of the scene, ignoring the shallow features of the scene (i.e., the bottom and middle layers), and mainly focus on the high-level semantic features in HRRSI, ignoring the effective feature information contained in multi-source data, which easily leads to confusion of some scenes with the same or similar geometric structure and spatial layout. For example, there are two office buildings with the same structure and spatial layout, one of which has all kinds of corporate office space on each floor, while the other has all kinds of service agencies on each floor. It is difficult for existing models to distinguish the categories of these scenes. (2) Although some models use convolution to better fit the statistical data in HRRSI, it may also prevent the model from obtaining global context information. Both global context information and local information in complex spatial distribution scenes are very important for scene classification. (3) Some CNN-based models usually use continuous convolution to cover HRRSI, but in fact the receptive field in CNN is narrower than the theoretical receptive field, and the use of large kernel convolution will also increase the parameters and computational complexity of the model. Therefore, it is difficult to achieve an effective balance between the classification accuracy and model complexity of the model. Summary of the invention

[0005] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a remote sensing scene classification method, system, storage medium and device, aiming to solve the problem in the prior art that it is difficult to achieve an effective balance between the remote sensing scene classification accuracy and model complexity, thereby improving the classification accuracy of the remote sensing scene while reducing the model complexity.

[0006] A first aspect of the present invention is to provide a remote sensing scene classification method, the method comprising:

[0007] Acquire high-resolution remote sensing image data and multi-source data, extract remote sensing scene features based on the high-resolution remote sensing image data and the multi-source data, and characterize the remote sensing scene features using a Lie group regional covariance feature matrix;

[0008] A parallel expansion large kernel convolution operation is used to extract multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data, and a feature-level fusion method based on the inner mean of a Lie group is used to compress the multi-layer and multi-scale features;

[0009] The short-range feature information of the channels is obtained by local channel separation and aggregation to characterize the dependency relationship between the channels, so as to classify the remote sensing scene according to the dependency relationship.

[0010] According to one aspect of the above technical solution, the steps of obtaining high-resolution remote sensing image data and multi-source data, extracting remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterizing the remote sensing scene features using a Lie group regional covariance feature matrix include:

[0011] Acquire high-resolution remote sensing image data and multi-source data respectively;

[0012] Based on the high-resolution remote sensing image data and the multi-source data, texture features, shape features, compactness features of various remote sensing scenes in the remote sensing image, and internal socioeconomic semantic features of target objects in the remote sensing scene are obtained to obtain remote sensing scene features;

[0013] The Lie group regional covariance is used to characterize the remote sensing scene characteristics.

[0014] According to one aspect of the above technical solution, the step of obtaining the internal socio-economic semantic features of the target object in the remote sensing scene includes:

[0015] Acquire social media information and volunteer geographic information corresponding to the target object in the remote sensing scene;

[0016] According to the social media information and the volunteer geographic information, the category information of the target object is identified to determine the internal socio-economic semantic features according to the category information.

[0017] According to one aspect of the above technical solution, the steps of extracting multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data by using parallel dilated large-kernel convolution operations, and compressing the multi-layer and multi-scale features by using a feature-level fusion method based on the mean within the Lie group include:

[0018] Based on the preset global self-attention module, the feature map is divided into four parts along the channel dimension to obtain a first channel, a second channel, a third channel and a fourth channel, wherein the first channel and the third channel represent low-frequency features and high-frequency features respectively;

[0019] A global deep convolution adaptive pooling operation is applied to the second channel and the third channel to simulate a global context key feature, and a weight value of each channel is calibrated using the global context key feature to achieve adaptive feature selection;

[0020] The calibrated weight values ​​are added to the low-frequency features of the first channel and the high-frequency features of the fourth channel respectively, so as to realize the interaction of feature information of different partition parts through feature multiplexing;

[0021] The multi-layer and multi-scale features are compressed by adopting a feature-level fusion method based on the inner mean of the Lie group.

[0022] According to one aspect of the above technical solution, the steps of obtaining short-range feature information of channels by local channel separation and aggregation to characterize the dependency relationship between channels, and then classifying the remote sensing scene according to the dependency relationship include:

[0023] The preset local self-attention module is used to combine the feature information and spatial attention in different channels through channel splitting and fusion;

[0024] Output the feature information of key areas in a specific remote sensing scene and ignore the feature information of non-related areas to enhance the semantic feature information of complex spatial distribution and obtain short-range feature information to characterize the dependency relationship between channels;

[0025] The remote sensing scenes are classified according to the dependency relationships.

[0026] According to one aspect of the above technical solution, the steps of combining feature information and spatial attention in different channels by channel splitting and fusion using a preset local self-attention module include:

[0027] Based on the preset local self-attention module, the layer normalization operation is performed to divide the feature map into several feature parts along the channel;

[0028] Acquire a specific semantic response of the feature part during model training, and generate a spatial attention corresponding to the feature part;

[0029] A point-by-point multi-layer perception module is used to perform channel modeling on the features of the feature part to enhance the feature fusion of the channel dimension space, and the obtained fusion features are connected to represent an efficient spatial descriptor;

[0030] Through the 3×3 parallel expansion convolution operation, a pair of parameters is added to each feature part to move the weight values ​​of different parts;

[0031] The spatial attention feature value is generated through Lie group Sigmoid activation function and channel aggregation to enhance the information exchange of each feature part, and the residual connection is used to connect all feature parts so that the feature information in different channels can be combined with the spatial attention.

[0032] A second aspect of the present invention is to provide a remote sensing scene classification system, which is applied to the method described in the above technical solution, and the system comprises:

[0033] A data characterization module is used to obtain high-resolution remote sensing image data and multi-source data, extract remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterize the remote sensing scene features using a Lie group regional covariance feature matrix;

[0034] A feature extraction module, for extracting multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data by using parallel expansion large-kernel convolution operations, and compressing the multi-layer and multi-scale features by using a feature-level fusion method based on the inner mean of a Lie group;

[0035] The scene classification module is used to obtain short-range feature information of channels by adopting local channel separation and aggregation to characterize the dependency relationship between channels, so as to classify the remote sensing scene according to the dependency relationship.

[0036] According to one aspect of the above technical solution, the data characterization module is specifically used for:

[0037] Acquire high-resolution remote sensing image data and multi-source data respectively;

[0038] Based on the high-resolution remote sensing image data and the multi-source data, texture features, shape features, compactness features of various remote sensing scenes in the remote sensing image, and internal socioeconomic semantic features of target objects in the remote sensing scene are obtained to obtain remote sensing scene features;

[0039] The Lie group regional covariance is used to characterize the remote sensing scene characteristics.

[0040] The third aspect of the present invention is to provide a readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the above technical solution.

[0041] The fourth aspect of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above technical solution when executing the program.

[0042] Compared with the prior art, the remote sensing scene classification method, system, storage medium and device shown in the present invention have the following beneficial effects:

[0043] A remote sensing scene classification model is designed by focusing on key feature information in HRRSI from the perspective of multi-source data, optimizing feature representation space, global self-attention mechanism and local self-attention mechanism. First, combining the characteristics of HRRSI itself and multi-source data (such as social media and volunteer geographic data), the important feature information in the image is selectively focused, and the efficient Lie group regional covariance feature matrix is ​​used to represent it. Then, a global self-attention mechanism is designed, and the traditional convolution is replaced by parallel dilated large kernel convolution, which can adaptively extract richer spatial context feature information under a larger receptive field, and an adaptive aggregation operation is used to adapt to the global environment, and the weight of each channel is recalibrated in combination with the global context feature information. As a supplement to the global self-attention mechanism, in the local self-attention mechanism, channel separation and aggregation are used to capture the feature relationship between channels. This mechanism is mainly used to simulate the local spatial dependency, selectively focus on the features of key areas, and ignore irrelevant areas, capturing and enhancing the different semantic distributions of each feature group in the spatial dimension. Experiments on four widely used and challenging datasets show that the model proposed in this paper is competitive compared with other general models. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0045] Figure 1 It is a flowchart of the remote sensing scene classification method in the first embodiment of the present invention;

[0046] Figure 2 It is a structural block diagram of the remote sensing scene classification system in the second embodiment of the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0048] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0050] Embodiment 1

[0051] See also Figure 1 The first embodiment of the present invention provides a remote sensing scene classification method, the method comprising steps S10 to S30:

[0052] Step S10, obtaining high-resolution remote sensing image data and multi-source data, extracting remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterizing the remote sensing scene features using a Lie group regional covariance feature matrix.

[0053] In this embodiment, the remote sensing scene classification method performs remote sensing classification based on a remote sensing scene classification model, which includes a feature extraction module, a global self-attention module, and a local self-attention module. Among them, the feature extraction module is used to extract the features of the scene by combining HRRSI and multi-source data (such as social media and volunteer geographic data), and characterize it using the Lie group regional covariance feature matrix; the global self-attention module is used to extract the global context key features (long-distance feature information) of HRRSI using parallel expansion large kernel convolution operations, and compress it using a feature-level fusion method based on the mean within the Lie group; and the local self-attention module is used to capture channel dependencies (short-distance feature information) by using local channel separation and aggregation.

[0054] Specifically, the remote sensing scene features are derived from high-resolution remote sensing impact data (HRRSI) and multi-source data. Prior art shows that the category of the scene is strongly correlated with shallow features (bottom-level and middle-level features) such as position, color, and gradient change [TGRS, RS, etc.]. Therefore, in this embodiment, some bottom-level and middle-level features in the scene are retained, such as coordinate position, color, etc. In addition, in order to better distinguish scenes with the same or similar geometric structure and spatial layout, various features of the scene are supplemented and optimized. For example, target objects (such as buildings) exhibit different spectral characteristics, so the standard deviation and mean of the red, green, and blue bands are extracted in this embodiment. Then, the spatial-spectral features in HRRSI are fully learned, and three-dimensional Gabor filtering is introduced.

[0055] In order to obtain more features without increasing the parameters and computational complexity of the model, 3D-GMK is applied in this embodiment, which includes 13 directions. In combination with GMK, an enhanced feature map can be obtained without increasing the parameters. Different scenes show different texture structures [TGRS, RS, etc.]. Therefore, eight common texture features of the scene are extracted in this embodiment, such as homogeneity, dissimilarity, etc., and are calculated using the gray-level co-occurrence matrix (GLCM). In addition, the shape structure of the scene is also closely related to the category of the scene, so eight typical shape features of the scene are also extracted in this embodiment, such as roundness, Boyce-Clark index, etc.

[0056] In order to solve the problem that some scenes with the same or similar geometric structure and spatial layout are easily confused, for example, two office buildings with the same structure and spatial layout, one of which has all kinds of corporate office space on each floor, and the other has all kinds of service agencies on each floor, it is difficult for traditional models to distinguish the categories of such remote sensing scenes. In fact, the category of remote sensing scenes is closely related to the internal socio-economic semantic features of the scene in addition to external physical structure and other features. Therefore, in this embodiment, the socio-economic semantic features of the scene are extracted from social media data and volunteer geographic information (VGI) and other data.

[0057] Specifically, VGI data and social media data are updated in real time and are free, and can reflect the category information of the scene from different perspectives. Point of interest data (POI data) contains rich semantic feature information, such as category, density, and location. However, POI data also has shortcomings, such as the lack of specific natural attributes.

[0058] In order to solve this problem, the present embodiment also extracts high-level semantic features of the scene. The scene features need not only to extract their internal attributes, but also to extract features between different target objects in the scene. In order to effectively distinguish the compactness of the distribution of different target objects (such as sparse residential buildings), the minimum spanning tree (MST) algorithm is introduced to calculate the shortest distance between different target objects, which is used to characterize the spatial relationship between different target objects.

[0059] Step S20, using parallel expansion large kernel convolution operation to extract multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data, and compressing the multi-layer and multi-scale features using a feature-level fusion method based on the inner mean of the Lie group.

[0060] In this embodiment, when the global self-attention module is used for multi-layer and multi-scale feature extraction and compression, the feature map is divided into four parts along the channel dimension, namely the first channel fm1, the second channel fm2, the third channel fm3 and the fourth channel fm4. In order to better cope with the complex spatial distribution and global environment of HRRSI, adaptive pooling, parallel expansion of large kernel convolution, Lie group SIGMOID kernel function and feature-level fusion are used to extract global context key features in complex space.

[0061] The first channel fm1 and the fourth channel fm4 represent low-frequency features and high-frequency features, respectively. For the first channels fm1 and fm4, this embodiment uses a large kernel convolution combined with an expansion rate operation. There are two main reasons: (1) Parallel expansion convolution can effectively increase the receptive field, and the number of parameters is much smaller than that of traditional convolution; (2) The features extracted in a deeper and larger receptive field are beneficial to the recognition and classification of scenes, and can learn more refined, more fine-grained and more discriminative features.

[0062] Among them, traditional methods usually use global average pooling to simulate global context feature information and environment, however, this is not enough for the complex spatial distribution and geological characteristics of the scene in HRRSI. In order to better simulate the global context feature information and environment in different scenes, the global deep convolution adaptive pooling operation is used to simulate the global context information for the second channel fm2 and the third channel fm3, and this feature information is used to calibrate the weight of each channel. Specifically, the CONV7 operation is performed to make the above feature information interact between different spatial descriptors, and the Lie group Sigmoid activation function is used to obtain the corresponding weights.

[0063] The weights obtained above are used to realize adaptive feature selection. Then, the values ​​are added to the low-frequency features of the first channel fm1 and the high-frequency features of the fourth channel fm4, and the feature information interaction of different partitions is realized by feature multiplexing.

[0064] Step S30, using local channel separation and aggregation to obtain short-range feature information of channels to characterize the dependency relationship between channels, thereby classifying the remote sensing scene according to the dependency relationship.

[0065] In this embodiment, a local self-attention module is used to extract short-range feature information to characterize the dependency between channels, that is, to selectively focus on key areas in the scene and ignore irrelevant areas, and to classify the remote sensing scene based on the dependency.

[0066] As a supplement to the global self-attention module, the local self-attention module combines the feature information in different channels with the spatial attention through channel splitting and fusion, focusing on the feature information of key areas in the scene in a targeted manner and ignoring the feature information of irrelevant areas, so as to enhance the semantic feature information distributed in complex space.

[0067] More specifically, the local self-attention module first divides the feature map obtained in the early stage into G parts along the channel through layer normalization operation, and captures the specific semantic response of the features of these G parts during the model training process, that is, generates the spatial attention corresponding to the features of these G parts. Then, the point-by-point multi-layer perception module is used to model the channels of the features in the above parts, enhance the feature fusion of the channel dimension space, and connect the obtained features to represent efficient spatial descriptors. Then, through a 3*3 parallel expansion convolution operation, inspired by previous studies, each part adds a pair of parameters to move the weights of different parts, which is mainly to prevent the coefficient deviation between different data samples from being insignificant, and the amount of parameters introduced by this operation is also small. Then, the spatial attention feature value is generated through the Lie group Sigmoid activation function and channel aggregation, which enhances the information exchange of each feature.

[0068] Compared with the prior art, the remote sensing scene classification method shown in this embodiment has the following beneficial effects:

[0069] The remote sensing scene classification model is designed by focusing on the key feature information in HRRSI and optimizing the feature representation space, global self-attention mechanism and local self-attention mechanism from the perspective of multi-source data. First, combining the characteristics of HRRSI itself and multi-source data (such as social media and volunteer geographic data), the important feature information in the image is selectively focused, and the efficient Lie group regional covariance feature matrix is ​​used for representation. Then, a global self-attention mechanism is designed, and the traditional convolution is replaced by parallel expansion large kernel convolution, which can adaptively extract richer spatial context feature information under a larger receptive field, and an adaptive aggregation operation is used to adapt to the global environment, and the weight of each channel is recalibrated in combination with the global context feature information. As a supplement to the global self-attention mechanism, in the local self-attention mechanism, channel separation and aggregation are used to capture the feature relationship between channels. This mechanism is mainly used to simulate the dependency relationship of the local space, selectively focus on the features of the key areas, ignore irrelevant areas, and capture and enhance the different semantic distributions of each feature group in the spatial dimension. Experiments on four widely used and challenging datasets show that the model proposed in this embodiment is competitive compared with other general models.

[0070] Embodiment 2

[0071] See also Figure 2 A second embodiment of the present invention provides a remote sensing scene classification system, which includes: a data representation module 10, a feature extraction module 20 and a scene classification module 30.

[0072] A data characterization module 10 is used to obtain high-resolution remote sensing image data and multi-source data, extract remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterize the remote sensing scene features using a Lie group regional covariance feature matrix;

[0073] A feature extraction module 20 is used to extract multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data by using a parallel expansion large-kernel convolution operation, and compress the multi-layer and multi-scale features by using a feature-level fusion method based on the inner mean of a Lie group;

[0074] The scene classification module 30 is used to obtain short-range feature information of channels by adopting a local channel separation and aggregation method to characterize the dependency relationship between channels, thereby classifying the remote sensing scene according to the dependency relationship.

[0075] In this embodiment, the data representation module 10 is specifically used for:

[0076] Acquire high-resolution remote sensing image data and multi-source data respectively;

[0077] Based on the high-resolution remote sensing image data and the multi-source data, texture features, shape features, compactness features of various remote sensing scenes in the remote sensing image, and internal socioeconomic semantic features of target objects in the remote sensing scene are obtained to obtain remote sensing scene features;

[0078] The Lie group regional covariance is used to characterize the remote sensing scene characteristics.

[0079] The data representation module 10 is also used for:

[0080] Acquire social media information and volunteer geographic information corresponding to the target object in the remote sensing scene;

[0081] According to the social media information and the volunteer geographic information, the category information of the target object is identified to determine the internal socio-economic semantic features according to the category information.

[0082] In this embodiment, the data extraction module 20 is specifically used for:

[0083] Based on the preset global self-attention module, the feature map is divided into four parts along the channel dimension to obtain a first channel, a second channel, a third channel and a fourth channel, wherein the first channel and the third channel represent low-frequency features and high-frequency features respectively;

[0084] A global deep convolution adaptive pooling operation is applied to the second channel and the third channel to simulate a global context key feature, and a weight value of each channel is calibrated using the global context key feature to achieve adaptive feature selection;

[0085] The calibrated weight values ​​are added to the low-frequency features of the first channel and the high-frequency features of the fourth channel respectively, so as to realize the interaction of feature information of different partition parts through feature multiplexing;

[0086] The multi-layer and multi-scale features are compressed by adopting a feature-level fusion method based on the inner mean of the Lie group.

[0087] In this embodiment, the scene classification module 30 is specifically used for:

[0088] The preset local self-attention module is used to combine the feature information and spatial attention in different channels through channel splitting and fusion;

[0089] Output the feature information of key areas in a specific remote sensing scene and ignore the feature information of non-related areas to enhance the semantic feature information of complex spatial distribution and obtain short-range feature information to characterize the dependency relationship between channels;

[0090] The remote sensing scenes are classified according to the dependency relationships.

[0091] The scene classification module 30 is also used for:

[0092] Based on the preset local self-attention module, the layer normalization operation is performed to divide the feature map into several feature parts along the channel;

[0093] Acquire a specific semantic response of the feature part during model training, and generate a spatial attention corresponding to the feature part;

[0094] A point-by-point multi-layer perception module is used to perform channel modeling on the features of the feature part to enhance the feature fusion of the channel dimension space, and the obtained fusion features are connected to represent an efficient spatial descriptor;

[0095] Through the 3×3 parallel expansion convolution operation, a pair of parameters is added to each feature part to move the weight values ​​of different parts;

[0096] The spatial attention feature value is generated through Lie group Sigmoid activation function and channel aggregation to enhance the information exchange of each feature part, and the residual connection is used to connect all feature parts so that the feature information in different channels can be combined with the spatial attention.

[0097] Compared with the prior art, the remote sensing scene classification system shown in this embodiment has the following beneficial effects:

[0098] The remote sensing scene classification model is designed by focusing on the key feature information in HRRSI and optimizing the feature representation space, global self-attention mechanism and local self-attention mechanism from the perspective of multi-source data. First, combining the characteristics of HRRSI itself and multi-source data (such as social media and volunteer geographic data), the important feature information in the image is selectively focused, and the efficient Lie group regional covariance feature matrix is ​​used for representation. Then, a global self-attention mechanism is designed, and the traditional convolution is replaced by parallel expansion large kernel convolution, which can adaptively extract richer spatial context feature information under a larger receptive field, and an adaptive aggregation operation is used to adapt to the global environment, and the weight of each channel is recalibrated in combination with the global context feature information. As a supplement to the global self-attention mechanism, in the local self-attention mechanism, channel separation and aggregation are used to capture the feature relationship between channels. This mechanism is mainly used to simulate the dependency relationship of the local space, selectively focus on the features of the key areas, ignore irrelevant areas, and capture and enhance the different semantic distributions of each feature group in the spatial dimension. Experiments on four widely used and challenging datasets show that the model proposed in this embodiment is competitive compared with other general models.

[0099] Embodiment 3

[0100] A third embodiment of the present invention provides a readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the method described in the above embodiment.

[0101] Embodiment 4

[0102] A fourth embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiment when executing the program.

[0103] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0104] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A remote sensing scene classification method, characterized in that: The method comprises: Acquire high-resolution remote sensing image data and multi-source data, extract remote sensing scene features based on the high-resolution remote sensing image data and the multi-source data, and characterize the remote sensing scene features using a Lie group regional covariance feature matrix; A parallel expansion large kernel convolution operation is used to extract multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data, and a feature-level fusion method based on the inner mean of a Lie group is used to compress the multi-layer and multi-scale features; The short-range feature information of the channels is obtained by using a local channel separation and aggregation method to characterize the dependency relationship between the channels, thereby classifying the remote sensing scene according to the dependency relationship; The steps of obtaining high-resolution remote sensing image data and multi-source data, extracting remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterizing the remote sensing scene features using a Lie group regional covariance feature matrix include: Acquire high-resolution remote sensing image data and multi-source data respectively; Based on the high-resolution remote sensing image data and the multi-source data, texture features, shape features, compactness features of various remote sensing scenes in the remote sensing image, and internal socioeconomic semantic features of target objects in the remote sensing scene are obtained to obtain remote sensing scene features; Using Lie group regional covariance to characterize the remote sensing scene features; The steps of extracting multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data by using parallel dilated large-kernel convolution operations, and compressing the multi-layer and multi-scale features by using a feature-level fusion method based on the mean within the Lie group include: Based on the preset global self-attention module, the feature map is divided into four parts along the channel dimension to obtain a first channel, a second channel, a third channel and a fourth channel, wherein the first channel and the third channel represent low-frequency features and high-frequency features respectively; A global deep convolution adaptive pooling operation is applied to the second channel and the third channel to simulate a global context key feature, and a weight value of each channel is calibrated using the global context key feature to achieve adaptive feature selection; The calibrated weight values ​​are added to the low-frequency features of the first channel and the high-frequency features of the fourth channel respectively, so as to realize the interaction of feature information of different partition parts through feature multiplexing; The multi-layer and multi-scale features are compressed by adopting a feature-level fusion method based on the mean within the Lie group; The step of obtaining short-range feature information of channels by local channel separation and aggregation to characterize the dependency relationship between channels and classifying the remote sensing scene according to the dependency relationship includes: The preset local self-attention module is used to combine the feature information and spatial attention in different channels through channel splitting and fusion; Output the feature information of key areas in a specific remote sensing scene and ignore the feature information of non-related areas to enhance the semantic feature information of complex spatial distribution and obtain short-range feature information to characterize the dependency relationship between channels; classifying the remote sensing scene according to the dependency relationship; Among them, the steps of combining feature information and spatial attention in different channels by using a preset local self-attention module through channel splitting and fusion include: Based on the preset local self-attention module, the layer normalization operation is performed to divide the feature map into several feature parts along the channel; Acquire a specific semantic response of the feature part during model training, and generate a spatial attention corresponding to the feature part; A point-by-point multi-layer perception module is used to perform channel modeling on the features of the feature part to enhance the feature fusion of the channel dimension space, and the obtained fusion features are connected to represent an efficient spatial descriptor; Through the 3*3 parallel expansion convolution operation, a pair of parameters is added to each feature part to move the weight values ​​of different parts; The spatial attention feature value is generated through Lie group Sigmoid activation function and channel aggregation to enhance the information exchange of each feature part, and the residual connection is used to connect all feature parts so that the feature information in different channels can be combined with the spatial attention.

2. The remote sensing scene classification method according to claim 1, characterized in that: The steps of obtaining the internal socio-economic semantic features of the target object in the remote sensing scene include: Acquire social media information and volunteer geographic information corresponding to the target object in the remote sensing scene; According to the social media information and the volunteer geographic information, the category information of the target object is identified to determine the internal socio-economic semantic features according to the category information.

3. A remote sensing scene classification system, characterized in that: The method applied to any one of claims 1-2, wherein the system comprises: A data characterization module is used to obtain high-resolution remote sensing image data and multi-source data, extract remote sensing scene features based on the high-resolution remote sensing image data and multi-source data, and characterize the remote sensing scene features using a Lie group regional covariance feature matrix; A feature extraction module, for extracting multi-layer and multi-scale features of long-distance feature information in the high-resolution remote sensing image data by using parallel expansion large-kernel convolution operations, and compressing the multi-layer and multi-scale features by using a feature-level fusion method based on the inner mean of a Lie group; The scene classification module is used to obtain short-range feature information of channels by adopting local channel separation and aggregation to characterize the dependency relationship between channels, so as to classify the remote sensing scene according to the dependency relationship.

4. The remote sensing scene classification system according to claim 3, characterized in that: The data characterization module is specifically used for: Acquire high-resolution remote sensing image data and multi-source data respectively; Based on the high-resolution remote sensing image data and the multi-source data, texture features, shape features, compactness features of various remote sensing scenes in the remote sensing image, and internal socioeconomic semantic features of target objects in the remote sensing scene are obtained to obtain remote sensing scene features; The Lie group regional covariance is used to characterize the remote sensing scene characteristics.

5. A readable storage medium having computer instructions stored thereon, characterized in that: When the instruction is executed by the processor, the steps of the method according to any one of claims 1 to 2 are implemented.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 2 are implemented.

Images