Remote sensing image scene classification method and system based on curved surface-frequency domain information map

Abstract

This invention discloses a method and system for classifying remote sensing image scenes based on surface-frequency domain information graphs. The method includes acquiring a feature map of the remote sensing image, performing feature enhancement, and converting it into a high-dimensional manifold surface graph structure; extracting features from the high-dimensional manifold surface graph structure to obtain graph structure features, and classifying the remote sensing image scene based on these graph structure features. This application overcomes the limitation of traditional feature modeling methods that only perform local aggregation in Euclidean space. By innovatively introducing Fourier transform for frequency domain spectrum optimization and combining it with surface representation association to construct a dynamic scene topology graph, it can effectively extract intrinsic topological features unaffected by scene rotation angles, ensuring that the model can ignore directional changes and focus on the core semantic structure of the scene.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically to a method and system for classifying remote sensing image scenes based on surface-frequency domain information maps. Background Technology

[0002] Remote sensing images have broad application prospects in fields such as national defense and security, environmental monitoring, and urban planning. Scene classification technology, as the core means of understanding the semantic content of high-resolution remote sensing images, has always been a research focus in the field of remote sensing.

[0003] Current mainstream remote sensing image scene classification methods are mainly based on convolutional neural networks (CNNs) to extract features. They obtain global feature representations of the image by stacking convolutional and pooling layers, and then determine the scene category. However, most existing scene classification methods are designed based on prior knowledge of natural image datasets (such as ImageNet), ignoring feature perturbations caused by the arbitrary orientation of scenes in remote sensing images and frequency domain noise introduced by the complexity of ground textures. This results in significant bottlenecks in the model's ability to extract rotation-invariant global scene representations.

[0004] In deep learning models, standard convolution operations and commonly used global pooling methods are primarily based on regular Euclidean grid structures. While this structure offers some robustness to translation, it severely lacks rotation invariance. In remote sensing images, typical scenes (such as "ports," "industrial areas," and "overpasses") are often arranged at arbitrary angles. When a scene rotates, its spatial distribution on the feature map undergoes drastic, non-linear changes. Due to the lack of intrinsic modeling capabilities for rotational geometric transformations, traditional convolutional networks often identify different rotational forms of the same scene as vastly different feature vectors when extracting scene features, leading to significant intra-class variability. Simultaneously, complex background textures and high-frequency noise (such as sea waves and cloud cover) can easily interfere with the model's capture of core ground features, resulting in feature confusion due to high inter-class similarity. Specifically, feature aggregation in Euclidean space struggles to overcome the representational misalignment caused by geometric rotation, failing to construct stable, topologically consistent scene descriptors.

[0005] Therefore, how to eliminate noise and overcome the limitations of mesh structure during scene feature extraction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] To address the challenges of rotation sensitivity and background noise interference in remote sensing image scene classification, this application proposes a remote sensing image scene classification method and system based on surface-frequency domain information maps that overcomes or at least partially solves these problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, embodiments of the present invention provide a remote sensing image scene classification method based on a surface-frequency domain information map, the steps of which include: After acquiring the feature map of the remote sensing image, performing feature enhancement, and converting it into a high-dimensional manifold surface map structure; Feature extraction is performed on the high-dimensional manifold surface graph structure to obtain graph structure features, and the remote sensing image scene is classified according to the graph structure features; wherein, converting to a high-dimensional manifold surface graph structure includes: Each spatial location in the enhanced feature map is mapped to a graph node, and the initial feature vector of each node is the multidimensional feature vector of the corresponding location in the enhanced feature map; Based on the feature vectors of the nodes, the manifold geodesic distance containing amplitude and direction differences in space is calculated; a local neighborhood set is determined for each node according to the manifold geodesic distance, and an edge set between nodes is constructed based on the local neighborhood set.

[0008] Preferably, the feature enhancement includes edge and texture feature enhancement as well as key feature enhancement in the frequency domain; The enhanced feature map is obtained by adding the edge and texture feature enhancement map and the key feature enhancement map.

[0009] Preferably, the gradient histogram (HOG) operator is used to enhance the edge and texture features of the remote sensing image feature map.

[0010] Preferably, a dual activation mechanism is used to enhance key features of the remote sensing image feature map. The dual activation mechanism includes a parallel gated activation function and a softmax function.

[0011] Preferably, the computational space includes manifold geodesic distances with differences in amplitude and direction, including:

[0012] In the formula, This represents the manifold geodesic distance between node i and node j. and For balance coefficient, Let i be the multidimensional feature vector of node i. Let be the multidimensional feature vector of node j.

[0013] Preferably, the remote sensing image scene is classified according to the graph structure features, including: Multi-layer feature extraction is performed on remote sensing images, and graph structure features corresponding to each layer of feature maps are obtained respectively; The system integrates the structural features of each layer of the graph from shallow to deep based on the exponential moving average, and classifies the remote sensing image scene based on the integrated features.

[0014] As a preferred method, focus loss is used to train the remote sensing image scene classification process based on surface-frequency domain information maps.

[0015] Secondly, embodiments of the present invention provide a remote sensing image scene classification system based on a surface-frequency domain information map, used to implement the remote sensing image scene classification method based on a surface-frequency domain information map as described in any of the preceding claims, including: A cascaded feature extraction module is used to perform multi-layer feature extraction on remote sensing images; Each feature extraction module is connected to a feature optimization module. The feature optimization module includes a surface-frequency correlation mapping unit and a multilayer perceptron. The surface-frequency correlation mapping unit is used to enhance the feature map of the remote sensing image and convert the enhanced features into a high-dimensional manifold surface map structure. It also extracts features from the high-dimensional manifold surface map structure to obtain graph structure features. The multilayer perceptron is used to classify the remote sensing image scene based on the graph structure features.

[0016] Preferably, the multilayer perceptrons are connected in sequence to integrate the graph structure features of each layer from shallow to deep based on the exponential moving average.

[0017] This invention provides a method and system for classifying remote sensing image scenes based on surface-frequency domain information maps. The aim is to combine frequency domain information to remove noise and utilize manifold geometry modeling rotation invariance to improve the accuracy of remote sensing image scene classification.

[0018] Compared with existing technologies, this application breaks through the limitation of traditional feature modeling methods that only perform local aggregation in Euclidean space. By innovatively introducing Fourier transform for frequency domain spectrum optimization and combining it with surface representation to construct a dynamic scene topology graph, it can effectively extract intrinsic topological features that are not affected by scene rotation angles, ensuring that the model can ignore directional changes and focus on the core semantic structure of the scene. Furthermore, the remote sensing image scene classification model constructed based on surface-frequency domain information graphs in this application can effectively improve robustness to scene rotation changes and anti-interference ability against complex texture backgrounds, thereby significantly improving the classification accuracy of complex categories such as "ports," "airports," and "residential areas."

[0019] The solution provided in this application offers a more reliable and efficient solution to the problem of automated recognition of large field-of-view and arbitrary-direction scenes in remote sensing images, and provides strong technical support for the intelligent interpretation of future remote sensing big data. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart of extracting map structure features from remote sensing image feature maps provided in an embodiment of the present invention; Figure 2 This is a structural diagram of a remote sensing image scene classification model based on surface-frequency correlation mapping unit optimization provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] To address the limitations of existing remote sensing image scene classification models in capturing rotation-invariant features and their susceptibility to high-frequency noise interference from complex terrain textures, which restricts their performance in classifying scenes in arbitrary directions, this invention discloses a remote sensing image scene classification method and system based on a surface-frequency domain information map. Specific embodiments are provided for illustration.

[0024] Example 1 In this embodiment, the remote sensing image scene classification method based on surface-frequency domain information map includes the following steps: After acquiring the feature map of the remote sensing image, performing feature enhancement, and converting it into a high-dimensional manifold surface map structure; Feature extraction is performed on the high-dimensional manifold surface graph structure to obtain graph structure features, and the remote sensing image scene is classified according to the graph structure features.

[0025] In one specific embodiment, the process of extracting map structure features from a remote sensing image feature map is as follows: Figure 1 As shown; in this embodiment, the remote sensing image feature map is first enhanced, including the enhancement of key features in the frequency domain and the enhancement of edge and texture features.

[0026] In some implementations, in order to eliminate high-frequency noise introduced by background clutter in remote sensing images and enhance the global semantic response of features of complex remote sensing scenes in the frequency domain, feature vectors are projected into the frequency domain for optimization to enhance key features.

[0027] In this implementation scheme, a dual activation mechanism is employed to enhance key features of the remote sensing image feature map. Specifically, this enhances the feature map extracted from the depth features of the remote sensing image. The size is First, frequency domain information is obtained based on the discrete Fourier transform. The calculation process is expressed as follows:

[0028]

[0029] In the formula, For the first multilayer perceptron, For Discrete Fourier Transform, This is the frequency domain representation of the feature map. This refers to the frequency domain feature information after filtering by a multilayer perceptron.

[0030] Then, a dual activation mechanism is employed, combining gated convolution and... Further filtering of information to highlight the highlight features under different activation suppressions can be formally represented as follows:

[0031]

[0032]

[0033] In the formula, Represents the gating activation mechanism. For activation function, and The feature maps output by each activation function. This is the key feature enhancement map output after feature addition.

[0034] Furthermore, the frequency domain features are mapped back to the original feature space via the inverse Fourier transform module. The calculation process can be formally represented as follows:

[0035] in, For inverse Fourier transform, This is a key feature enhancement graph representation after mapping back to the original feature space.

[0036] In some implementations, the gradient histogram (HOG) operator is used to modify the original feature map. The process involves processing to enhance edge information and highlight the textural differences of ground features. The calculation process can be formally represented as follows:

[0037] in For gradient histogram operators, For the second multilayer perceptron, Enhanced feature maps for edges and textures. Furthermore, fused frequency-domain optimized feature information. and edge optimization feature information Based on this, a graph representation is constructed, and the computation process can be formally represented as:

[0038] in, Feature information optimized for fusion in the frequency domain and edge optimization feature information The enhanced feature map.

[0039] Furthermore, to overcome the limitation of Euclidean spatial distance in representing complex land cover structures, this invention introduces a mapping mechanism based on high-dimensional manifold metrics, transforming the enhanced features into a high-dimensional manifold surface map structure. That is, for each input sub-enhanced feature map... ,have

[0040] For a graph construction mechanism based on high-dimensional manifold metrics, These are the graph structure features obtained after graph construction.

[0041] In one embodiment, the conversion process is as follows: Each spatial location, that is, each pixel. Consider it as a node in a graph To facilitate subsequent mathematical expression and calculation, two-dimensional spatial coordinates will be used. The pixel at that location is mapped to a one-dimensional index. The nodes, each node The initial feature vector is the pixel's position in the feature map. On Dimensional feature vector In this way, all the pixels in the feature map constitute the node set of the graph. .

[0042] To construct a topology in the graph that can adapt to changes in target rotation, the optimized features are treated as points on a high-dimensional manifold surface. Nodes are defined by comprehensively considering the difference in feature magnitudes (amplitude information) and the surface angle (phase / direction information). and The surrogate value of the "manifold geodesic distance" between ;

[0043] in, and These are balancing coefficients, used to adjust the weights of characteristic amplitude differences and directional consistency, respectively. This represents the manifold geodesic distance between node i and node j. Let i be the multidimensional feature vector of node i. Let be the multidimensional feature vector of node j. This formula not only measures the absolute distance between nodes, but also captures the trend of feature variation on the manifold surface.

[0044] Based on the calculated manifold geodesic distance For each central node Select the K nearest nodes as its neighbors to construct a local neighborhood set. Based on this, we construct the edge set E of the graph, if the node... Belongs to node If the K nearest neighbors are found, then a directed edge is established between them. This mapping method, based on surface representation association and joint Fourier optimization, can effectively filter out background noise and establish robust semantic connections based on the intrinsic geometric structure of features, significantly improving the model's ability to represent rotating and deformed targets.

[0045] Furthermore, in a preferred embodiment, convolutional filtering is used to mine high-value information in the high-dimensional manifold surface graph structure, thereby obtaining graph structure features. The computation process can be formally represented as follows:

[0046] in, For convolution operators, This is a high-value feature map after convolutional filtering.

[0047] Example 2 To further optimize the above technical solution, this embodiment performs multi-layer feature extraction on the remote sensing image when classifying remote sensing image scenes based on the graph structure features, and obtains the graph structure features corresponding to each layer of feature map respectively; then, the graph structure features of each layer are integrated from shallow to deep based on the exponential moving average, and the remote sensing image scene is classified based on the integrated features.

[0048] In one specific embodiment, a surface-frequency correlation mapping unit capable of adapting to geometric rotation changes and possessing noise resistance is designed for feature enhancement of remote sensing image feature maps and for converting the enhanced features into a high-dimensional manifold surface map structure. Furthermore, the feature extraction network of the scene classification model is optimized based on this unit to address the inconsistency in representation when handling rotating scenes using existing methods.

[0049] Figure 2A structural diagram of a remote sensing image scene classification model based on surface-frequency correlation mapping unit optimization is provided. This classification model includes: A cascaded feature extraction module is used to perform multi-layer feature extraction on remote sensing images; Each feature extraction module is connected to a feature optimization module. The feature optimization module includes a surface-frequency correlation mapping unit and a multilayer perceptron. The multilayer perceptrons are connected in sequence and are used to integrate the structural features of each layer from shallow to deep based on the exponential moving average. Then, the remote sensing image scene is classified according to the integrated features.

[0050] In a preferred embodiment, the classification process is as follows: First, for the input remote sensing image G, which has dimensions H×W and 3 channels, feature extraction is performed through a series of stacked convolutional blocks or Transformer blocks to obtain a series of feature maps. Among them, the definition These are shallow, medium, and deep feature maps obtained after feature extraction, each containing feature information.

[0051] The proposed surface-frequency correlation mapping unit will then be used to optimize shallow, medium, and deep feature maps, formally represented as:

[0052] in, This represents the graph representation feature information corresponding to the i-th layer. The proposed surface-frequency correlation mapping unit is used.

[0053] Then, a multilayer perceptron is used to refine the feature information in layers, which is formally represented as:

[0054]

[0055]

[0056] This represents the refined feature map corresponding to the i-th layer. This represents the multilayer perceptron model corresponding to level i. To alleviate information misalignment and semantic ambiguity in the subgraph structures and further improve the model's decoupling ability for classification tasks, a feature integration mechanism based on Exponential Moving Average (EMA) is introduced. This mechanism utilizes the smoothing properties of EMA to dynamically weight and average the features of different branches to construct a globally consistent semantic prototype. This not only eliminates random noise in single-layer features but also promotes the effective complementarity between deep abstract semantics and shallow detailed information.

[0057] Considering the stronger complementarity of features at adjacent scales, this application divides the features into two groups for consistent integration:

[0058]

[0059] in, and These represent the feature maps after EMA processing. Through this EMA-based information integration, the model can obtain more robust and consistent feature representations, laying a solid foundation for subsequent accurate classification.

[0060] at last, It is used for classification and is formally represented as:

[0061] in, This represents the classification information obtained from the prediction. This represents the classification function.

[0062] Example 3 In this embodiment, a focus loss is used to train the remote sensing image scene classification process based on the surface-frequency domain information map. Formalized as:

[0063] in, This represents focus loss, used for classification prediction in remote sensing image scene classification tasks. This represents the total loss used in network training. Once the network training continues until the loss no longer decreases, the training has reached a stable state, the training process ends, and the improved remote sensing scene classification model combining the surface-frequency domain information graph representation module is obtained.

[0064] Furthermore, the trained and improved remote sensing scene classification model is used to test the remote sensing image to be tested, as shown below:

[0065] in, This represents the improved remote sensing scene classification model that combines the surface-frequency domain information graph representation module after training. and These represent the remote sensing image to be tested and its corresponding classification result, respectively.

[0066] Compared with the prior art, the beneficial effects of the present invention include: 1. By combining frequency domain optimization and surface representation correlation metrics, deep feature processing is achieved. Instead of being limited to the absolute position of pixels in Euclidean space, it analyzes the intrinsic geometric relationships (such as geodesic distance) of the optimized feature vectors on manifold surfaces, explicitly capturing and aggregating global semantic relationships within the scene (such as key features and spatial layouts in different areas). This mechanism enables the model to organize features based on deep content similarity rather than spatial proximity, thereby reconstructing the complete topological structure of the scene in the feature space and laying the foundation for subsequent global scene understanding.

[0067] 2. Benefiting from the aforementioned dynamic graph topology based on surface representation, this invention can extract global representations that are intrinsically invariant to scene rotation and deformation. When dealing with scenes in remote sensing images with arbitrary viewpoints and complex feature arrangements (such as "ports," "industrial areas," and "farmland" from different angles), even if the scene rotates or its internal features undergo local deformation, the neighborhood relationships and global association patterns of its features in the topological space remain stable. This ensures that the model can obtain highly consistent and robust scene semantic responses, effectively solving the problems of "large intra-class differences" and feature misalignment caused by scene rotation in traditional methods, thereby significantly improving the accuracy and generalization ability of remote sensing image scene classification.

[0068] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0069] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image scene classification method based on surface-frequency domain information map, characterized in that, After acquiring the feature map of the remote sensing image, performing feature enhancement, and converting it into a high-dimensional manifold surface map structure; Feature extraction is performed on the high-dimensional manifold surface graph structure to obtain graph structure features, and the remote sensing image scene is classified based on the graph structure features; wherein, The feature enhancement includes first obtaining frequency domain information based on discrete Fourier transform, and then using a dual activation mechanism, combining gated convolution and Softmax to further filter information, so as to highlight the bright features under different activation suppression. Transforming into a high-dimensional manifold surface graph structure includes: Each spatial location in the enhanced feature map is mapped to a graph node, and the initial feature vector of each node is the multidimensional feature vector of the corresponding location in the enhanced feature map; Based on the initial feature vector of the node, calculate the manifold geodesic distance in space that includes amplitude and direction differences; determine a local neighborhood set for each node according to the manifold geodesic distance, and construct an edge set between nodes based on the local neighborhood set.

2. The remote sensing image scene classification method according to claim 1, characterized in that, The feature enhancement includes edge and texture feature enhancement as well as key feature enhancement in the frequency domain; The enhanced feature map is obtained by adding the edge and texture feature enhancement map and the key feature enhancement map.

3. The remote sensing image scene classification method according to claim 2, characterized in that, The gradient histogram (HOG) operator is used to enhance the edge and texture features of the remote sensing image feature map.

4. The remote sensing image scene classification method according to claim 2, characterized in that, A dual activation mechanism is employed to enhance key features of the remote sensing image feature map. The dual activation mechanism includes a parallel gated activation function and a softmax function.

5. The remote sensing image scene classification method according to claim 1, characterized in that, The computational space includes manifold geodesic distances with differences in amplitude and direction, including: In the formula, This represents the manifold geodesic distance between node i and node j. and For balance coefficient, Let i be the multidimensional feature vector of node i. Let be the multidimensional feature vector of node j.

6. The remote sensing image scene classification method according to claim 1, characterized in that, Classifying remote sensing image scenes based on the aforementioned graph structure features includes: Multi-layer feature extraction is performed on remote sensing images, and graph structure features corresponding to each layer of feature maps are obtained respectively; The system integrates the structural features of each layer of the graph from shallow to deep based on the exponential moving average, and classifies the remote sensing image scene based on the integrated features.

7. The remote sensing image scene classification method according to claim 1, characterized in that, A focus loss method is used to train the scene classification process of remote sensing images based on surface-frequency domain information maps.

8. A remote sensing image scene classification system based on surface-frequency domain information maps, characterized in that, To implement the remote sensing image scene classification method based on surface-frequency domain information map as described in any one of claims 1-7, comprising: A cascaded feature extraction module is used to perform multi-layer feature extraction on remote sensing images; Each feature extraction module is connected to a feature optimization module. The feature optimization module includes a surface-frequency correlation mapping unit and a multilayer perceptron. The surface-frequency correlation mapping unit is used to enhance the feature map of the remote sensing image and convert the enhanced features into a high-dimensional manifold surface map structure. It also extracts features from the high-dimensional manifold surface map structure to obtain graph structure features. The multilayer perceptron is used to classify the remote sensing image scene based on the graph structure features.

9. The remote sensing image scene classification system according to claim 8, characterized in that, The multilayer perceptrons are connected in sequence to integrate the graph structure features of each layer from shallow to deep based on the exponential moving average.

Images