Rapid Extraction and Recognition Method and System for Regional Features Based on Remote Sensing Images

Abstract

The present invention discloses a method and system for rapid extraction and recognition of regional features based on remote sensing images, which relates to the technical field of remote sensing images. By pre-collecting a set of remote sensing images and a set of knowledge in the field of ground objects, a knowledge graph of ground objects is constructed based on the set of knowledge in the field of ground objects. Through the knowledge graph of ground objects, a preliminary feature combination of the set of remote sensing images is selected, and a lightweight classification model is trained based on the preliminary feature combination. A deep feature learning model integrating the knowledge graph of ground objects is constructed based on the set of remote sensing images and the set of knowledge in the field of ground objects, and the deep feature learning model is trained based on the output of the lightweight classification model. Based on the lightweight classification model and the deep feature learning model, the regional features of newly collected remote sensing images are extracted; the present invention realizes the rapid extraction of regional features of remote sensing images.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing images, and specifically to a method and system for quickly extracting and identifying regional features based on remote sensing images. Background Art

[0002] In the field of virtual reality (VR), the extraction of deep features from remote sensing images has extensive and crucial applications, providing core support for constructing highly realistic and immersive virtual scenes.

[0003] Taking the construction of a virtual city as an example, by extracting deep features from remote sensing images of urban areas, rich urban structure information can be obtained. Using high-resolution remote sensing images, features such as the height, shape, and distribution density of buildings, as well as the orientation, width, and traffic network layout of roads, can be extracted. These deep features can not only help accurately restore the appearance of the city but also provide a data basis for functions such as navigation and path planning in virtual scenes. For example, in virtual tourism applications, users can freely stroll in the virtual city with the help of these precise urban structure features and truly experience the spatial layout of the city.

[0004] In the process of traditional remote sensing image feature extraction, there are many deficiencies. On the one hand, when extracting features solely based on image data, the effective use of domain knowledge of ground objects in the target area is lacking, resulting in poor pertinence and accuracy of feature extraction.

[0005] Therefore, the present invention proposes a method and system for quickly extracting and identifying regional features based on remote sensing images. Summary of the Invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. For this purpose, the present invention proposes a method and system for quickly extracting and identifying regional features based on remote sensing images, realizing the rapid extraction of regional features of remote sensing images.

[0007] To achieve the above object, a method for quickly extracting and identifying regional features based on remote sensing images is proposed, including the following steps:

[0008] Step 1: Collect a set of remote sensing images and a set of domain knowledge of ground objects in advance;

[0009] Step 2: Construct a knowledge graph of ground objects based on the set of domain knowledge of ground objects;

[0010] Step 3: Select a preliminary feature combination of the set of remote sensing images through the knowledge graph of ground objects, and train a lightweight classification model based on the preliminary feature combination;

[0011] Step 4: Construct a deep feature learning model for fusing the ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set, and train the deep feature learning model based on the output of the lightweight classification model;

[0012] Step 5: Based on the lightweight classification model and the deep feature learning model, perform regional feature extraction on newly collected remote sensing images.

[0013] The remote sensing image set is collected and stored through the following means for training subsequent lightweight classification models and deep feature learning models:

[0014] Collect historical multispectral images and historical hyperspectral images within the target type area through remote sensing image acquisition technology to form a remote sensing image set.

[0015] The collection method of the ground object domain knowledge set is as follows:

[0016] Extract various ground object domain knowledge in the target type area through at least one of the existing literature, geographic information system, on-site investigation, and surveying and mapping records to form a ground object domain knowledge set.

[0017] Constructing the ground object knowledge graph based on the ground object domain knowledge set includes the following steps:

[0018] Step 21: According to the specific type of the target type area, determine the key entities and entity relationships included in the ground object knowledge graph;

[0019] Step 22: Use semantic web standards to define and represent key entities and their entity relationships. The semantic web standards include Resource Description Framework (RDF) and Web Ontology Language (OWL), etc.;

[0020] Step 23: According to the collected ground object domain knowledge set, extract instances of each key entity and entity relationship through a text recognition model or manual entry;

[0021] Step 24: Use semantic web technology to model the ground object knowledge graph and organize the extracted instances of key entities and entity relationships into a structured network.

[0022] Selecting the preliminary feature combination of the remote sensing image set through the ground object knowledge graph includes the following steps:

[0023] Step 31: Based on the topological structure of the ground object knowledge graph, execute a graph traversal algorithm to locate feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list;

[0024] Step 32: Based on the candidate feature list, use a feature ranking algorithm to rank the candidate features by importance to obtain a feature ranking list;

[0025] Step 33: Sort the feature list, and use a feature selection algorithm to screen the preliminary feature combinations.

[0026] The method for performing importance ranking is as follows:

[0027] Extract semantic information related to the candidate features from the geospatial knowledge graph, such as the description of the candidate features, the relationships between the candidate features and other entities, and the centrality of the candidate features in the geospatial knowledge graph. Combine the semantic information and use a feature ranking algorithm to perform importance ranking on the candidate features in the candidate feature list. The feature ranking algorithm includes algorithms based on PageRank, information gain, or chi-square test.

[0028] The method for screening the preliminary feature combinations using a feature selection algorithm is as follows:

[0029] Select any one of the feature selection algorithms of recursive feature elimination, sequential forward selection, or sequential backward selection, and combine it with any one of the classification models of the SVM algorithm or the RF algorithm. Use any one of the performance metrics of the accuracy, F1 value, or Kappa coefficient of the classification model as the evaluation criterion, and select the feature combination that maximizes the performance metric of the classification model from the sorted feature list as the preliminary feature combination.

[0030] The method for training the lightweight classification model based on the preliminary feature combination is as follows:

[0031] Take each historical multispectral image and each historical hyperspectral image in the remote sensing image set as an image sample respectively;

[0032] For each image sample, perform annotation of the land cover type respectively to generate a land cover type label;

[0033] For each feature in the preliminary feature combination, extract the corresponding feature from each image sample according to the feature extraction technology corresponding to each feature to form the preliminary feature vector of each image sample;

[0034] Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the preliminary feature vector and the land cover type label of each image sample.

[0035] The method for constructing a deep feature learning model of a fused geospatial knowledge graph based on the remote sensing image set and the geospatial domain knowledge set is as follows:

[0036] Construct an image feature extraction branch based on a set of remote sensing images, extract the regional object knowledge graph corresponding to each remote sensing image from the object knowledge graph, construct a knowledge graph feature fusion branch based on the regional object knowledge graph, use an attention fusion mechanism to fuse the image feature extraction branch and the knowledge graph feature fusion branch, and then extract the deep features of the remote sensing image through a knowledge-guided CNN feature extraction structure to form a deep feature learning model.

[0037] The method for training the deep feature learning model based on the output of the lightweight classification model is as follows:

[0038] Add a fully connected layer and an output layer after the knowledge-guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the object type of the remote sensing image;

[0039] Use the classification result of the lightweight classification model for each remote sensing image as the image type label;

[0040] Use each remote sensing image and the object knowledge graph of the object type corresponding to each remote sensing image as the input of the deep feature learning model;

[0041] Use the mean square error between the predicted value of the object type of the remote sensing image and the image type label as the loss function of the deep feature learning model;

[0042] Train the deep feature learning model through the set of remote sensing images and the classification results of the lightweight classification model for each remote sensing image.

[0043] The method for extracting regional features of remote sensing images based on the lightweight classification model and the deep feature learning model is as follows:

[0044] For newly collected remote sensing images, use an exploration box of the same size as the historical multispectral images and historical hyperspectral images of the target region type in the set of remote sensing images, and traverse and retrieve them in spatial order in the newly collected remote sensing images to obtain regional remote sensing images of the corresponding size of the exploration box in turn;

[0045] Extract the feature values of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, input the preliminary feature vector into the lightweight classification model, and obtain the predicted value of the classification result of the lightweight classification model for the regional remote sensing image;

[0046] If the predicted value of the output classification result is not the target type region, do not process it;

[0047] If the predicted value of the classification result output is the target type area, the regional remote sensing image and the regional object knowledge graph corresponding to the predicted value of the classification result are input into the deep feature learning model to obtain the deep features output by the deep feature learning model; the deep features are the regional features of the regional remote sensing image.

[0048] A regional feature rapid extraction and recognition system based on remote sensing images includes an input data collection module, a knowledge graph construction module, a classification model training module, a feature extraction model training module, and a feature extraction module; among them, each module is connected electrically.

[0049] The input data collection module pre-collects a remote sensing image set and a domain knowledge set of objects, sends the domain knowledge set of objects to the knowledge graph construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module.

[0050] The knowledge graph construction module constructs an object knowledge graph based on the domain knowledge set of objects and sends the object knowledge graph to the classification model training module and the feature extraction model training module.

[0051] The classification model training module selects a preliminary feature combination of the remote sensing image set through the object knowledge graph and trains a lightweight classification model based on the preliminary feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module.

[0052] The feature extraction model training module constructs a deep feature learning model integrating the object knowledge graph based on the remote sensing image set and the domain knowledge set of objects, and trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module.

[0053] The feature extraction module extracts the regional features of the newly collected remote sensing images based on the lightweight classification model and the deep feature learning model.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] First, collect the remote sensing image set and the ground object domain knowledge set. Then, construct a ground object knowledge graph to select the initial feature combination of the remote sensing images and train a lightweight classification model. Next, construct a deep feature learning model that integrates the ground object knowledge graph and train it using the output of the lightweight classification model. Finally, extract the regional features of the newly collected remote sensing images based on these two models. In terms of feature extraction efficiency, with the help of the ground object knowledge graph and the classification model, key features are quickly located and screened, greatly improving the speed of feature extraction and reducing the waste of computing resources. In terms of classification accuracy, the deep feature learning model that integrates the knowledge graph makes full use of domain knowledge, enhances the recognition ability of complex ground objects, effectively improves the classification accuracy, and realizes the rapid extraction of regional features of remote sensing images. Description of the Drawings

[0056] Figure 1 It is a flowchart of the method for rapid extraction and recognition of regional features based on remote sensing images in Embodiment 1 of the present invention;

[0057] Figure 2 It is an example diagram of the network structure of a ground object knowledge graph in Embodiment 1 of the present invention;

[0058] Figure 3 It is a schematic diagram of the structure of the deep feature learning model in Embodiment 1 of the present invention;

[0059] Figure 4 It is a diagram of the module connection relationship of the system for rapid extraction and recognition of regional features based on remote sensing images in Embodiment 2 of the present invention. Detailed Embodiments

[0060] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0061] Embodiment 1

[0062] As Figure 1 shown, the method for rapid extraction and recognition of regional features based on remote sensing images includes the following steps:

[0063] Step 1: Collect the remote sensing image set and the ground object domain knowledge set in advance;

[0064] Step 2: Construct a ground object knowledge graph based on the ground object domain knowledge set;

[0065] Step 3: Select the initial feature combination of the remote sensing image set through the ground object knowledge graph and train a lightweight classification model based on the initial feature combination;

[0066] Step 4: Construct a deep feature learning model for fusing the ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set, and train the deep feature learning model based on the output of the lightweight classification model;

[0067] Step 5: Based on the lightweight classification model and the deep feature learning model, extract the regional features of newly collected remote sensing images.

[0068] Among them, the remote sensing image set is collected through the following channels and stored systematically for training subsequent lightweight classification models and deep feature learning models:

[0069] Collect historical multispectral images and historical hyperspectral images within the target type area through remote sensing image acquisition technology to form a remote sensing image set.

[0070] As an example, in virtual reality technology, the target type area usually refers to a specific geographical area that needs to be detailedly modeled and simulated. For example, in an AR / VR game scene with a grassland as the background, it is necessary to pre-collect remote sensing images of the grassland in different seasons in the real world to extract the features of the grassland, so as to reconstruct the grassland in virtual reality devices.

[0071] In more virtual reality applications, the target type area may also include areas for building the foundation of virtual terrain, such as landforms like mountains, cities, lakes, or areas for simulating natural ecological environments, including natural ground object features such as forests, wetlands, grasslands, or simulations of urban or building-intensive areas, or historical sites or cultural landscape areas, for scene reconstruction for different virtual reality applications respectively, which will not be elaborated here.

[0072] And the remote sensing image acquisition technology includes satellite remote sensing, unmanned aerial vehicle (UAV) remote sensing, radar remote sensing, etc.

[0073] Specifically, satellite remote sensing refers to using the historical data repositories of some commercial satellites to download remote sensing images accumulated over the years in the target type area. These data are geometrically and radiometrically corrected and processed using a standardization process to ensure consistency and reliability;

[0074] UAV remote sensing refers to collecting time-series image data of the target type area through historical UAV aerial photography records and analyzing the regular aerial photography data of the UAV in a specific area, which is especially suitable for data collection in small areas or areas with high changes;

[0075] Radar remote sensing refers to using synthetic aperture radar data to obtain image information that is not affected by weather. Obviously, this image information is suitable for collection in cloudy or nighttime environments;

[0076] Therefore, the specific remote sensing image acquisition technology used is determined according to the scenes specifically required by the virtual reality technology.

[0077] The historical multispectral image is an image obtained through multiple discrete spectral bands, which usually include visible light such as red, green, and blue, and invisible light such as near-infrared and short-wave infrared. Each band provides specific information on the characteristics of ground objects. Combining the data of multiple bands can enhance the ability to identify and classify ground objects;

[0078] The historical hyperspectral image data is an image obtained through hundreds to thousands of continuous bands, which can capture detailed spectral feature information of each pixel.

[0079] Furthermore, the collection method of the set of ground object domain knowledge is as follows:

[0080] Extract various ground object domain knowledge in the target type area through at least one of the existing literature, geographic information system, field investigation, and mapping record methods, and form a set of ground object domain knowledge.

[0081] In the disclosed embodiments of the present application, the collection of the ground object domain knowledge includes:

[0082] Collect the ground object attribute content including spectral features and shape features in the target type area through literature and database retrieval methods;

[0083] Collect the spatial feature content including spatial distribution patterns and ground object adjacency relationships in the target type area through geographic information system analysis methods;

[0084] Collect the ground object relationship content including ground object category relationships and land use types in the target type area through field investigation and mapping record methods;

[0085] The ground object attribute content, spatial feature content, and ground object relationship content form a set of ground object domain knowledge.

[0086] Specifically, the spectral feature refers to the typical spectral reflection characteristics of different ground objects in the target type area, such as vegetation, water bodies, bare land, etc., such as indices like NDVI and EVI;

[0087] The shape feature refers to the geometric shape parameters of different ground objects in the target type area, such as the regularity of buildings and the boundary complexity of forests;

[0088] The spatial distribution pattern includes the distribution rules of different ground objects in the topographic map of the target type area, such as the distribution of green spaces in the city and the relative positions of water bodies and mountains;

[0089] The adjacency relationship of the features includes the spatial interaction between features in the target type area, such as the adjacent relationship between a river and a wetland;

[0090] The category relationship of the features includes the classification hierarchy relationship between different features in the target type area, such as the classification of forest land and vegetation, and buildings and roads;

[0091] The land use type includes the land use forms of different areas in the target type area, providing support for land use planning.

[0092] It should be noted that the above feature domain knowledge is only some exemplary knowledge provided by this application. In the actual application process, some other aspects of feature domain knowledge can also be added adaptively. Any relevant domain knowledge reflecting the target type area belongs to the feature domain knowledge.

[0093] It can be understood that by systematically collecting and integrating various feature domain knowledge in the target type area, a comprehensive knowledge set about the target type area is formed, and the feature domain knowledge set provides important reference and support for the analysis and application of remote sensing images, ensuring the accuracy and effectiveness of feature recognition and classification under different topographies and landforms.

[0094] Further, based on the feature domain knowledge set, constructing a feature knowledge graph includes the following steps:

[0095] Step 21: According to the specific type of the target type area, determine the key entities and entity relationships included in the feature knowledge graph; specifically, the key entities can be feature categories such as forests and wetlands, feature attributes such as spectral features and geometric shapes, and feature instances such as specific plant species; the entity relationships are the relationships between each key entity determined in advance, such as "belong to", "has feature", "adjacent", "change rule", etc., and these relationships are used to connect different entities;

[0096] Step 22: Use semantic web standards to define and represent key entities and their entity relationships, and the semantic web standards include the Resource Description Framework RDF and the Web Ontology Language OWL, etc.;

[0097] Step 23: According to the collected feature domain knowledge set, extract the instances of each key entity and entity relationship through a text recognition model or manual input;

[0098] Step 24: Use semantic web technology to model the feature knowledge graph and organize the extracted instances of key entities and entity relationships into a structured network.

[0099] The network structure of a specific feature knowledge graph is as Figure 2As shown in the figure, in this feature knowledge graph of ground objects, it includes the core entity of the forest. The "forest" entity is connected to the "high NDVI" feature through the "has feature" relationship. It is defined that the "forest" entity has the attribute of "complex boundary", which is described through the "has feature" relationship. The forest is distributed in the "northern mountainous area" region, which is connected through the "is located in" relationship. The "adjacent" relationship between the forest and the water body, as well as their ecological interactions (such as water source dependence), and the "forest" is connected to the "seasonal change" through the "change rule" relationship, etc. This feature knowledge graph of ground objects describes some basic features and some deep features of the forest. This information can provide additional supplementary information for the subsequent lightweight classification model and deep feature learning model, thereby reducing the training time. It should be understood that this knowledge graph is only a simple example for easy understanding. In the actual application process, the key entities and entity relationships included in each target type area are more complex, and the features that can be extracted are also more abundant. However, the construction method of the knowledge graph is universal and will not be elaborated here.

[0100] Further, the selection of the preliminary feature combination of the remote sensing image set through the feature knowledge graph of ground objects includes the following steps:

[0101] Step 31: Based on the topological structure of the feature knowledge graph of ground objects, execute the graph traversal algorithm to locate the feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list;

[0102] Specifically, the graph traversal algorithm includes the depth-first traversal algorithm and the breadth-first traversal algorithm. For example, starting from the target ground object category node, traverse along the relationship paths such as "has feature" and "adjacent", and set the maximum search depth to 3 layers. Then, through the relationship weight filtering, only retain the relationship paths with a confidence greater than the preset confidence threshold. The relationship weights are preset through historical data statistics or expert experience. The execution of the specific graph traversal algorithm can be retrieved using the SPARQL query language. The location of the feature attribute nodes refers to retrieving the "has feature" relationship paths directly associated with the target ground object category through the SPARQL query language, and extracting the basic feature attributes in the end nodes of the relationship paths, such as the key parameters of the reflectance curve such as the red edge position and absorption valley depth of typical ground objects, morphological features represented by the boundary fractal dimension, area perimeter ratio, etc., and the spatial distribution pattern of adjacent ground objects.

[0103] Step 32: Based on the candidate feature list, use the feature sorting algorithm to sort the candidate features by importance to obtain a feature sorted list;

[0104] In the embodiment of the present invention, the method of performing the importance sorting is as follows:

[0105] Extract semantic information related to candidate features from the geographic object knowledge graph, such as descriptions of candidate features, relationships between candidate features and other entities, and the centrality of candidate features in the geographic object knowledge graph. Combine the semantic information and use a feature ranking algorithm to rank the importance of candidate features in the candidate feature list. The feature ranking algorithm includes algorithms based on PageRank, information gain, or chi-square test.

[0106] Step 33: Use a feature selection algorithm to screen the preliminary feature combinations from the feature ranking list.

[0107] In an embodiment of the present invention, the method of using a feature selection algorithm to screen the preliminary feature combinations is as follows:

[0108] Select any one of the feature selection algorithms of recursive feature elimination, sequential forward selection, or sequential backward selection, combine it with any one of the classification models of the SVM algorithm or the RF algorithm, and use any one of the performance indicators of the accuracy, F1 value, or Kappa coefficient of the classification model as the evaluation criterion. Select the feature combination that maximizes the performance indicator of the classification model from the sorted feature ranking list as the preliminary feature combination.

[0109] For the processes of the above steps 31 to 33, a specific application example is as follows:

[0110] For a constructed geographic object knowledge graph of urban green spaces, which includes entity types of green spaces such as "park", "street greening", "roof green space", etc., and feature attribute entities such as "NDVI", "EVI", "vegetation coverage", "building density", "road density", "height", "area", "shape complexity", etc., and the relationships between them, such as "park" - "has feature" - "high NDVI", "street greening" - "adjacent to" - "road", "roof green space" - "located on" - "top of building", etc.;

[0111] Adopt the depth-first traversal algorithm, starting from the "green space" entity node, traverse along the "has feature" and "adjacent" relationship paths, set the maximum search depth to 2, and in terms of relationship weights, set the confidence of the "has feature" relationship to 0.8 and the confidence of the "adjacent" relationship to 0.6;

[0112] Through SPARQL query statements, obtain the features related to the "adjacent" relationship. The final obtained candidate feature list may include: NDVI, EVI, vegetation coverage, building density, road density, height, area, shape complexity, etc.;

[0113] Extract the semantic information of candidate features from the knowledge graph. For example, the description of "NDVI" is "Normalized Difference Vegetation Index, reflecting the vegetation growth status", which has a strong association with the entity "green space"; "building density" is associated with "roof green space" and "street greening", but the association strengths are different;

[0114] Then, use an algorithm based on information gain to calculate the information gain value of each candidate feature for distinguishing different green space types. For example, NDVI and vegetation coverage have relatively high information gain for distinguishing "park" from other green space types, while building density has relatively high information gain for distinguishing "roof green space" from "street greening". Assume the final sorting result is: NDVI > vegetation coverage > building density > road density > EVI > height > area > shape complexity;

[0115] Adopt the sequential forward selection algorithm (SFS), combined with the SVM classifier, to gradually select features from the sorted feature list. For example, first select the feature NDVI ranked first, train the SVM model and evaluate its performance; then sequentially add features ranked lower, such as adding vegetation coverage, train the SVM model again and evaluate the performance. If the performance improves, retain the feature, otherwise discard it. And so on until the model performance no longer improves significantly.

[0116] It should be noted that in the implementation process of steps 31 to 33, first, by using the ground object knowledge graph and the graph traversal algorithm, the feature attributes related to the target ground object can be quickly located, avoiding blind search, significantly improving the efficiency and pertinence of feature extraction. Then, by combining the feature sorting algorithm and the feature selection algorithm, the candidate features can be sorted and screened according to the characteristics of the data itself and the classification target, and the feature combination that contributes the most to the subsequent classification model can be selected, thus avoiding the subjectivity and limitations of manual feature selection and effectively reducing the feature dimension and computational redundancy.

[0117] Furthermore, the method of training the lightweight classification model based on the preliminary feature combination is as follows:

[0118] Take each historical multispectral image and each historical hyperspectral image in the remote sensing image set as an image sample respectively;

[0119] For each image sample, label the ground object type respectively to generate the ground object type label;

[0120] For each feature in the initial feature combination, according to the feature extraction technology corresponding to each feature, extract the corresponding feature from each image sample to form the initial feature vector of each image sample; for example, obtain the NDVI value of the image sample by calculating the ratio of the difference between the reflectance in the near-infrared band and the red band to the sum of the reflectances in each image sample, or estimate the vegetation coverage using methods such as the pixel dichotomy model or the vegetation index regression model;

[0121] Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the initial feature vector of each image sample and the ground object type label.

[0122] Preferably, the lightweight classification model is any one of the support vector machine or the random forest algorithm.

[0123] In the embodiment of the present invention, when the lightweight classification model is a support vector machine, the training process of the lightweight classification model includes:

[0124] Select a suitable kernel function and parameters, such as the radial basis function kernel and the penalty parameter, and use methods such as cross-validation or grid search to optimize the model parameters;

[0125] In another embodiment of the present invention, when the lightweight classification model is a random forest algorithm, the training process of the lightweight classification model includes:

[0126] Pre-determine parameters such as the number of trees and the maximum depth of each tree, and use methods such as cross-validation or grid search to optimize the parameters.

[0127] It should be noted that the above support vector machine or random forest algorithm is only some optional representative classification models of the lightweight classification model, and other classification models that can achieve the same function as the lightweight classification model are within the protection scope of the present invention.

[0128] Furthermore, the method for constructing the deep feature learning model of the fused ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set is as follows:

[0129] Construct an image feature extraction branch based on the remote sensing image set, extract the regional ground object knowledge graph corresponding to each remote sensing image from the ground object knowledge graph, construct a knowledge graph feature fusion branch based on the regional ground object knowledge graph, use the attention fusion mechanism to fuse the image feature extraction branch and the knowledge graph feature fusion branch, and then extract the deep features of the remote sensing image through the knowledge-guided CNN feature extraction structure to form the deep feature learning model.

[0130] Specifically, as Figure 3As shown in the figure, it is a schematic structural diagram of a deep feature learning model. The left branch (including the remote sensing image input node, the CNN feature extraction node, and the image depth feature node) is the image feature extraction branch, and this image feature extraction branch uses a convolutional neural network deep learning model to extract the image features of the remote sensing image; the right branch (including the knowledge graph input node, the GNN feature encoding node, and the entity vector representation node) is the knowledge graph feature fusion branch. This knowledge graph feature fusion branch takes the regional ground object knowledge graph corresponding to the ground object type of the remote sensing image as the input, and uses a graph neural network such as GCN or GAT for processing. By performing message passing and aggregation on the graph structure, it learns the vector representation of each entity node in the regional ground object knowledge graph. This vector representation is the graph feature, and the vector representation of the entity contains the unique semantic information of each entity in the knowledge graph; the regional ground object knowledge graph refers to a knowledge graph composed of only some entities and some relationships related to the ground object type corresponding to the remote sensing image. For a specific remote sensing image, only the prior knowledge about the ground object type corresponding to this remote sensing image needs to be provided. Therefore, providing the regional ground object knowledge graph can greatly reduce the input dimension of the model;

[0131] Then, the fusion of the image feature and the graph feature is realized through the attention fusion mechanism. In this attention fusion mechanism, first calculate the correlation (such as cosine similarity) between the image feature and the vectorized representation of the knowledge graph entity, then perform a normalization operation on the correlation to obtain the attention weight, use the attention weight to perform a weighted sum on the normalized vectorized representation of the entity, obtain the knowledge feature vector of the entity node after fusion, and finally splice the knowledge feature vector and the image feature through the feature splicing method to obtain the fusion feature, so as to dynamically adjust the weight of the entity vector representation and realize knowledge-guided feature learning;

[0132] Finally, a knowledge-guided CNN feature extraction structure is used to perform deep feature extraction on the fusion feature after the fusion of the image feature and the vectorized representation of the entity. The knowledge-guided CNN feature extraction structure can include multiple convolutional layers, pooling layers, activation functions, etc.

[0133] It can be understood that in the structure of this deep feature learning model, the entity vector representation encoded by the ground object knowledge graph through the graph neural network essentially transforms the domain expert knowledge (such as ground object spectral characteristics, spatial relationship rules) into high-dimensional semantic features, thus bringing prior knowledge to constrain the feature space, and this prior knowledge can make the knowledge-guided CNN feature extraction structure lightweight.

[0134] Specifically, traditional remote sensing image CNN models usually require 8 - 10 convolutional layers to gradually extract low - level to high - level features. In the knowledge - guided CNN feature extraction structure of the deep feature learning model of the present invention, it can be composed of two convolutional layers for processing the interaction between knowledge features and image features, one convolutional layer for extracting multi - scale context, and one convolutional layer for generating the final deep features. And the channel number setting can be adjusted from [64, 128, 256, 512] of the traditional model to [32, 64, 128]. Based on the supplementary information provided by the knowledge features, the channel redundancy of each layer is reduced, and the fully - connected layer is further completely removed, significantly reducing the scale of the remote sensing image CNN model.

[0135] Furthermore, the method for training the deep feature learning model based on the output of the lightweight classification model is as follows:

[0136] Add a fully - connected layer and an output layer after the knowledge - guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the ground object type of the remote sensing image;

[0137] Use the classification result of the lightweight classification model for each remote sensing image as the image type label;

[0138] Take each remote sensing image and the ground object knowledge graph of the ground object type corresponding to each remote sensing image as the input of the deep feature learning model;

[0139] Use the mean square error between the predicted value of the ground object type of the remote sensing image and the image type label as the loss function of the deep feature learning model;

[0140] Train the deep feature learning model through the remote sensing image set and the classification results of the lightweight classification model for each remote sensing image.

[0141] It can be understood that by using the remote sensing image set of each ground object type as a sample, the deep feature learning model can learn the deep features of each ground object type, so that when a new remote sensing image appears, the trained deep feature learning model can be directly used to extract deep features.

[0142] Furthermore, the method for extracting regional features of remote sensing images based on the lightweight classification model and the deep feature learning model is as follows:

[0143] For newly collected remote sensing images, use an exploration box of the same size as the historical multi - spectral images and historical hyperspectral images of the target region type in the remote sensing image set, and traverse and retrieve them in spatial order in the newly collected remote sensing images to sequentially obtain regional remote sensing images of the size corresponding to the exploration box;

[0144] Extract the eigenvalue of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, input the preliminary feature vector into the lightweight classification model, and obtain the predicted value of the classification result of the regional remote sensing image by the lightweight classification model;

[0145] If the predicted value of the output classification result is not the target type area, no processing is performed;

[0146] If the predicted value of the output classification result is the target type area, input the regional remote sensing image and the regional object knowledge graph corresponding to the predicted value of the classification result into the deep feature learning model to obtain the deep feature output by the deep feature learning model; the deep feature is the regional feature of the regional remote sensing image.

[0147] Embodiment 2

[0148] As Figure 4 shown, the regional feature rapid extraction and recognition system based on remote sensing images includes an input data collection module, a knowledge graph construction module, a classification model training module, a feature extraction model training module, and a feature extraction module; among them, each module is connected electrically;

[0149] The input data collection module pre-collects a remote sensing image set and a ground object domain knowledge set, sends the ground object domain knowledge set to the knowledge graph construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module;

[0150] The knowledge graph construction module constructs a ground object knowledge graph based on the ground object domain knowledge set, and sends the ground object knowledge graph to the classification model training module and the feature extraction model training module;

[0151] The classification model training module selects the preliminary feature combination of the remote sensing image set through the ground object knowledge graph, and trains the lightweight classification model based on the preliminary feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module;

[0152] The feature extraction model training module constructs a deep feature learning model integrating the ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set, and trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module;

[0153] The feature extraction module extracts the regional features of the newly collected remote sensing images based on the lightweight classification model and the deep feature learning model.

[0154] In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the corresponding technical solutions in the prior art in terms of implementation principles are not described in detail to avoid excessive elaboration.

[0155] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0156] The above preset parameters or preset thresholds are set by those skilled in the art according to the actual situation or obtained through a large amount of data simulation.

[0157] The above embodiments are only used to illustrate the technical method of the present invention and not to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for rapid extraction and recognition of regional features based on remote sensing images, characterized in that, It includes the following steps: Step 1: Pre-collect a set of remote sensing images and a set of knowledge in the field of ground objects; Step 2: Based on the set of knowledge in the field of ground objects, construct a ground object knowledge graph; Step 3: Select a preliminary feature combination of the set of remote sensing images through the ground object knowledge graph, and train a lightweight classification model based on the preliminary feature combination; Step 4: Construct a deep feature learning model integrating the ground object knowledge graph based on the set of remote sensing images and the set of knowledge in the field of ground objects, and train the deep feature learning model based on the output of the lightweight classification model; Step 5: Based on the lightweight classification model and the deep feature learning model, extract regional features of newly collected remote sensing images; The method for constructing a deep feature learning model integrating the ground object knowledge graph based on the set of remote sensing images and the set of knowledge in the field of ground objects is as follows: Construct an image feature extraction branch based on the set of remote sensing images, extract the regional ground object knowledge graph corresponding to each remote sensing image from the ground object knowledge graph, construct a knowledge graph feature fusion branch based on the regional ground object knowledge graph, use an attention fusion mechanism to fuse the image feature extraction branch and the knowledge graph feature fusion branch, and then extract the deep features of the remote sensing image through a knowledge-guided CNN feature extraction structure to form a deep feature learning model; The method for training the deep feature learning model based on the output of the lightweight classification model is as follows: Add a fully connected layer and an output layer after the knowledge-guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the ground object type of the remote sensing image; Use the classification result of the lightweight classification model for each remote sensing image as the image type label; Use each remote sensing image and the ground object knowledge graph of the ground object type corresponding to each remote sensing image as the input of the deep feature learning model; Use the mean square error between the predicted value of the ground object type of the remote sensing image and the image type label as the loss function of the deep feature learning model; Train the deep feature learning model through the set of remote sensing images and the classification results of the lightweight classification model for each remote sensing image.

2. The rapid extraction and recognition method for regional features based on remote sensing images according to claim 1, characterized in that The set of remote sensing images is collected and stored through the following channels for training the lightweight classification model and the deep feature learning model: Collect historical multi-spectral images and historical hyperspectral images in the target type area through remote sensing image acquisition technology to form a set of remote sensing images.

3. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 2, wherein The collection method of the set of knowledge in the field of ground objects is as follows: Extract various pieces of knowledge in the field of ground objects in the target type area through at least one of the existing literature, geographic information system, field investigation, and mapping record methods to form a set of knowledge in the field of ground objects.

4. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 3, characterized in that, Based on the set of knowledge in the field of ground objects, constructing a ground object knowledge graph includes the following steps: Step 21: Determine the key entities and entity relationships included in the ground object knowledge graph according to the specific type of the target type area; Step 22: Use semantic web standards to define and represent the key entities and their entity relationships; Step 23: According to the collected set of knowledge in the field of ground objects, extract instances of each key entity and entity relationship through a text recognition model or manual entry. Step 24: Use semantic web technology to model the ground object knowledge graph, and organize the extracted key entities and instance of entity relationships into a structured network.

5. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 4, characterized in that The selection of the preliminary feature combination of the remote sensing image set through the ground object knowledge graph includes the following steps: Step 31: Based on the topological structure of the ground object knowledge graph, execute the graph traversal algorithm to locate the feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list; Step 32: Based on the candidate feature list, use the feature ranking algorithm to rank the candidate features by importance to obtain a feature ranking list; Step 33: For the feature ranking list, use the feature selection algorithm to screen the preliminary feature combination.

6. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 5, characterized in that, The method of performing importance ranking is as follows: Extract the semantic information related to the candidate features from the ground object knowledge graph, and combine the semantic information to use the feature ranking algorithm to rank the candidate features in the candidate feature list by importance. The feature ranking algorithm includes the algorithm based on PageRank, the algorithm based on information gain, or the algorithm based on chi-square test.

7. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 6, characterized in that The method of using the feature selection algorithm to screen the preliminary feature combination is as follows: Select any one of the feature selection algorithms of recursive feature elimination, sequential forward selection, or sequential backward selection, and combine any one of the classification models of the SVM algorithm or the RF algorithm. Use any one of the performance indicators of the accuracy, F1 value, or Kappa coefficient of the classification model as the evaluation criterion, and select the feature combination that maximizes the performance indicator of the classification model from the sorted feature ranking list as the preliminary feature combination.

8. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 7, characterized in that, The method of training the lightweight classification model based on the preliminary feature combination is as follows: Take each historical multi-spectral image and each historical hyperspectral image in the remote sensing image set as an image sample respectively; For each image sample, perform the annotation of the ground object type respectively to generate the ground object type label; For each feature in the preliminary feature combination, extract the corresponding feature from each image sample according to the feature extraction technology corresponding to each feature to form the preliminary feature vector of each image sample; Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the preliminary feature vector and the ground object type label of each image sample.

9. The method for rapid extraction and recognition of regional features based on remote sensing images according to claim 8, wherein The method of extracting the regional features of the remote sensing image based on the lightweight classification model and the deep feature learning model is as follows: For the newly collected remote sensing image, use an exploration box of the same size as the historical multi-spectral image and historical hyperspectral image of the target region type in the remote sensing image set, and traverse and retrieve it in the newly collected remote sensing image in spatial order to obtain the regional remote sensing image corresponding to the size of the exploration box in turn; Extract the feature values of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, and input the preliminary feature vector into the lightweight classification model to obtain the predicted value of the classification result of the lightweight classification model for the regional remote sensing image; If the predicted value of the output classification result is not the target type region, no processing is performed; If the predicted value of the output classification result is the target type area, the regional remote sensing image and the regional ground object knowledge graph corresponding to the predicted value of the classification result are input into the deep feature learning model to obtain the deep features output by the deep feature learning model; the deep features are the regional features of the regional remote sensing image.

10. A rapid extraction and recognition system for regional features based on remote sensing images, which is used to implement the rapid extraction and recognition method for regional features based on remote sensing images described in any one of claims 1-9, characterized in that, It includes an input data collection module, a knowledge graph construction module, a classification model training module, a feature extraction model training module, and a feature extraction module; among them, each module is connected electrically. The input data collection module pre-collects a remote sensing image set and a ground object domain knowledge set, sends the ground object domain knowledge set to the knowledge graph construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module. The knowledge graph construction module constructs a ground object knowledge graph based on the ground object domain knowledge set, and sends the ground object knowledge graph to the classification model training module and the feature extraction model training module. The classification model training module selects the initial feature combination of the remote sensing image set through the ground object knowledge graph, and trains a lightweight classification model based on the initial feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module. The feature extraction model training module constructs a deep feature learning model integrating the ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set, and trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module. The feature extraction module extracts the regional features of the newly collected remote sensing images based on the lightweight classification model and the deep feature learning model.

Images