Regional feature rapid extraction and identification method and system based on remote sensing image

Abstract

The invention discloses a method and a system for quickly extracting and identifying regional features based on remote sensing images, and relates to the technical field of remote sensing image.A remote sensing image set and a ground feature domain knowledge set are collected in advance, and a ground feature knowledge graph is constructed based on the ground feature domain knowledge set; selecting a preliminary feature combination of the remote sensing image set through the ground object knowledge graph, training a lightweight classification model based on the preliminary feature combination, and constructing a deep feature learning model fusing the ground object knowledge graph based on the remote sensing image set and the ground object domain knowledge set; training a deep feature learning model based on the output of the lightweight classification model, and extracting the region features of the newly collected remote sensing image based on the lightweight classification model and the deep feature learning model; according to the invention, rapid extraction of remote sensing image region features is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing images, and in particular to a method and system for rapidly extracting and identifying regional features based on remote sensing images. Background Art

[0002] In the field of virtual reality (VR), deep feature extraction of remote sensing images has extensive and critical applications, providing core support for building highly realistic and immersive virtual scenes.

[0003] Taking virtual city construction as an example, by extracting deep features from remote sensing images of urban areas, we can obtain rich urban structure information. Using high-resolution remote sensing images, we can extract features such as the height, shape, and distribution density of buildings, as well as the direction, width, and transportation network layout of roads. These deep features can not only help accurately restore the appearance of the city, but also provide a data basis for functions such as navigation and path planning in virtual scenes. For example, in virtual tourism applications, users can use these accurate urban structure features to walk freely in the virtual city and truly experience the spatial layout of the city.

[0004] There are many deficiencies in the traditional remote sensing image feature extraction process. On the one hand, when feature extraction is performed based solely on image data, there is a lack of effective use of domain knowledge of the target area, resulting in poor pertinence and accuracy of feature extraction.

[0005] To this end, the present invention proposes a method and system for rapidly extracting and identifying regional features based on remote sensing images. Summary of the invention

[0006] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a method and system for rapidly extracting and identifying regional features based on remote sensing images, thereby realizing rapid extraction of regional features of remote sensing images.

[0007] To achieve the above purpose, a method for rapid extraction and recognition of regional features based on remote sensing images is proposed, which includes the following steps: Step 1: Collect remote sensing image collection and ground object domain knowledge collection in advance; Step 2: Construct a knowledge graph of land features based on the knowledge set of land features; Step 3: Select the preliminary feature combination of the remote sensing image set through the ground feature knowledge graph, and train the lightweight classification model based on the preliminary feature combination; Step 4: Build a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, and train the deep feature learning model based on the output of the lightweight classification model; Step 5: Based on the lightweight classification model and deep feature learning model, regional feature extraction of the newly collected remote sensing images is performed.

[0008] The remote sensing image set is collected and stored through the following channels for training subsequent lightweight classification models and deep feature learning models: Through remote sensing image acquisition technology, historical multispectral images and historical hyperspectral images in the target type area are collected to form a remote sensing image collection.

[0009] The collection method of the ground feature domain knowledge set is as follows: Through at least one of the existing literature, geographic information system, field investigation and surveying and mapping record methods, various ground feature domain knowledge in the target type area is extracted to form a ground feature domain knowledge set.

[0010] The method of constructing a knowledge graph of land objects based on a set of land object domain knowledge includes the following steps: Step 21: According to the specific type of the target type area, determine the key entities and entity relationships contained in the feature knowledge graph; Step 22: Use semantic web standards to define and represent key entities and their entity relationships, such as resource description framework RDF and web ontology language OWL; Step 23: Based on the collected domain knowledge set of land objects, extract instances of each key entity and entity relationship through a text recognition model or manual entry; Step 24: Use semantic web technology to model the feature knowledge graph and organize the extracted key entities and entity relationship instances into a structured network.

[0011] The method of selecting a preliminary feature combination of a remote sensing image set through a ground object knowledge graph comprises the following steps: Step 31: Based on the topological structure of the feature knowledge graph, execute the graph traversal algorithm to locate the feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list; Step 32: Based on the candidate feature list, use a feature ranking algorithm to rank the candidate features by importance to obtain a feature ranking list; Step 33: For the feature sorting list, use feature selection algorithm to screen the preliminary feature combination.

[0012] The method of ranking the importance is as follows: Semantic information related to the candidate features is extracted from the feature knowledge graph, such as the description of the candidate features, the relationship between the candidate features and other entities, and the centrality of the candidate features in the feature knowledge graph, etc., and the candidate features in the candidate feature list are ranked by importance using a feature ranking algorithm in combination with the semantic information. The feature ranking algorithm includes an algorithm based on PageRank, an algorithm based on information gain, or an algorithm based on a chi-square test.

[0013] The method of using the feature selection algorithm to screen the preliminary feature combination is: Select any feature selection algorithm among recursive feature elimination, sequential forward selection or sequential backward selection, combined with any classification model among SVM algorithm or RF algorithm, use any performance index among accuracy rate, F1 value or Kappa coefficient of classification model as evaluation criterion, select the feature combination that maximizes the performance index of classification model from the sorted feature ranking list as the preliminary feature combination.

[0014] The method of training a lightweight classification model based on a preliminary feature combination is: Each historical multispectral image and each historical hyperspectral image in the remote sensing image collection is respectively regarded as an image sample; For each image sample, mark the type of the object and generate a label for the object type; For each feature in the preliminary feature combination, the corresponding feature is extracted from each image sample according to the feature extraction technology corresponding to each feature to form a preliminary feature vector for each image sample; Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the preliminary feature vector and ground object type label of each image sample.

[0015] The method of constructing a deep feature learning model integrating a ground object knowledge graph based on a remote sensing image set and a ground object domain knowledge set is as follows: Based on the remote sensing image collection, an image feature extraction branch is constructed, and the regional feature knowledge graph corresponding to each remote sensing image is extracted from the feature knowledge graph. Based on the regional feature knowledge graph, a knowledge graph feature fusion branch is constructed. The image feature extraction branch and the knowledge graph feature fusion branch are fused using the attention fusion mechanism. The deep features of the remote sensing image are extracted through the knowledge-guided CNN feature extraction structure to form a deep feature learning model.

[0016] The method of training the deep feature learning model based on the output of the lightweight classification model is: A fully connected layer and an output layer are added after the knowledge-guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the ground object type of the remote sensing image; The classification result of each remote sensing image using the lightweight classification model is used as the image type label; Each remote sensing image and the feature knowledge graph of the feature type corresponding to each remote sensing image are used as the input of the deep feature learning model; The mean square error between the predicted value of the object type of the remote sensing image and the image type label is used as the loss function of the deep feature learning model; The deep feature learning model is trained through the remote sensing image collection and the classification results of each remote sensing image by the lightweight classification model.

[0017] The method of extracting regional features of remote sensing images based on the lightweight classification model and the deep feature learning model is as follows: For newly collected remote sensing images, an exploration frame of the same size as the historical multispectral images and historical hyperspectral images of the target area type in the remote sensing image collection is used to traverse and search in the newly collected remote sensing images in spatial order, and regional remote sensing images of the corresponding size of the exploration frame are obtained in turn; Extracting the feature value of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, inputting the preliminary feature vector into the lightweight classification model, and obtaining the predicted value of the classification result of the lightweight classification model for the regional remote sensing image; If the predicted value of the output classification result is not the target type area, no processing is performed; If the predicted value of the output classification result is the target type area, the regional remote sensing image and the regional land feature knowledge graph corresponding to the predicted value of the classification result are input into the deep feature learning model to obtain the deep features output by the deep feature learning model; the deep features are the regional features of the regional remote sensing image.

[0018] A regional feature rapid extraction and recognition system based on remote sensing images, comprising an input data collection module, a map construction module, a classification model training module, a feature extraction model training module and a feature extraction module; wherein each module is electrically connected; The input data collection module collects the remote sensing image set and the ground object domain knowledge set in advance, and sends the ground object domain knowledge set to the atlas construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module; A graph construction module, which constructs a knowledge graph of ground objects based on a knowledge set of ground objects, and sends the knowledge graph of ground objects to a classification model training module and a feature extraction model training module; The classification model training module selects a preliminary feature combination of the remote sensing image set through the ground object knowledge graph, trains a lightweight classification model based on the preliminary feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module; The feature extraction model training module builds a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module; The feature extraction module extracts regional features of newly collected remote sensing images based on lightweight classification models and deep feature learning models.

[0019] Compared with the prior art, the present invention has the following beneficial effects: First, we collect remote sensing image collections and domain knowledge collections of land objects, then build a knowledge graph of land objects, select preliminary feature combinations of remote sensing images and train lightweight classification models; then we build a deep feature learning model that integrates the knowledge graph of land objects, and train it using the output of the lightweight classification model; finally, we extract regional features from the newly collected remote sensing images based on these two models. In terms of feature extraction efficiency, with the help of the knowledge graph of land objects and the classification model, we can quickly locate and filter out key features, greatly improve the speed of feature extraction, and reduce the waste of computing resources. In terms of classification accuracy, the deep feature learning model that integrates the knowledge graph makes full use of domain knowledge, enhances the ability to identify complex land objects, effectively improves classification accuracy, and realizes the rapid extraction of regional features of remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a flow chart of a method for rapid extraction and identification of regional features based on remote sensing images in Example 1 of the present invention; Figure 2 This is an example diagram of the network structure of a geographical feature knowledge graph in Example 1 of the present invention; Figure 3 This is a schematic diagram of the structure of the deep feature learning model in Example 1 of the present invention; Figure 4 This is a module connection relationship diagram of the system for rapid extraction and recognition of regional features based on remote sensing images in Example 2 of the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Example 1

[0023] like Figure 1 As shown, the method for rapid extraction and recognition of regional features based on remote sensing images includes the following steps: Step 1: Collect remote sensing image collection and ground object domain knowledge collection in advance; Step 2: Construct a knowledge graph of land features based on the knowledge set of land features; Step 3: Select the preliminary feature combination of the remote sensing image set through the ground feature knowledge graph, and train the lightweight classification model based on the preliminary feature combination; Step 4: Build a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, and train the deep feature learning model based on the output of the lightweight classification model; Step 5: Based on the lightweight classification model and deep feature learning model, regional feature extraction of the newly collected remote sensing images is performed.

[0024] The remote sensing image set is collected and systematically stored through the following channels for training subsequent lightweight classification models and deep feature learning models: Through remote sensing image acquisition technology, historical multispectral images and historical hyperspectral images in the target type area are collected to form a remote sensing image collection.

[0025] As an example, in virtual reality technology, a target type area usually refers to a specific geographical area that needs to be modeled and simulated in detail. For example, in an AR / VR game scene with a grassland as the background, it is necessary to collect remote sensing images of the grassland in the real world at different seasons in advance to extract the characteristics of the grassland, so as to reconstruct the grassland in the virtual reality device.

[0026] In more virtual reality applications, the target type area may also include areas used to construct the basis of virtual terrain, such as mountains, cities, lakes and other landforms, or areas simulating natural ecological environments, including natural land features such as forests, wetlands, grasslands, or simulations of cities or densely built areas, or historical sites or cultural landscapes, so as to reconstruct scenes for different virtual reality applications, which will not be elaborated here.

[0027] The remote sensing image acquisition technology includes satellite remote sensing, UAV remote sensing, radar remote sensing, etc.

[0028] Specifically, satellite remote sensing refers to using the historical data repositories of some commercial satellites to download remote sensing images accumulated over the years for target type areas. These data have been geometrically and radiometrically corrected and processed using standardized processes to ensure consistency and reliability. Drone remote sensing refers to the collection of time-series image data of target type areas through historical drone aerial photography records, and the analysis of regular drone aerial photography data in specific areas, which is particularly suitable for data collection in small areas or highly variable areas; Radar remote sensing refers to the use of synthetic aperture radar data to obtain image information that is not affected by weather. Obviously, this image information is suitable for collection in cloudy or nighttime environments; Therefore, the specific remote sensing image acquisition technology used is determined according to the specific scenes that need to be collected by virtual reality technology.

[0029] The historical multispectral images are images acquired through multiple discrete spectral bands, which usually include visible light such as red, green, and blue, and invisible light such as near infrared and short-wave infrared. Each band provides specific information on the characteristics of land objects. Combining data from multiple bands can enhance the ability to identify and classify land objects. The historical hyperspectral image data are images acquired through hundreds to thousands of continuous bands, which can capture detailed spectral characteristic information of each pixel.

[0030] Furthermore, the collection method of the ground feature domain knowledge set is as follows: Through at least one of the existing literature, geographic information system, field investigation and surveying and mapping record methods, various ground feature domain knowledge in the target type area is extracted to form a ground feature domain knowledge set.

[0031] In the disclosed embodiments of the present application, the collection of the domain knowledge of ground objects includes: Through literature and database retrieval, collect the attributes of the objects in the target type area, including spectral characteristics and shape characteristics; Through geographic information system analysis, the spatial characteristics of the target type area, including spatial distribution patterns and adjacency relationships of land objects, are collected; Through field investigation and mapping records, collect the relationship content of land features in the target type area, including the relationship between land feature categories and land use types; The content of land feature attributes, spatial characteristics and land feature relationship constitute the land feature domain knowledge set.

[0032] Specifically, the spectral characteristics refer to typical spectral reflectance characteristics of different objects in the target type area, such as vegetation, water bodies, bare land, etc., such as NDVI, EVI and other indexes; The shape features refer to the geometric parameters of different objects in the target type area, such as the regularity of buildings, the complexity of forest boundaries, etc. The spatial distribution pattern includes the distribution patterns of different landforms in the target type area in the topographic map, such as the distribution of green space in the city, the relative positions of water bodies and mountains, etc.; The feature adjacency relationship includes the spatial interaction between features in the target type area, such as the adjacency relationship between rivers and wetlands; The feature category relationship includes the classification hierarchical relationship between different features in the target type area, such as the classification of forest land and vegetation, buildings and roads; The land use type includes the land use forms of different areas in the target type area, providing support for land use planning.

[0033] It should be noted that the above-mentioned domain knowledge of land features is only some exemplary knowledge provided in this application. In the actual application process, some other aspects of domain knowledge of land features can be adaptively added. Any relevant domain knowledge that reflects the target type area can belong to the domain knowledge of land features.

[0034] It can be understood that by systematically collecting and integrating various land object domain knowledge in the target type area, a comprehensive knowledge set about the target type area is formed. The land object domain knowledge set provides important reference and support for the analysis and application of remote sensing images, ensuring the accuracy and effectiveness of land object identification and classification under different terrain and landforms.

[0035] Furthermore, the construction of the ground feature knowledge graph based on the ground feature domain knowledge set includes the following steps: Step 21: According to the specific type of the target type area, determine the key entities and entity relationships contained in the feature knowledge graph; specifically, the key entities may be feature categories such as forests and wetlands, such as spectral features, characteristic attributes of geometric shapes, and feature instances such as specific plant species; the entity relationships are pre-determined relationships between key entities, such as "belonging to", "having characteristics", "adjacent", "changing rules", etc., and these relationships are used to connect different entities; Step 22: Use semantic web standards to define and represent key entities and their entity relationships, such as resource description framework RDF and web ontology language OWL; Step 23: Based on the collected domain knowledge set of land objects, extract instances of each key entity and entity relationship through a text recognition model or manual entry; Step 24: Use semantic web technology to model the feature knowledge graph and organize the extracted key entities and entity relationship instances into a structured network.

[0036] The network structure of a specific feature knowledge graph is as follows: Figure 2As shown in the figure, in this feature knowledge graph, the core entity of forest is included, and the "forest" entity is connected with the "high NDVI" feature through the "has feature" relationship, and the "forest" entity is defined to have the "complex boundary" attribute. The "has feature" relationship describes that forests are distributed in the "northern mountainous area" area, and the "adjacent" relationship between forests and water bodies, as well as their ecological interactions (such as water source dependence) are connected through the "located in" relationship. "Forest" and "seasonal changes" are connected through the "change law" relationship, etc.; this feature knowledge graph describes some basic characteristics and some deep-level characteristics of forests. This information can provide additional supplementary information for subsequent lightweight classification models and deep feature learning models, thereby reducing the training time; it should be understood that this knowledge graph is only a simple example that is easy to understand. In actual application, the key entities and entity relationships contained in each target type area are more complex, and the features that can be extracted are also richer, but the construction method of the knowledge graph is universal and will not be repeated here.

[0037] Furthermore, the selecting of a preliminary feature combination of a remote sensing image set through a ground object knowledge graph comprises the following steps: Step 31: Based on the topological structure of the feature knowledge graph, execute the graph traversal algorithm to locate the feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list; Specifically, the graph traversal algorithm includes a depth traversal algorithm and a breadth traversal algorithm. For example, starting from the target feature category node, traversing along the relationship paths such as "having characteristics" and "adjacent", and setting the maximum search depth to 3 layers, and then filtering by relationship weight, only retaining the relationship paths with confidence greater than a preset confidence threshold, and the relationship weight is obtained by historical data statistics or expert experience. The execution of the specific graph traversal algorithm can be retrieved using the SPARQL query language, and the positioning feature attribute node refers to retrieving the "having characteristics" relationship path directly associated with the target feature category through the SPARQL query language, and extracting the basic feature attributes in the terminal nodes of the relationship path, such as the red edge position of typical features, absorption valley depth and other key parameters of the reflectivity curve, such as the morphological characteristics represented by the boundary fractal dimension, area-to-perimeter ratio, etc., and the spatial distribution pattern of adjacent features.

[0038] Step 32: Based on the candidate feature list, use a feature ranking algorithm to rank the candidate features by importance to obtain a feature ranking list; In an embodiment of the present invention, the method of ranking the importance is: Semantic information related to the candidate features is extracted from the feature knowledge graph, such as the description of the candidate features, the relationship between the candidate features and other entities, and the centrality of the candidate features in the feature knowledge graph, etc., and the candidate features in the candidate feature list are ranked by importance using a feature ranking algorithm in combination with the semantic information. The feature ranking algorithm includes an algorithm based on PageRank, an algorithm based on information gain, or an algorithm based on a chi-square test.

[0039] Step 33: sort the feature list and use feature selection algorithm to select preliminary feature combinations; In an embodiment of the present invention, the method of using the feature selection algorithm to screen the preliminary feature combination is: Select any feature selection algorithm among recursive feature elimination, sequential forward selection or sequential backward selection, combined with any classification model among SVM algorithm or RF algorithm, use any performance index among accuracy rate, F1 value or Kappa coefficient of classification model as evaluation criterion, select the feature combination that maximizes the performance index of classification model from the sorted feature ranking list as the preliminary feature combination.

[0040] For the above-mentioned process from step 31 to step 33, a specific application example is as follows: For a completed urban green space feature knowledge graph, it contains green space type entities such as "park", "street greening", "rooftop green space", and feature attribute entities such as "NDVI", "EVI", "vegetation coverage", "building density", "road density", "height", "area", "shape complexity", and the relationships between them, such as "park" - "with features" - "high NDVI", "street greening" - "adjacent" - "road", "rooftop green space" - "located" - "on top of building", etc.; The depth-first traversal algorithm is used. Starting from the "green space" entity node, the paths of "having characteristics" and "adjacent" relationships are traversed. The maximum search depth is set to 2. In terms of relationship weights, the confidence of the "having characteristics" relationship is set to 0.8, and the confidence of the "adjacent" relationship is set to 0.6. Through SPARQL query statements, features related to the "adjacent" relationship are obtained. The final list of candidate features may include: NDVI, EVI, vegetation coverage, building density, road density, height, area, shape complexity, etc. Extract semantic information of candidate features from the knowledge graph. For example, the description of “NDVI” is “Normalized Difference Vegetation Index, reflecting the growth status of vegetation”, which is strongly associated with the “Green Space” entity; “Building Density” is associated with “Rooftop Green Space” and “Streetside Greenery”, but the association strength is different; Then, an algorithm based on information gain is used to calculate the information gain value of each candidate feature for distinguishing different types of green space. For example, NDVI and vegetation coverage have higher information gain for distinguishing "parks" from other types of green space, while building density has higher information gain for distinguishing "rooftop green space" from "streetside greening". Assuming that the final ranking result is: NDVI>vegetation coverage>building density>road density>EVI>height>area>shape complexity; The sequential forward selection algorithm (SFS) is used in combination with the SVM classifier to gradually select features from the sorted feature ranking list. For example, the first-ranked feature NDVI is selected first, the SVM model is trained and its performance is evaluated; then the features with lower rankings are added in sequence, such as vegetation coverage, the SVM model is trained again and the performance is evaluated. If the performance improves, the feature is retained, otherwise it is discarded. This process is repeated until the model performance is no longer significantly improved.

[0041] It should be noted that in the implementation process of steps 31 to 33, the feature knowledge graph and graph traversal algorithm are first used to quickly locate the feature attributes related to the target feature, avoiding blind search and significantly improving the efficiency and pertinence of feature extraction. Then, by combining the feature sorting algorithm and the feature selection algorithm, the candidate features can be sorted and screened according to the characteristics of the data itself and the classification goals, and the feature combination that contributes the most to the subsequent classification model can be selected, thereby avoiding the subjectivity and limitations of manual feature selection, and effectively reducing the feature dimension and reducing computational redundancy.

[0042] Furthermore, the method of training the lightweight classification model based on the preliminary feature combination is: Each historical multispectral image and each historical hyperspectral image in the remote sensing image collection is respectively regarded as an image sample; For each image sample, mark the type of the object and generate a label for the object type; For each feature in the preliminary feature combination, the corresponding feature is extracted from each image sample according to the feature extraction technology corresponding to each feature to form a preliminary feature vector for each image sample; for example, the NDVI value of the image sample is obtained by calculating the ratio of the difference between the reflectance of the near-infrared band and the red band in each image sample to the sum of the reflectances, or the vegetation coverage is estimated by using methods such as the pixel binary model or the vegetation index regression model; Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the preliminary feature vector and ground object type label of each image sample.

[0043] Preferably, the lightweight classification model is any one of a support vector machine or a random forest algorithm.

[0044] In an embodiment of the present invention, when the lightweight classification model is a support vector machine, the training process of the lightweight classification model includes: Select appropriate kernel functions and parameters, such as radial basis function kernels and penalty parameters, and optimize model parameters using methods such as cross-validation or grid search; In another embodiment of the present invention, when the lightweight classification model is a random forest algorithm, the training process of the lightweight classification model includes: Parameters such as the number of trees and the maximum depth of each tree are determined in advance and optimized using methods such as cross-validation or grid search.

[0045] It should be noted that the above-mentioned support vector machine or random forest algorithm is only some optional representative classification models of the lightweight classification model, and other classification models that can achieve the same function as the lightweight classification model are within the protection scope of the present invention.

[0046] Furthermore, the method of constructing a deep feature learning model integrating a ground object knowledge graph based on a remote sensing image set and a ground object domain knowledge set is as follows: Based on the remote sensing image collection, an image feature extraction branch is constructed, and the regional feature knowledge graph corresponding to each remote sensing image is extracted from the feature knowledge graph. Based on the regional feature knowledge graph, a knowledge graph feature fusion branch is constructed. The image feature extraction branch and the knowledge graph feature fusion branch are fused using the attention fusion mechanism. The deep features of the remote sensing image are extracted through the knowledge-guided CNN feature extraction structure to form a deep feature learning model.

[0047] Specifically, if Figure 3As shown in the figure, the structural diagram of the deep feature learning model, the left branch (including remote sensing image input node, CNN feature extraction node and image deep feature node) is the image feature extraction branch, which uses a convolutional neural network deep learning model to extract image features of remote sensing images; the right branch (including knowledge graph input node, GNN feature encoding node and entity vector representation node) is the knowledge graph feature fusion branch, which uses the regional feature knowledge graph corresponding to the feature type corresponding to the remote sensing image as input, using, for example, GCN or GA The graph neural network of T is processed, and the vector representation of the node of each entity in the regional feature knowledge graph is learned by message passing and aggregation on the graph structure. The vector represents the graph feature, and the vector representation of the entity contains the semantic information unique to each entity in the knowledge graph. The regional feature knowledge graph refers to a knowledge graph consisting of only some entities and some relationships related to the feature type corresponding to the remote sensing image. For a specific remote sensing image, only the prior knowledge about the feature type corresponding to the remote sensing image needs to be provided. Therefore, providing the regional feature knowledge graph can greatly reduce the input dimension of the model. Then, the fusion of image features and graph features is realized through the attention fusion mechanism. In the attention fusion mechanism, the correlation between the image features and the vectorized representation of the knowledge graph entity is first calculated (for example, cosine similarity), and then the correlation is normalized to obtain the attention weight. The normalized vectorized representation of the entity is weighted summed using the attention weight to obtain the knowledge feature vector of the entity node after fusion. Finally, the knowledge feature vector and the image feature are spliced ​​by feature splicing to obtain the fusion feature, thereby dynamically adjusting the weight of the entity vector representation and realizing knowledge-guided feature learning. Finally, a knowledge-guided CNN feature extraction structure is used to perform deep feature extraction on the fused features obtained by fusing the image features and the vectorized representation of the entity. The knowledge-guided CNN feature extraction structure may include multiple convolutional layers, pooling layers, activation functions, and the like.

[0048] It can be understood that in the structure of this deep feature learning model, the knowledge graph of land objects is represented by entity vectors encoded through the graph neural network, which essentially converts domain expert knowledge (such as spectral characteristics of land objects and spatial relationship rules) into high-dimensional semantic features, thereby bringing about prior knowledge constraints on the feature space. This prior knowledge can make the knowledge-guided CNN feature extraction structure lightweight.

[0049] Specifically, the traditional remote sensing image CNN model usually requires 8-10 convolutional layers to gradually extract low-level to high-level features, while the knowledge-guided CNN feature extraction structure of the deep feature learning model of the present invention can be composed of two convolutional layers for processing the interaction between knowledge features and image features, one convolutional layer for extracting multi-scale contexts, and one convolutional layer for generating the final deep features; and the number of channels can be set from [64,128,256,512] of the traditional model to [32,64,128]. Based on the supplementary information provided by the knowledge features, the redundancy of the channels at each layer is reduced, and the fully connected layer is further completely removed, which greatly reduces the scale of the remote sensing image CNN model.

[0050] Furthermore, the method of training the deep feature learning model based on the output of the lightweight classification model is: A fully connected layer and an output layer are added after the knowledge-guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the ground object type of the remote sensing image; The classification result of each remote sensing image using the lightweight classification model is used as the image type label; Each remote sensing image and the feature knowledge graph of the feature type corresponding to each remote sensing image are used as the input of the deep feature learning model; The mean square error between the predicted value of the object type of the remote sensing image and the image type label is used as the loss function of the deep feature learning model; The deep feature learning model is trained through the remote sensing image collection and the classification results of each remote sensing image by the lightweight classification model.

[0051] It can be understood that by using a set of remote sensing images of each type of land object as samples, the deep feature learning model can learn the deep features of each type of land object, so that when new remote sensing images appear, the trained deep feature learning model can be directly used to extract deep features.

[0052] Furthermore, the method of extracting regional features of remote sensing images based on the lightweight classification model and the deep feature learning model is as follows: For newly collected remote sensing images, an exploration frame of the same size as the historical multispectral images and historical hyperspectral images of the target area type in the remote sensing image collection is used to traverse and search in the newly collected remote sensing images in spatial order, and regional remote sensing images of the corresponding size of the exploration frame are obtained in turn; Extracting the feature value of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, inputting the preliminary feature vector into the lightweight classification model, and obtaining the predicted value of the classification result of the lightweight classification model for the regional remote sensing image; If the predicted value of the output classification result is not the target type area, no processing is performed; If the predicted value of the output classification result is the target type area, the regional remote sensing image and the regional land feature knowledge graph corresponding to the predicted value of the classification result are input into the deep feature learning model to obtain the deep features output by the deep feature learning model; the deep features are the regional features of the regional remote sensing image.

[0053] Example 2

[0054] like Figure 4 As shown, the regional feature rapid extraction and recognition system based on remote sensing images includes an input data collection module, a map construction module, a classification model training module, a feature extraction model training module and a feature extraction module; wherein each module is electrically connected; The input data collection module collects the remote sensing image set and the ground object domain knowledge set in advance, and sends the ground object domain knowledge set to the atlas construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module; A graph construction module, which constructs a knowledge graph of ground objects based on a knowledge set of ground objects, and sends the knowledge graph of ground objects to a classification model training module and a feature extraction model training module; The classification model training module selects a preliminary feature combination of the remote sensing image set through the ground object knowledge graph, trains a lightweight classification model based on the preliminary feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module; The feature extraction model training module builds a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module; The feature extraction module extracts regional features of newly collected remote sensing images based on lightweight classification models and deep feature learning models.

[0055] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0056] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0057] The above preset parameters or preset thresholds are all set by technicians in this field according to actual conditions or obtained through large-scale data simulation.

[0058] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A method for rapid extraction and recognition of regional features based on remote sensing images, characterized in that: The following steps are involved: Step 1: Collect remote sensing image collection and ground object domain knowledge collection in advance; Step 2: Construct a knowledge graph of land features based on the knowledge set of land features; Step 3: Select the preliminary feature combination of the remote sensing image set through the ground feature knowledge graph, and train the lightweight classification model based on the preliminary feature combination; Step 4: Build a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, and train the deep feature learning model based on the output of the lightweight classification model; Step 5: Based on the lightweight classification model and deep feature learning model, regional feature extraction of the newly collected remote sensing images is performed.

2. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 1 is characterized in that: The remote sensing image collection is collected and stored through the following channels for training lightweight classification models and deep feature learning models: Through remote sensing image acquisition technology, historical multispectral images and historical hyperspectral images in the target type area are collected to form a remote sensing image collection.

3. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 2 is characterized in that: The collection method of the ground feature domain knowledge set is as follows: Through at least one of the existing literature, geographic information system, field investigation and surveying and mapping record methods, various ground feature domain knowledge in the target type area is extracted to form a ground feature domain knowledge set.

4. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 3 is characterized in that: The method of constructing a knowledge graph of land objects based on a set of land object domain knowledge includes the following steps: Step 21: According to the specific type of the target type area, determine the key entities and entity relationships contained in the feature knowledge graph; Step 22: Use semantic web standards to define and represent key entities and their entity relationships; Step 23: Based on the collected domain knowledge set of land objects, extract instances of each key entity and entity relationship through a text recognition model or manual entry; Step 24: Use semantic web technology to model the feature knowledge graph and organize the extracted key entities and entity relationship instances into a structured network.

5. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 4 is characterized in that: The method of selecting a preliminary feature combination of a remote sensing image set through a ground object knowledge graph comprises the following steps: Step 31: Based on the topological structure of the feature knowledge graph, execute the graph traversal algorithm to locate the feature attribute nodes, and use the located feature attribute nodes as candidate features to form a candidate feature list; Step 32: Based on the candidate feature list, use a feature ranking algorithm to rank the candidate features by importance to obtain a feature ranking list; Step 33: For the feature sorting list, use feature selection algorithm to screen the preliminary feature combination.

6. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 5 is characterized in that: The method of ranking the importance is as follows: Semantic information related to the candidate features is extracted from the feature knowledge graph, and the candidate features in the candidate feature list are ranked by importance using a feature ranking algorithm in combination with the semantic information. The feature ranking algorithm includes an algorithm based on PageRank, an algorithm based on information gain, or an algorithm based on a chi-square test.

7. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 6 is characterized in that: The method of using the feature selection algorithm to screen the preliminary feature combination is: Select any feature selection algorithm among recursive feature elimination, sequential forward selection or sequential backward selection, combined with any classification model among SVM algorithm or RF algorithm, use any performance index among accuracy rate, F1 value or Kappa coefficient of classification model as evaluation criterion, select the feature combination that maximizes the performance index of classification model from the sorted feature ranking list as the preliminary feature combination.

8. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 7 is characterized in that: The method of training a lightweight classification model based on a preliminary feature combination is: Each historical multispectral image and each historical hyperspectral image in the remote sensing image collection is respectively regarded as an image sample; For each image sample, mark the type of the object and generate a label for the object type; For each feature in the preliminary feature combination, the corresponding feature is extracted from each image sample according to the feature extraction technology corresponding to each feature to form a preliminary feature vector for each image sample; Select any lightweight classification algorithm as the lightweight classification model, and train the lightweight classification model based on the preliminary feature vector and ground object type label of each image sample.

9. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 8, characterized in that: The method of constructing a deep feature learning model integrating a ground object knowledge graph based on a remote sensing image set and a ground object domain knowledge set is as follows: Based on the remote sensing image collection, an image feature extraction branch is constructed, and the regional feature knowledge graph corresponding to each remote sensing image is extracted from the feature knowledge graph. Based on the regional feature knowledge graph, a knowledge graph feature fusion branch is constructed. The image feature extraction branch and the knowledge graph feature fusion branch are fused using the attention fusion mechanism. The deep features of the remote sensing image are extracted through the knowledge-guided CNN feature extraction structure to form a deep feature learning model.

10. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 9, characterized in that: The method of training the deep feature learning model based on the output of the lightweight classification model is: A fully connected layer and an output layer are added after the knowledge-guided CNN feature extraction structure in the deep feature learning model, where the output of the output layer is the predicted value of the ground object type of the remote sensing image; The classification result of each remote sensing image using the lightweight classification model is used as the image type label; Each remote sensing image and the feature knowledge graph of the feature type corresponding to each remote sensing image are used as the input of the deep feature learning model; The mean square error between the predicted value of the object type of the remote sensing image and the image type label is used as the loss function of the deep feature learning model; The deep feature learning model is trained through the remote sensing image collection and the classification results of each remote sensing image by the lightweight classification model.

11. The method for rapid extraction and identification of regional features based on remote sensing images according to claim 10, characterized in that: The method of extracting regional features of remote sensing images based on the lightweight classification model and the deep feature learning model is as follows: For newly collected remote sensing images, an exploration frame of the same size as the historical multispectral images and historical hyperspectral images of the target area type in the remote sensing image collection is used to traverse and search in the newly collected remote sensing images in spatial order, and regional remote sensing images of the corresponding size of the exploration frame are obtained in turn; Extracting the feature value of the preliminary feature combination of each regional remote sensing image to form a preliminary feature vector, inputting the preliminary feature vector into the lightweight classification model, and obtaining the predicted value of the classification result of the lightweight classification model for the regional remote sensing image; If the predicted value of the output classification result is not the target type area, no processing is performed; If the predicted value of the output classification result is the target type area, the regional remote sensing image and the regional land feature knowledge graph corresponding to the predicted value of the classification result are input into the deep feature learning model to obtain the deep features output by the deep feature learning model; the deep features are the regional features of the regional remote sensing image.

12. A system for rapid extraction and identification of regional features based on remote sensing images, which is used to implement the method for rapid extraction and identification of regional features based on remote sensing images as described in any one of claims 1 to 11, characterized in that: It includes an input data collection module, a graph construction module, a classification model training module, a feature extraction model training module and a feature extraction module; wherein each module is electrically connected; The input data collection module collects the remote sensing image set and the ground object domain knowledge set in advance, and sends the ground object domain knowledge set to the atlas construction module, and sends the remote sensing image set to the classification model training module and the feature extraction model training module; A graph construction module, which constructs a knowledge graph of ground objects based on a knowledge set of ground objects, and sends the knowledge graph of ground objects to a classification model training module and a feature extraction model training module; The classification model training module selects a preliminary feature combination of the remote sensing image set through the ground object knowledge graph, trains a lightweight classification model based on the preliminary feature combination, and sends the lightweight classification model to the feature extraction model training module and the feature extraction module; The feature extraction model training module builds a deep feature learning model that integrates the ground object knowledge graph based on the remote sensing image collection and the ground object domain knowledge collection, trains the deep feature learning model based on the output of the lightweight classification model, and sends the deep feature learning model to the feature extraction module; The feature extraction module extracts regional features of newly collected remote sensing images based on lightweight classification models and deep feature learning models.

Images