Geographic information model construction method and computer program product based on aerial data intelligent segmentation and recognition

Abstract

The application discloses a kind of based on aerial data intelligent segmentation identification geographic information model construction method and computer program product. Including: first, SAM model is used to the segmentation of aerial image, and the segmentation image of key element is obtained with marking. Then, the key element in segmentation image is artificially classified, and data set is constructed according to artificial classification result. After that, KAN network of fusion knowledge graph is trained through data set. Then, the aerial image of the region to be built is accurately segmented using the SAM model, and the trained KAN network is used to accurately identify the key element category in the segmentation image by fusing the relevant knowledge graph. Finally, the geographic information model of the region to be built is constructed using the identification result of the key element. SAM model can accurately segment the key element through sparse and dense hints, and the missegmentation of SAM model can be corrected by using the graph relationship in the knowledge graph through KAN network, thereby improving the accuracy of key element classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource planning methods, and particularly relates to a geographic information model construction method based on aerial data intelligent segmentation and recognition and a computer program product. BACKGROUND

[0002] Traditional oil and gas facility site selection methods generally rely on manual annotation or traditional image segmentation algorithms (such as threshold segmentation and edge detection), and have high error rates in identifying complex terrains. In addition, the boundaries of ground objects in high-resolution images collected by unmanned aerial vehicles are blurred, and traditional image segmentation algorithms use fixed-size image segmentation strategies, which cannot adaptively adjust input parameters when facing different precision requirements, such as 1-meter fine segmentation and 5-meter rapid analysis. This rigid block mode easily causes block size to be too small in high-precision scenarios, resulting in calculation redundancy, and block size to be too large in low-precision scenarios, resulting in blurred details, which limits the engineering applicability of the method. Moreover, the conventional convolutional neural network used for element classification only relies on image features for training, lacks embedding of domain knowledge, and thus has low classification result accuracy. The updating of elements in dynamic scenarios (such as seasonal rivers and temporary buildings) relies on manual intervention and cannot be optimized adaptively through data-driven methods. SUMMARY

[0003] In view of the deficiencies in the prior art, the present application provides a geographic information model construction method based on aerial data intelligent segmentation and recognition and a computer program product, which can improve the accuracy of element classification. The specific technical solutions are as follows:

[0004] In a first aspect, a geographic information model construction method based on aerial data intelligent segmentation and recognition is provided. In a first implementable manner of the first aspect, the method comprises the following steps:

[0005] segmenting the ground features of each aerial image obtained by using a SAM model;

[0006] manually classifying the key elements obtained by segmentation, and constructing a data set according to the classification results;

[0007] training a KAN network based on a knowledge graph through the data set;

[0008] segmenting the aerial images of the to-be-built area by using the SAM model, and intelligently recognizing the key elements obtained by segmentation through the trained KAN network;

[0009] constructing a geographic information model of the to-be-built area according to the recognition results of the key elements.

[0010] In a second implementable manner of the first aspect, the data set is constructed according to the classification results, which comprises the following steps:

[0011] According to the artificial classification result, a multi-level classification data board is constructed, and feature information and mutual relationships of geographic elements are extracted from the multi-level classification data board to construct a knowledge graph;

[0012] Element features and mutual relationships of geographic elements in the knowledge graph are converted into trainable feature vectors, and the data set is constructed by fusing the artificial classification result.

[0013] In a third implementation manner of the first aspect, the KAN network is used to intelligently identify the key elements, and the method comprises the following steps of:

[0014] The image features corresponding to the key elements are extracted;

[0015] The image features are matched with the feature vectors of the geographic elements in the knowledge graph to determine the element categories corresponding to the key elements.

[0016] In a fourth implementation manner of the first aspect, the matching of the image features with the feature vectors of the geographic elements in the knowledge graph comprises the following steps of:

[0017] The similarity between the image features and the element features of the feature vectors is calculated;

[0018] Based on the similarity between the image features and the element features, the mutual relationships between the geographic elements in the feature vectors are combined to infer the element categories corresponding to the key elements.

[0019] In a fifth implementation manner of the first aspect, the KAN network is used to intelligently identify the key elements, and the method comprises the following steps of:

[0020] The confidence of the KAN network identification result is measured by the probability distribution entropy.

[0021] In a sixth implementation manner of the first aspect, a geographic information model of the to-be-built area is constructed according to the identification result of the key elements, and the method comprises the following steps of:

[0022] According to the identification result of the key elements, an ArcGIS Pro software is used to construct a three-dimensional element scene of the to-be-built area.

[0023] In a seventh implementation manner of the first aspect, the geographic information model of the to-be-built area is constructed according to the identification result of the key elements, and the method comprises the following steps of:

[0024] Construct a high-precision digital baseboard based on the identification result of the key elements.

[0025] In a sixth implementation manner of the first aspect, according to the identification result of the key elements, a geographic information model of the to-be-built area is constructed, including:

[0026] According to the identification result of the key elements in the to-be-built area and the three-dimensional element scene, a CAD file corresponding to each key element is generated, and the CAD file includes all attribute information of the key element.

[0027] In a second aspect, a computer program product is provided, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the geographic information model construction method in any one of the first to eighth implementation manners of the first aspect are implemented.

[0028] Beneficial effects: By using the geographic information model construction method and the computer program product based on aerial data intelligent segmentation and identification provided by the present application, the SAM model can accurately segment the key elements in the aerial image of the complex scene through sparse and dense prompt two ways, and provide a basis for the subsequent KAN network to identify the key element categories in the aerial image. Through the KAN network, the key elements segmented by the SAM model can be matched with the knowledge graph in the related field, and the mis-segmentation of the SAM model can be corrected through relationship reasoning, so as to improve the accuracy of key element classification. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the specific embodiments of the present application, the drawings needed in the specific embodiments will be briefly introduced below. In all the drawings, the elements or parts are not necessarily drawn according to the actual proportions.

[0030] Figure 1 The flowchart of the geographic information model construction method based on aerial data intelligent segmentation and identification provided by an embodiment of the present application is shown in the figure.

[0031] Figure 2 The flowchart of the SAM model segmenting the ground features provided by an embodiment of the present application is shown in the figure.

[0032] Figure 3 The segmented image obtained by the SAM model provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0033] The embodiments of the technical solutions of the present application will be described in detail below with reference to the drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present application, and therefore only serve as examples, but cannot limit the protection scope of the present application.

[0034] AsFigure 1 A flowchart of a geographic information model construction method based on aerial data intelligent segmentation and recognition is shown. The construction method includes:

[0035] Step 1: Using the SAM model to segment the ground features of each aerial image obtained;

[0036] Step 2: Artificially classifying the key elements obtained by segmentation, and constructing a data set according to the classification results;

[0037] Step 3: Training the KAN network based on the knowledge graph through the data set;

[0038] Step 4: Using the SAM model to segment the aerial images of the region to be built, and intelligently recognizing the key elements obtained by segmentation through the trained KAN network;

[0039] Step 5: Constructing the geographic information model of the region to be built according to the recognition results of the key elements.

[0040] Specifically, first, the aerial images of the target region can be collected by a drone, and the trained SAM model can be used to segment the ground features of each aerial image, thereby obtaining a segmented image with the key element region range labeled. Then, all key elements labeled in the segmented image can be artificially classified by referring to the pre-constructed key element feature classification table, and the corresponding classification labels can be labeled, generating a vector map with classification labels, and constructing a data set based on the vector map. Then, the KAN network fused with the knowledge graph can be trained through each sample in the data set. Then, the aerial images of the region to be built can be collected by a drone, and the SAM model can be used to accurately segment the aerial images of the region to be built, obtaining the corresponding segmented image of the region to be built, and then using the trained KAN network to accurately identify the element categories of all key elements in the segmented image by fusing the knowledge graph in the field.

[0041] In this embodiment, as Figure 2 shown, the SAM model includes an image encoder, a prompt encoder, a mask encoder, and a transformer decoder. Among them, the image encoder can use the ViT model pre-trained by the mask autoencoder (MAE) to process high-resolution aerial images to provide a basis for the subsequent prompt encoder.

[0042] The prompt encoder designs two prompt modes, sparse (such as points, boxes, and text) and dense (mask). Points and boxes are represented by position encoding and added to the learnable embedding of each prompt. Text is processed by using the ready-made text encoder from the contrastive language-image pre-training model (CLIP). Dense prompts can be embedded by convolution and added to the image embedding. The mask encoder can effectively map the image embedding, prompt embedding, and output token to a mask, and modify the transformer decoder module to enable bidirectional cross-attention calculation of the image embedding and the prompt embedding.

[0043] In the data processing related to oil well site selection and pipeline site selection, specific features in the image can be used as prompt information, such as geological features and surface vegetation distribution features, so that the SAM model can segment houses, roads, rivers, ponds, and vegetation, as shown in FIG. 8, to prepare for subsequent classification, identification, and site selection. Figure 3

[0044] Based on the "encoding-prompt-decoding" architecture, multi-modal prompts (points, boxes, and text) and mask auto-encoding technology are fused to realize pixel-level surface feature segmentation with a segmentation accuracy of ≥95%. Moreover, by embedding oil and gas facility-specific labels (such as oil wells and pipelines), the SAM model can accurately extract the boundaries of target regions (such as vegetation, roads, and water bodies) in complex terrain and support the rapid update of dynamic scenes (such as temporary buildings).

[0045] A mapping table of "target accuracy" and "optimal block size" is established through training data, and the table is dynamically called for blocking. The SAM model is trained under different block sizes, and the segmentation accuracy Dice coefficient is recorded. After selecting the target accuracy, the SAM model can automatically match the corresponding block size and cut the image. After inputting the regional image and target accuracy, the SAM model can use the U-Net model to identify high-frequency regions (such as feature boundaries) in the image, and then block them accordingly. Subsequently, a sliding window blocking is used, and an overlapping area (10%-20%) is set to avoid edge information loss, and then the results are merged through weighted fusion.

[0046] Through experiments, a mapping relationship model of block size and target accuracy is established, such as 1-meter accuracy corresponding to 60x80-meter block. Based on the accuracy requirement input by the user, the model automatically calculates the optimal block size, automatically cuts the large-scale image into standard sub-blocks, and then performs pixel-level segmentation on the standard sub-blocks. Finally, the edge fusion algorithm is used to splice the global result, realizing a flexible double-layer segmentation architecture from coarse granularity to fine granularity, balancing efficiency and accuracy. This method maintains a segmentation accuracy of more than 95% while improving the resolution to 1 meter.

[0047] In this embodiment, optionally, a data set is constructed according to the classification result, including:​

[0048] According to the artificial classification result, a multi-level classification data board is constructed, and the feature information and mutual relationship of the geographic elements are extracted from the multi-level classification data board to construct a knowledge graph;

[0049] The element features and mutual relationships of the geographic elements in the knowledge graph are converted into trainable feature vectors, and the artificial classification result is fused to construct the data set.

[0050] Specifically, first, according to the vector map with classification labels, the geographic elements of different classification labels can be divided into multiple levels according to different influencing factors and importance of site selection, a multi-level classification data board is constructed, and the element features and mutual relationships of various geographic elements are extracted from the constructed multi-level classification data board to construct the corresponding knowledge graph.

[0051] Specifically, first, based on the vector map with classification labels, a multi-level classification system is constructed according to the attributes and spatial relationships of geographic elements to form a structured data board. Then, the following contents are extracted from the board to construct a knowledge graph:

[0052] Among them, the element features include geometric properties (such as area, length), semantic properties (such as road grade, building height).

[0053] The mutual relationship includes spatial relationship (such as "road connecting house"), field rule (such as "oil well needs to avoid ecological red line").

[0054] The specific classification is shown in the following table:

[0055]

[0056] Then, the element features and mutual relationships of the geographic elements in the knowledge graph can be converted into trainable feature vectors, and the image features of each geographic element in the vector map are fused with the corresponding feature vectors to generate corresponding training samples to construct the data set and train the KAN network.

[0057] Specifically, the KAN network includes a feature extraction module, a knowledge graph embedding module, and a relationship reasoning module. The feature extraction module can use a convolutional neural network such as ResNet-50 to extract image features from the input image. The knowledge graph embedding module can map the feature vectors of the knowledge graph to the same dimension as the image features through a fully connected layer. The relationship reasoning module can combine the topological relationship of the knowledge graph through a graph attention network to perform joint reasoning and obtain the element category probability distribution of the key elements in the input image.

[0058] During the training process, the KAN network can be trained by using a forward propagation method, and a cross-entropy loss algorithm and a contrastive loss algorithm can be used to calculate the classification loss and the knowledge matching loss of the KAN network respectively, and then the total loss of the KAN network can be evaluated by comprehensively considering the two kinds of losses, and the Adam optimizer can be used for iterative optimization. After each round of training, the labels of the misclassified samples can be dynamically modified by using the relationship matrix of the knowledge graph, so as to iteratively update the training data in the data set.

[0059] The specific calculation formula of the total loss of the KAN network is as follows:

[0060] ;

[0061] wherein, , are the classification loss and the knowledge matching loss respectively, , are the weights corresponding to the classification loss and the knowledge matching loss respectively.

[0062] The specific calculation formula of the classification loss is as follows:

[0063] ;

[0064] wherein, is the true label, and is 0 or 1, indicating whether the sample belongs to the i-th element category, is the probability of the sample predicted by the KAN network to belong to the i-th element category. is the number of element categories.

[0065] The specific calculation formula of the knowledge matching loss is as follows:

[0066] ;

[0067] wherein, is the fusion feature obtained by fusing the image feature and the feature vector, is the positive sample feature, i.e., the feature belonging to the same category as the current sample. is the negative sample feature, i.e., the feature not belonging to the same category as the current sample, is a preset positive hyperparameter, used to control the minimum expected value of the similarity difference between the positive sample pair and the negative sample pair.

[0068] In this embodiment, when training the KAN network, the classification performance of the KAN network can be evaluated according to the overall classification accuracy, the precision, the recall and the F1 score of the KAN network.

[0069] In this embodiment, the KAN network can be used to intelligently identify the key elements, including:

[0070] extracting an image feature corresponding to the key element;

[0071] performing feature matching on the image feature and a feature vector of a geographic element in the knowledge graph to determine an element category corresponding to the key element.

[0072] Specifically, first, the convolutional neural network can perform feature extraction on the segmented image of the region to be built to obtain an image feature corresponding to each key element in the segmented image. Then, the feature matching module can match the image feature of each key element with the element feature of each geographic element in the knowledge graph constructed according to the multi-level classification data platform, and determine the element category of the key element based on the geographic element matched with the key element.

[0073] The KAN network combines a geographic entity knowledge graph (such as engineering specifications and ecological constraints), corrects segmentation errors through feature matching and relationship reasoning, and improves the element classification accuracy by more than 40%. Based on confidence evaluation (entropy quantification), the output results support manual verification in high-risk areas, and the model is iteratively optimized based on historical data to adapt to seasonal changes (such as river flow fluctuations).

[0074] In this embodiment, the feature matching of the image feature and the feature vector of the geographic element in the knowledge graph comprises:

[0075] calculating the similarity between the image feature and the element feature of the feature vector:

[0076] Based on the similarity between the image feature and the element feature, and the mutual relationship between the geographic elements in the feature vector, the element category corresponding to the key element is obtained.

[0077] Specifically, when matching the geographic element corresponding to the key element, first, the similarity between the key element and the geographic element in the knowledge graph can be calculated according to the image feature, and the specific calculation formula is as follows:

[0078] ;

[0079] wherein, is the image feature of the key element, is the element feature.

[0080] Then, based on the similarity between the key element and the geographic element, the reasoning function corresponding to the mutual relationship of the geographic elements in the knowledge graph can be used for reasoning, so as to obtain the final element classification result of the key element. The specific calculation formula is as follows:

[0081] ;

[0082] wherein, the number of categories of geographic elements in the knowledge graph, a reasoning function corresponding to the mutual relationship, specifically:

[0083]

[0084] wherein, a relationship matrix, a relationship bias term.

[0085] In combination with the mutual relationship of the geographic elements in the knowledge graph, the element category of the key element can be reasoned, which can correct the misclassification of the SAM model and improve the accuracy of the key element classification of the KAN network.

[0086] In the embodiment, the KAN network can be used to intelligently identify the key elements, including: measuring the confidence of the KAN network identification result by the probability distribution entropy.

[0087] Specifically, the KAN network can not only identify the element category of the key element in the segmented image, but also calculate the probability distribution entropy of the identification result, measure the confidence of the identification result by the probability distribution entropy, and assist in manually checking the high-risk area. The specific calculation formula of the probability distribution entropy is as follows:

[0088]

[0089] wherein, the entropy value of the identification result, the specific calculation formula is as follows:

[0090]

[0091] wherein, the predicted probability of the model to the i-th category,

[0092] In the embodiment, the geographic information model of the to-be-built area can be constructed according to the identification result of the key element, including:

[0093] According to the identification result of the key element, the ArcGIS Pro software is used to construct a three-dimensional element scene of the to-be-built area.

[0094] Specifically, after identifying the element category of each key element in the aerial image of the to-be-built area, the aerial image and the element identification result of each key element can be input into the ArcGIS Pro software, and the ArcGIS Pro software is used to construct a visual three-dimensional element scene of the to-be-built area to assist in site selection decision-making.

[0095] ​​​​In this embodiment, optionally, a geographic information model of the to-be-built area is constructed according to the identification result of the key elements, including:

[0096] A high-precision digital baseboard is constructed based on the identification result of the key elements.

[0097] Specifically, a block raster image and a vector layer are constructed based on the identification result of the key elements, seamless splicing and multi-level geographic entity scene construction are realized through an edge fusion algorithm and a three-dimensional dynamic modeling technology. Based on an ArcGIS Pro platform, a "coordinate correction-block fusion-topological optimization" process is adopted to eliminate joint errors, and the classified data is further processed. According to different influencing factors and importance of site selection, multiple levels are divided to form a high-precision digital baseboard supporting site selection decision.

[0098] Specifically, first, all block raster images are converted to WGS84 Web Mercator coordinate system using the "projected raster" tool in the ArcGIS Pro platform, and the vector layer is corrected synchronously through the "projection" tool. Then, the "mosaic to new raster" tool is called, all sub-block images are input, the output pixel type, the number of bands are set, the Blend fusion algorithm is selected, and the 20% feathering weight is used for smooth transition in the overlapping area. The "focus statistics" tool is executed on the spliced image, a 3*3 pixel neighborhood range is defined, and the center pixel value is calculated by weighted average (the weight coefficient is set to center 0.5 and edge 0.25), and the block boundary sawtooth is eliminated. Then, all block vector data is integrated using the "merge" tool to generate a complete area vector layer. Then, the unmanned aerial vehicle DSM point cloud data is loaded, a continuous terrain surface is generated through the "create TIN" tool, and the contour line vector is superimposed to correct the elevation details. Then, based on the building model attribute field "layer height", the "feature to 3D" tool is called to generate a block model by vertical stretching. For the pipeline model, the burial depth parameter is manually set in the "edit" mode, and the engineering constraints in the knowledge graph (such as burial depth ≥1.2m, avoiding ecological red line) are associated.

[0099] In this embodiment, optionally, a geographic information model of the to-be-built area is constructed according to the identification result of the key elements, including:

[0100] According to the identification result of the key elements in the to-be-built area and the three-dimensional element scene, a CAD file corresponding to each key element is generated, which includes all attribute information of the key element.

[0101] Specifically, after the visual three-dimensional element scene of the to-be-built area is constructed, the CAD file of the key elements such as roads and pipelines can be automatically exported according to the classification labels of the key elements in the scene and the three-dimensional scene data. The CAD file can include the coordinate, material, engineering constraint and other attribute information of the key elements, so as to be directly connected to the engineering design software for design in the subsequent stage, thereby reducing the manual conversion error.

[0102] The pixel-level segmentation is realized through the SAM model, high-precision input is provided for subsequent classification, and recognition deviation caused by segmentation error in traditional methods is avoided. Then, the segmentation result is combined with the domain knowledge graph through the KAN model, the mis-segmentation is corrected through relationship reasoning, and the classification accuracy and engineering adaptability are improved. Finally, the segmentation and recognition results are directly associated with the three-dimensional geological model, the collaborative analysis of the surface and underground conditions is realized, and the site selection risk is reduced.

[0103] A computer program product includes computer programs / instructions, which, when executed by a processor, implement the steps of the above-mentioned geographic information model construction method.

[0104] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the specification of the present application.

Claims

1. A method for constructing a geographic information model based on aerial photography data intelligent segmentation and recognition, characterized in that, The application relates to a geographic information model construction method. The SAM model is used to segment the ground features of each aerial image obtained; The key elements segmented are manually classified, and a data set is constructed according to the classification results; The KAN network based on the knowledge graph is trained through the data set; The SAM model is used to segment the aerial image of the region to be built, and the key elements segmented are intelligently identified through the trained KAN network; A geographic information model of the region to be built is constructed according to the identification results of the key elements; The data set is constructed according to the classification results, including: A multi-level classification data baseboard is constructed according to the artificial classification results, and the feature information and mutual relationship of the geographic elements are extracted from the multi-level classification data baseboard to construct a knowledge graph; The element features and mutual relationship of the geographic elements in the knowledge graph are converted into trainable feature vectors, and the data set is constructed by fusing the artificial classification results; In the training process, the specific calculation formula of the total loss of the KAN network is as follows: ； wherein, , are a classification loss and a knowledge matching loss, respectively, , are weights corresponding to the classification loss and the knowledge matching loss, respectively. The specific calculation formula of the classification loss is as follows: ； wherein, is true label, 0 or 1, indicating whether the sample belongs to the element category of the class, probability that the sample predicted by the KAN network belongs to the element category of the element category number; The specific calculation formula of the knowledge matching loss is as follows: ； wherein, is a fusion feature fused by the image feature and the feature vector, is a positive sample feature, is a negative sample feature, is a preset positive hyperparameter for controlling a minimum expected value of a similarity difference between the positive sample pair and the negative sample pair.

2. The geographic information model building method of claim 1, wherein, Intelligently identifying the key elements through the KAN network includes: Extracting the image features corresponding to the key elements; Performing feature matching on the image features and the feature vectors of the geographic elements in the knowledge graph to determine the element categories corresponding to the key elements.

3. The geographic information model building method of claim 2, wherein, Performing feature matching on the image features and the feature vectors of the geographic elements in the knowledge graph includes: Calculating the similarity between the image features and the element features of the feature vectors: Based on the similarity between the image features and the element features, the mutual relationship between the geographic elements in the feature vectors is combined to infer the element categories corresponding to the key elements.

4. The geographic information model building method of claim 1, wherein, Intelligently identifying the key elements through the KAN network includes: Measuring the confidence of the KAN network identification result through the probability distribution entropy.

5. The geographic information model building method of claim 1, wherein, Constructing a geographic information model of the region to be built according to the identification results of the key elements includes: According to the identification results of the key elements, an ArcGIS Pro software is used to construct a three-dimensional element scene of the region to be built.

6. The geographic information model building method of claim 5, wherein, Constructing a geographic information model of the region to be built according to the identification results of the key elements includes: Based on the identification results of the key elements, a high-precision digital baseboard is constructed.

7. The geographic information model building method of claim 5, wherein, Constructing a geographic information model of the region to be built according to the identification results of the key elements includes: According to the identification results of the key elements and the three-dimensional element scene in the region to be built, a CAD file corresponding to each key element is generated, and the CAD file includes all attribute information of the key elements.

8. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to realize the steps of the geographic information model construction method in any one of claims 1-7.

Images